org.apache.drill.exec.memory

Class BaseAllocator

java.lang.Object

org.apache.drill.exec.memory.Accountant

org.apache.drill.exec.memory.BaseAllocator

All Implemented Interfaces:

AutoCloseable, BufferAllocator

Direct Known Subclasses:

RootAllocator

public abstract class BaseAllocator
extends Accountant
implements BufferAllocator

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
class	BaseAllocator.Reservation
static class	BaseAllocator.Verbosity

Nested classes/interfaces inherited from class org.apache.drill.exec.memory.Accountant

Accountant.AllocationOutcome

Field Summary

Fields

Modifier and Type	Field and Description
static boolean	DEBUG
static String	DEBUG_ALLOCATOR
static int	DEBUG_LOG_LENGTH

Fields inherited from class org.apache.drill.exec.memory.Accountant

ALLOW_LENIENCY, ALLOW_LENIENT_ALLOCATION, GRACE_MARGIN, MAX_GRACE, parent, reservation

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	BaseAllocator(BaseAllocator parentAllocator, String name, long initReservation, long maxAllocation)

Method Summary

Modifier and Type	Method and Description
void	assertOpen() Asserts (using java assertions) that the provided allocator is currently open.
DrillBuf	buffer(int initialRequestSize) Allocate a new or reused buffer of the provided size.
DrillBuf	buffer(int initialRequestSize, BufferManager manager) Allocate a new or reused buffer of the provided size.
void	close() Close this Accountant.
io.netty.buffer.ByteBufAllocator	getAsByteBufAllocator() Returns the allocator this allocator falls back to when it needs more memory.
DrillBuf	getEmpty() Get a reference to the empty buffer associated with this allocator.
byte[]	getIOBuffer()
String	getName() Return the name of this allocator.
static StringBuilder	indent(StringBuilder sb, int indent)
static boolean	isDebug()
static long	longNextPowerOfTwo(long val) Rounds up the provided value to the nearest power of two.
BufferAllocator	newChildAllocator(String name, long initReservation, long maxAllocation) Create a new child allocator.
AllocationReservation	newReservation() Create an allocation reservation.
static int	nextPowerOfTwo(int val) Rounds up the provided value to the nearest power of two.
void	read(DrillBuf buf, int length, InputStream in) Read the contents of a DrillBuf from a stream.
DrillBuf	read(int length, InputStream in) Reads the specified number of bytes into a new Drillbuf.
String	toString()
String	toVerboseString() Provide a verbose string of the current allocator state.
void	write(DrillBuf buf, int length, OutputStream out) Write the contents of a DrillBuf to a stream.
void	write(DrillBuf buf, OutputStream out) Write the contents of a DrillBuf to a stream.

Methods inherited from class org.apache.drill.exec.memory.Accountant

forceLenient, getAllocatedMemory, getLimit, getPeakMemoryAllocation, isOverLimit, releaseBytes, setLenient, setLimit

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.apache.drill.exec.memory.BufferAllocator

getAllocatedMemory, getLimit, getPeakMemoryAllocation, isOverLimit, setLenient, setLimit

Field Detail

DEBUG_ALLOCATOR

public static final String DEBUG_ALLOCATOR

See Also:

Constant Field Values

DEBUG_LOG_LENGTH

public static final int DEBUG_LOG_LENGTH

See Also:

Constant Field Values

DEBUG

public static final boolean DEBUG

Constructor Detail

BaseAllocator

protected BaseAllocator(BaseAllocator parentAllocator,
                        String name,
                        long initReservation,
                        long maxAllocation)
                 throws OutOfMemoryException

Throws:

OutOfMemoryException

Method Detail

assertOpen

public void assertOpen()

Description copied from interface: BufferAllocator

Asserts (using java assertions) that the provided allocator is currently open. If assertions are disabled, this is a no-op.

Specified by:

assertOpen in interface BufferAllocator

getName

public String getName()

Description copied from interface: BufferAllocator

Return the name of this allocator. This is a human readable name that can help debugging. Typically provides coordinates about where this allocator was created

Specified by:

getName in interface BufferAllocator

getEmpty

public DrillBuf getEmpty()

Description copied from interface: BufferAllocator

Get a reference to the empty buffer associated with this allocator. Empty buffers are special because we don't worry about them leaking or managing reference counts on them since they don't actually point to any memory.

Specified by:

getEmpty in interface BufferAllocator

buffer

public DrillBuf buffer(int initialRequestSize)

Description copied from interface: BufferAllocator

Allocate a new or reused buffer of the provided size. Note that the buffer may technically be larger than the requested size for rounding purposes. However, the buffer's capacity will be set to the configured size.

Specified by:

buffer in interface BufferAllocator

Parameters:

initialRequestSize - The size in bytes.

Returns:

a new DrillBuf, or null if the request can't be satisfied

buffer

public DrillBuf buffer(int initialRequestSize,
                       BufferManager manager)

Description copied from interface: BufferAllocator

Allocate a new or reused buffer of the provided size. Note that the buffer may technically be larger than the requested size for rounding purposes. However, the buffer's capacity will be set to the configured size.

Specified by:

buffer in interface BufferAllocator

Parameters:

initialRequestSize - The size in bytes.

manager - A buffer manager to manage reallocation.

Returns:

a new DrillBuf, or null if the request can't be satisfied

getAsByteBufAllocator

public io.netty.buffer.ByteBufAllocator getAsByteBufAllocator()

Description copied from interface: BufferAllocator

Returns the allocator this allocator falls back to when it needs more memory.

Specified by:

getAsByteBufAllocator in interface BufferAllocator

Returns:

the underlying allocator used by this allocator

newChildAllocator

public BufferAllocator newChildAllocator(String name,
                                         long initReservation,
                                         long maxAllocation)

Description copied from interface: BufferAllocator

Create a new child allocator.

Specified by:

newChildAllocator in interface BufferAllocator

Parameters:

name - the name of the allocator.

initReservation - the initial space reservation (obtained from this allocator)

maxAllocation - maximum amount of space the new allocator can allocate

Returns:

the new allocator, or null if it can't be created

newReservation

public AllocationReservation newReservation()

Description copied from interface: BufferAllocator

Create an allocation reservation. A reservation is a way of building up a request for a buffer whose size is not known in advance. See AllocationReservation.

Specified by:

newReservation in interface BufferAllocator

Returns:

the newly created reservation

close

public void close()

Description copied from class: Accountant

Close this Accountant. This will release any reservation bytes back to a parent Accountant.

Specified by:

close in interface AutoCloseable

Specified by:

close in interface BufferAllocator

Overrides:

close in class Accountant

toString

public String toString()

Overrides:

toString in class Object

toVerboseString

public String toVerboseString()

Provide a verbose string of the current allocator state. Includes the state of all child allocators, along with historical logs of each object and including stacktraces.

Specified by:

toVerboseString in interface BufferAllocator

Returns:

A Verbose string of current allocator state.

nextPowerOfTwo

public static int nextPowerOfTwo(int val)

Rounds up the provided value to the nearest power of two.

Parameters:

val - An integer value.

Returns:

The closest power of two of that value.

longNextPowerOfTwo

public static long longNextPowerOfTwo(long val)

Rounds up the provided value to the nearest power of two.

Parameters:

val - An integer long value.

Returns:

The closest power of two of that value.

indent

public static StringBuilder indent(StringBuilder sb,
                                   int indent)

isDebug

public static boolean isDebug()

getIOBuffer

public byte[] getIOBuffer()

read

public void read(DrillBuf buf,
                 int length,
                 InputStream in)
          throws IOException

Description copied from interface: BufferAllocator

Read the contents of a DrillBuf from a stream. Use this method, rather than calling the DrillBuf.writeBytes() method, because this method avoids repeated heap allocation for the intermediate heap buffer. The buffer must have already been allocated.

Specified by:

read in interface BufferAllocator

Parameters:

buf - the buffer to read with space already allocated

length - number of bytes to read

in - input stream from which to read data

Throws:

IOException - if a read error occurs

read

public DrillBuf read(int length,
                     InputStream in)
              throws IOException

Description copied from interface: BufferAllocator

Reads the specified number of bytes into a new Drillbuf.

Specified by:

read in interface BufferAllocator

Parameters:

length - number of bytes to read

in - input stream from which to read data

Returns:

the buffer holding the data read from the stream

Throws:

IOException - if a read error occurs

write

public void write(DrillBuf buf,
                  OutputStream out)
           throws IOException

Description copied from interface: BufferAllocator

Write the contents of a DrillBuf to a stream. Use this method, rather than calling the DrillBuf.getBytes() method, because this method avoids repeated heap allocation for the intermediate heap buffer. Uses the reader and writer indexes to determine the number of bytes to write. Useful only for bufs created using those indexes.

Specified by:

write in interface BufferAllocator

Parameters:

buf - the Drillbuf to write

out - the output stream

Throws:

IOException - if a write error occurs

write

public void write(DrillBuf buf,
                  int length,
                  OutputStream out)
           throws IOException

Description copied from interface: BufferAllocator

Write the contents of a DrillBuf to a stream. Use this method, rather than calling the DrillBuf.getBytes() method, because this method avoids repeated heap allocation for the intermediate heap buffer. Writes the specified number of bytes starting from the head of the given Drillbuf.

Specified by:

write in interface BufferAllocator

Parameters:

buf - the Drillbuf to write

length - the number of bytes to read. Must be less than or equal to number of bytes allocated in the buffer.

out - the output stream

Throws:

IOException - if a write error occurs