Package org.apache.drill.exec.store.dfs
Class DrillFileSystem
java.lang.Object
org.apache.hadoop.conf.Configured
org.apache.hadoop.fs.FileSystem
org.apache.drill.exec.store.dfs.DrillFileSystem
- All Implemented Interfaces:
Closeable
,AutoCloseable
,OpenFileTracker
,org.apache.hadoop.conf.Configurable
,org.apache.hadoop.fs.PathCapabilities
,org.apache.hadoop.security.token.DelegationTokenIssuer
DrillFileSystem is the wrapper around the actual FileSystem implementation. The
DrillFileSystem
is
immutable.
If OperatorStats
are provided it returns an instrumented FSDataInputStream to
measure IO wait time and tracking file open/close operations.-
Nested Class Summary
Nested classes/interfaces inherited from class org.apache.hadoop.fs.FileSystem
org.apache.hadoop.fs.FileSystem.DirectoryEntries, org.apache.hadoop.fs.FileSystem.DirListingIterator<T extends org.apache.hadoop.fs.FileStatus>, org.apache.hadoop.fs.FileSystem.Statistics
-
Field Summary
Fields inherited from class org.apache.hadoop.fs.FileSystem
DEFAULT_FS, FS_DEFAULT_NAME_KEY, LOG, SHUTDOWN_HOOK_PRIORITY, statistics, TRASH_PREFIX, USER_HOME_PREFIX
-
Constructor Summary
ConstructorDescriptionDrillFileSystem
(org.apache.hadoop.conf.Configuration fsConf) DrillFileSystem
(org.apache.hadoop.conf.Configuration fsConf, OperatorStats operatorStats) -
Method Summary
Modifier and TypeMethodDescriptionvoid
access
(org.apache.hadoop.fs.Path path, org.apache.hadoop.fs.permission.FsAction mode) org.apache.hadoop.security.token.Token<?>[]
addDelegationTokens
(String renewer, org.apache.hadoop.security.Credentials credentials) org.apache.hadoop.fs.FSDataOutputStream
append
(org.apache.hadoop.fs.Path f) org.apache.hadoop.fs.FSDataOutputStream
append
(org.apache.hadoop.fs.Path f, int bufferSize) org.apache.hadoop.fs.FSDataOutputStream
append
(org.apache.hadoop.fs.Path f, int bufferSize, org.apache.hadoop.util.Progressable progress) boolean
cancelDeleteOnExit
(org.apache.hadoop.fs.Path f) void
close()
void
completeLocalOutput
(org.apache.hadoop.fs.Path fsOutputFile, org.apache.hadoop.fs.Path tmpLocalFile) void
concat
(org.apache.hadoop.fs.Path trg, org.apache.hadoop.fs.Path[] psrcs) void
copyFromLocalFile
(boolean delSrc, boolean overwrite, org.apache.hadoop.fs.Path[] srcs, org.apache.hadoop.fs.Path dst) void
copyFromLocalFile
(boolean delSrc, boolean overwrite, org.apache.hadoop.fs.Path src, org.apache.hadoop.fs.Path dst) void
copyFromLocalFile
(boolean delSrc, org.apache.hadoop.fs.Path src, org.apache.hadoop.fs.Path dst) void
copyFromLocalFile
(org.apache.hadoop.fs.Path src, org.apache.hadoop.fs.Path dst) void
copyToLocalFile
(boolean delSrc, org.apache.hadoop.fs.Path src, org.apache.hadoop.fs.Path dst) void
copyToLocalFile
(boolean delSrc, org.apache.hadoop.fs.Path src, org.apache.hadoop.fs.Path dst, boolean useRawLocalFileSystem) void
copyToLocalFile
(org.apache.hadoop.fs.Path src, org.apache.hadoop.fs.Path dst) org.apache.hadoop.fs.FSDataOutputStream
create
(org.apache.hadoop.fs.Path f) org.apache.hadoop.fs.FSDataOutputStream
create
(org.apache.hadoop.fs.Path f, boolean overwrite) org.apache.hadoop.fs.FSDataOutputStream
create
(org.apache.hadoop.fs.Path f, boolean overwrite, int bufferSize) org.apache.hadoop.fs.FSDataOutputStream
create
(org.apache.hadoop.fs.Path f, boolean overwrite, int bufferSize, short replication, long blockSize) org.apache.hadoop.fs.FSDataOutputStream
create
(org.apache.hadoop.fs.Path f, boolean overwrite, int bufferSize, short replication, long blockSize, org.apache.hadoop.util.Progressable progress) org.apache.hadoop.fs.FSDataOutputStream
create
(org.apache.hadoop.fs.Path f, boolean overwrite, int bufferSize, org.apache.hadoop.util.Progressable progress) org.apache.hadoop.fs.FSDataOutputStream
create
(org.apache.hadoop.fs.Path f, short replication) org.apache.hadoop.fs.FSDataOutputStream
create
(org.apache.hadoop.fs.Path f, short replication, org.apache.hadoop.util.Progressable progress) org.apache.hadoop.fs.FSDataOutputStream
create
(org.apache.hadoop.fs.Path f, org.apache.hadoop.fs.permission.FsPermission permission, boolean overwrite, int bufferSize, short replication, long blockSize, org.apache.hadoop.util.Progressable progress) org.apache.hadoop.fs.FSDataOutputStream
create
(org.apache.hadoop.fs.Path f, org.apache.hadoop.fs.permission.FsPermission permission, EnumSet<org.apache.hadoop.fs.CreateFlag> flags, int bufferSize, short replication, long blockSize, org.apache.hadoop.util.Progressable progress) org.apache.hadoop.fs.FSDataOutputStream
create
(org.apache.hadoop.fs.Path f, org.apache.hadoop.fs.permission.FsPermission permission, EnumSet<org.apache.hadoop.fs.CreateFlag> flags, int bufferSize, short replication, long blockSize, org.apache.hadoop.util.Progressable progress, org.apache.hadoop.fs.Options.ChecksumOpt checksumOpt) org.apache.hadoop.fs.FSDataOutputStream
create
(org.apache.hadoop.fs.Path f, org.apache.hadoop.util.Progressable progress) boolean
createNewFile
(org.apache.hadoop.fs.Path f) org.apache.hadoop.fs.FSDataOutputStream
createNonRecursive
(org.apache.hadoop.fs.Path f, boolean overwrite, int bufferSize, short replication, long blockSize, org.apache.hadoop.util.Progressable progress) Deprecated.org.apache.hadoop.fs.FSDataOutputStream
createNonRecursive
(org.apache.hadoop.fs.Path f, org.apache.hadoop.fs.permission.FsPermission permission, boolean overwrite, int bufferSize, short replication, long blockSize, org.apache.hadoop.util.Progressable progress) Deprecated.org.apache.hadoop.fs.FSDataOutputStream
createNonRecursive
(org.apache.hadoop.fs.Path f, org.apache.hadoop.fs.permission.FsPermission permission, EnumSet<org.apache.hadoop.fs.CreateFlag> flags, int bufferSize, short replication, long blockSize, org.apache.hadoop.util.Progressable progress) Deprecated.org.apache.hadoop.fs.Path
createSnapshot
(org.apache.hadoop.fs.Path path, String snapshotName) void
createSymlink
(org.apache.hadoop.fs.Path target, org.apache.hadoop.fs.Path link, boolean createParent) boolean
delete
(org.apache.hadoop.fs.Path f) Deprecated.boolean
delete
(org.apache.hadoop.fs.Path f, boolean recursive) boolean
deleteOnExit
(org.apache.hadoop.fs.Path f) void
deleteSnapshot
(org.apache.hadoop.fs.Path path, String snapshotName) boolean
exists
(org.apache.hadoop.fs.Path f) void
fileClosed
(DrillFSDataInputStream fsDataInputStream) Remove the givenDrillFSDataInputStream
from opened file list.void
fileOpened
(org.apache.hadoop.fs.Path path, DrillFSDataInputStream fsDataInputStream) Add new file location andDrillFSDataInputStream
to list.org.apache.hadoop.fs.permission.AclStatus
getAclStatus
(org.apache.hadoop.fs.Path path) long
getBlockSize
(org.apache.hadoop.fs.Path f) Deprecated.org.apache.hadoop.fs.FileSystem[]
org.apache.hadoop.io.compress.CompressionCodec
getCodec
(org.apache.hadoop.fs.Path path) Returns theCompressionCodec
for a given file.org.apache.hadoop.conf.Configuration
getConf()
Returns a copy ofConfiguration
for thisDrillFileSystem
.org.apache.hadoop.fs.ContentSummary
getContentSummary
(org.apache.hadoop.fs.Path f) long
Deprecated.long
getDefaultBlockSize
(org.apache.hadoop.fs.Path f) short
Deprecated.short
getDefaultReplication
(org.apache.hadoop.fs.Path path) org.apache.hadoop.security.token.Token<?>
getDelegationToken
(String renewer) org.apache.hadoop.fs.BlockLocation[]
getFileBlockLocations
(org.apache.hadoop.fs.FileStatus file, long start, long len) org.apache.hadoop.fs.BlockLocation[]
getFileBlockLocations
(org.apache.hadoop.fs.Path p, long start, long len) org.apache.hadoop.fs.FileChecksum
getFileChecksum
(org.apache.hadoop.fs.Path f) org.apache.hadoop.fs.FileChecksum
getFileChecksum
(org.apache.hadoop.fs.Path f, long length) org.apache.hadoop.fs.FileStatus
getFileLinkStatus
(org.apache.hadoop.fs.Path f) org.apache.hadoop.fs.FileStatus
getFileStatus
(org.apache.hadoop.fs.Path f) org.apache.hadoop.fs.Path
long
getLength
(org.apache.hadoop.fs.Path f) Deprecated.org.apache.hadoop.fs.Path
getLinkTarget
(org.apache.hadoop.fs.Path f) getName()
Deprecated.short
getReplication
(org.apache.hadoop.fs.Path src) Deprecated.org.apache.hadoop.fs.FsServerDefaults
Deprecated.org.apache.hadoop.fs.FsServerDefaults
getServerDefaults
(org.apache.hadoop.fs.Path p) org.apache.hadoop.fs.FsStatus
org.apache.hadoop.fs.FsStatus
getStatus
(org.apache.hadoop.fs.Path p) org.apache.hadoop.fs.FileSystem
Returns a copy of the underlying file system.getUri()
long
getUsed()
org.apache.hadoop.fs.Path
byte[]
getXAttrs
(org.apache.hadoop.fs.Path path) org.apache.hadoop.fs.FileStatus[]
globStatus
(org.apache.hadoop.fs.Path pathPattern) org.apache.hadoop.fs.FileStatus[]
globStatus
(org.apache.hadoop.fs.Path pathPattern, org.apache.hadoop.fs.PathFilter filter) void
initialize
(URI name, org.apache.hadoop.conf.Configuration conf) This method should never be used onDrillFileSystem
sinceDrillFileSystem
is immutable.boolean
isDirectory
(org.apache.hadoop.fs.Path f) boolean
isFile
(org.apache.hadoop.fs.Path f) org.apache.hadoop.fs.RemoteIterator<org.apache.hadoop.fs.Path>
listCorruptFileBlocks
(org.apache.hadoop.fs.Path path) org.apache.hadoop.fs.RemoteIterator<org.apache.hadoop.fs.LocatedFileStatus>
listFiles
(org.apache.hadoop.fs.Path f, boolean recursive) org.apache.hadoop.fs.RemoteIterator<org.apache.hadoop.fs.LocatedFileStatus>
listLocatedStatus
(org.apache.hadoop.fs.Path f) org.apache.hadoop.fs.FileStatus[]
listStatus
(org.apache.hadoop.fs.Path f) org.apache.hadoop.fs.FileStatus[]
listStatus
(org.apache.hadoop.fs.Path[] files) org.apache.hadoop.fs.FileStatus[]
listStatus
(org.apache.hadoop.fs.Path[] files, org.apache.hadoop.fs.PathFilter filter) org.apache.hadoop.fs.FileStatus[]
listStatus
(org.apache.hadoop.fs.Path f, org.apache.hadoop.fs.PathFilter filter) org.apache.hadoop.fs.RemoteIterator<org.apache.hadoop.fs.FileStatus>
listStatusIterator
(org.apache.hadoop.fs.Path p) listXAttrs
(org.apache.hadoop.fs.Path path) org.apache.hadoop.fs.Path
makeQualified
(org.apache.hadoop.fs.Path path) boolean
mkdirs
(org.apache.hadoop.fs.Path folderPath) boolean
mkdirs
(org.apache.hadoop.fs.Path f, org.apache.hadoop.fs.permission.FsPermission permission) void
modifyAclEntries
(org.apache.hadoop.fs.Path path, List<org.apache.hadoop.fs.permission.AclEntry> aclSpec) void
moveFromLocalFile
(org.apache.hadoop.fs.Path[] srcs, org.apache.hadoop.fs.Path dst) void
moveFromLocalFile
(org.apache.hadoop.fs.Path src, org.apache.hadoop.fs.Path dst) void
moveToLocalFile
(org.apache.hadoop.fs.Path src, org.apache.hadoop.fs.Path dst) org.apache.hadoop.fs.FSDataInputStream
open
(org.apache.hadoop.fs.Path f) If OperatorStats are provided return a instrumentedFSDataInputStream
.org.apache.hadoop.fs.FSDataInputStream
open
(org.apache.hadoop.fs.Path f, int bufferSize) If OperatorStats are provided return a instrumentedFSDataInputStream
.openPossiblyCompressedStream
(org.apache.hadoop.fs.Path path) Returns an InputStream from a Hadoop path.void
removeAcl
(org.apache.hadoop.fs.Path path) void
removeAclEntries
(org.apache.hadoop.fs.Path path, List<org.apache.hadoop.fs.permission.AclEntry> aclSpec) void
removeDefaultAcl
(org.apache.hadoop.fs.Path path) void
removeXAttr
(org.apache.hadoop.fs.Path path, String name) boolean
rename
(org.apache.hadoop.fs.Path src, org.apache.hadoop.fs.Path dst) void
renameSnapshot
(org.apache.hadoop.fs.Path path, String snapshotOldName, String snapshotNewName) org.apache.hadoop.fs.Path
resolvePath
(org.apache.hadoop.fs.Path p) void
void
setConf
(org.apache.hadoop.conf.Configuration conf) This method should never be used onDrillFileSystem
sinceDrillFileSystem
is immutable.void
void
setPermission
(org.apache.hadoop.fs.Path p, org.apache.hadoop.fs.permission.FsPermission permission) boolean
setReplication
(org.apache.hadoop.fs.Path src, short replication) void
setTimes
(org.apache.hadoop.fs.Path p, long mtime, long atime) void
setVerifyChecksum
(boolean verifyChecksum) This method should never be used onDrillFileSystem
sinceDrillFileSystem
is immutable.void
setWorkingDirectory
(org.apache.hadoop.fs.Path new_dir) This method should never be used onDrillFileSystem
sinceDrillFileSystem
is immutable.void
setWriteChecksum
(boolean writeChecksum) This method should never be used onDrillFileSystem
sinceDrillFileSystem
is immutable.void
void
setXAttr
(org.apache.hadoop.fs.Path path, String name, byte[] value, EnumSet<org.apache.hadoop.fs.XAttrSetFlag> flag) org.apache.hadoop.fs.Path
startLocalOutput
(org.apache.hadoop.fs.Path fsOutputFile, org.apache.hadoop.fs.Path tmpLocalFile) boolean
boolean
truncate
(org.apache.hadoop.fs.Path f, long newLength) Methods inherited from class org.apache.hadoop.fs.FileSystem
appendFile, areSymlinksEnabled, canonicalizeUri, checkPath, clearStatistics, closeAll, closeAllForUGI, create, createFile, createPathHandle, createSnapshot, enableSymlinks, fixRelativePart, get, get, get, getAdditionalTokenIssuers, getAllStatistics, getAllStoragePolicies, getCanonicalUri, getDefaultPort, getDefaultUri, getFileSystemClass, getFSofPath, getGlobalStorageStatistics, getInitialWorkingDirectory, getLocal, getNamed, getPathHandle, getQuotaUsage, getStatistics, getStatistics, getStoragePolicy, getStorageStatistics, getTrashRoot, getTrashRoots, getUsed, hasPathCapability, listLocatedStatus, listStatusBatch, mkdirs, msync, newInstance, newInstance, newInstance, newInstanceLocal, open, open, primitiveCreate, primitiveMkdir, primitiveMkdir, printStatistics, processDeleteOnExit, rename, resolveLink, satisfyStoragePolicy, setDefaultUri, setDefaultUri, setQuota, setQuotaByStorageType, setStoragePolicy, unsetStoragePolicy
-
Field Details
-
UNDERSCORE_PREFIX
- See Also:
-
DOT_PREFIX
- See Also:
-
-
Constructor Details
-
DrillFileSystem
- Throws:
IOException
-
DrillFileSystem
public DrillFileSystem(org.apache.hadoop.conf.Configuration fsConf, OperatorStats operatorStats) throws IOException - Throws:
IOException
-
-
Method Details
-
setConf
public void setConf(org.apache.hadoop.conf.Configuration conf) This method should never be used onDrillFileSystem
sinceDrillFileSystem
is immutable.- Specified by:
setConf
in interfaceorg.apache.hadoop.conf.Configurable
- Overrides:
setConf
in classorg.apache.hadoop.conf.Configured
- Throws:
UnsupportedOperationException
- when called.
-
getConf
public org.apache.hadoop.conf.Configuration getConf()Returns a copy ofConfiguration
for thisDrillFileSystem
. Note: a copy of theConfiguration
is returned in order to enforce immutability.- Specified by:
getConf
in interfaceorg.apache.hadoop.conf.Configurable
- Overrides:
getConf
in classorg.apache.hadoop.conf.Configured
- Returns:
- A copy of
Configuration
for thisDrillFileSystem
.
-
getUnderlyingFs
public org.apache.hadoop.fs.FileSystem getUnderlyingFs()Returns a copy of the underlying file system. This class implementsFileSystem
- Returns:
- The underlying
FileSystem
-
open
public org.apache.hadoop.fs.FSDataInputStream open(org.apache.hadoop.fs.Path f, int bufferSize) throws IOException If OperatorStats are provided return a instrumentedFSDataInputStream
.- Specified by:
open
in classorg.apache.hadoop.fs.FileSystem
- Throws:
IOException
-
open
If OperatorStats are provided return a instrumentedFSDataInputStream
.- Overrides:
open
in classorg.apache.hadoop.fs.FileSystem
- Throws:
IOException
-
initialize
This method should never be used onDrillFileSystem
sinceDrillFileSystem
is immutable.- Overrides:
initialize
in classorg.apache.hadoop.fs.FileSystem
- Throws:
UnsupportedOperationException
- when called.
-
getScheme
- Overrides:
getScheme
in classorg.apache.hadoop.fs.FileSystem
-
create
public org.apache.hadoop.fs.FSDataOutputStream create(org.apache.hadoop.fs.Path f) throws IOException - Overrides:
create
in classorg.apache.hadoop.fs.FileSystem
- Throws:
IOException
-
create
public org.apache.hadoop.fs.FSDataOutputStream create(org.apache.hadoop.fs.Path f, boolean overwrite) throws IOException - Overrides:
create
in classorg.apache.hadoop.fs.FileSystem
- Throws:
IOException
-
create
public org.apache.hadoop.fs.FSDataOutputStream create(org.apache.hadoop.fs.Path f, org.apache.hadoop.util.Progressable progress) throws IOException - Overrides:
create
in classorg.apache.hadoop.fs.FileSystem
- Throws:
IOException
-
create
public org.apache.hadoop.fs.FSDataOutputStream create(org.apache.hadoop.fs.Path f, short replication) throws IOException - Overrides:
create
in classorg.apache.hadoop.fs.FileSystem
- Throws:
IOException
-
create
public org.apache.hadoop.fs.FSDataOutputStream create(org.apache.hadoop.fs.Path f, short replication, org.apache.hadoop.util.Progressable progress) throws IOException - Overrides:
create
in classorg.apache.hadoop.fs.FileSystem
- Throws:
IOException
-
create
public org.apache.hadoop.fs.FSDataOutputStream create(org.apache.hadoop.fs.Path f, boolean overwrite, int bufferSize) throws IOException - Overrides:
create
in classorg.apache.hadoop.fs.FileSystem
- Throws:
IOException
-
create
public org.apache.hadoop.fs.FSDataOutputStream create(org.apache.hadoop.fs.Path f, boolean overwrite, int bufferSize, org.apache.hadoop.util.Progressable progress) throws IOException - Overrides:
create
in classorg.apache.hadoop.fs.FileSystem
- Throws:
IOException
-
create
public org.apache.hadoop.fs.FSDataOutputStream create(org.apache.hadoop.fs.Path f, boolean overwrite, int bufferSize, short replication, long blockSize) throws IOException - Overrides:
create
in classorg.apache.hadoop.fs.FileSystem
- Throws:
IOException
-
create
public org.apache.hadoop.fs.FSDataOutputStream create(org.apache.hadoop.fs.Path f, boolean overwrite, int bufferSize, short replication, long blockSize, org.apache.hadoop.util.Progressable progress) throws IOException - Overrides:
create
in classorg.apache.hadoop.fs.FileSystem
- Throws:
IOException
-
getFileStatus
public org.apache.hadoop.fs.FileStatus getFileStatus(org.apache.hadoop.fs.Path f) throws IOException - Specified by:
getFileStatus
in classorg.apache.hadoop.fs.FileSystem
- Throws:
IOException
-
createSymlink
public void createSymlink(org.apache.hadoop.fs.Path target, org.apache.hadoop.fs.Path link, boolean createParent) throws IOException - Overrides:
createSymlink
in classorg.apache.hadoop.fs.FileSystem
- Throws:
IOException
-
getFileLinkStatus
public org.apache.hadoop.fs.FileStatus getFileLinkStatus(org.apache.hadoop.fs.Path f) throws IOException - Overrides:
getFileLinkStatus
in classorg.apache.hadoop.fs.FileSystem
- Throws:
IOException
-
supportsSymlinks
public boolean supportsSymlinks()- Overrides:
supportsSymlinks
in classorg.apache.hadoop.fs.FileSystem
-
getLinkTarget
- Overrides:
getLinkTarget
in classorg.apache.hadoop.fs.FileSystem
- Throws:
IOException
-
getFileChecksum
public org.apache.hadoop.fs.FileChecksum getFileChecksum(org.apache.hadoop.fs.Path f) throws IOException - Overrides:
getFileChecksum
in classorg.apache.hadoop.fs.FileSystem
- Throws:
IOException
-
setVerifyChecksum
public void setVerifyChecksum(boolean verifyChecksum) This method should never be used onDrillFileSystem
sinceDrillFileSystem
is immutable.- Overrides:
setVerifyChecksum
in classorg.apache.hadoop.fs.FileSystem
- Throws:
UnsupportedOperationException
- when called.
-
setWriteChecksum
public void setWriteChecksum(boolean writeChecksum) This method should never be used onDrillFileSystem
sinceDrillFileSystem
is immutable.- Overrides:
setWriteChecksum
in classorg.apache.hadoop.fs.FileSystem
- Throws:
UnsupportedOperationException
- when called.
-
getStatus
- Overrides:
getStatus
in classorg.apache.hadoop.fs.FileSystem
- Throws:
IOException
-
getStatus
- Overrides:
getStatus
in classorg.apache.hadoop.fs.FileSystem
- Throws:
IOException
-
setPermission
public void setPermission(org.apache.hadoop.fs.Path p, org.apache.hadoop.fs.permission.FsPermission permission) throws IOException - Overrides:
setPermission
in classorg.apache.hadoop.fs.FileSystem
- Throws:
IOException
-
setOwner
public void setOwner(org.apache.hadoop.fs.Path p, String username, String groupname) throws IOException - Overrides:
setOwner
in classorg.apache.hadoop.fs.FileSystem
- Throws:
IOException
-
setTimes
- Overrides:
setTimes
in classorg.apache.hadoop.fs.FileSystem
- Throws:
IOException
-
createSnapshot
public org.apache.hadoop.fs.Path createSnapshot(org.apache.hadoop.fs.Path path, String snapshotName) throws IOException - Overrides:
createSnapshot
in classorg.apache.hadoop.fs.FileSystem
- Throws:
IOException
-
renameSnapshot
public void renameSnapshot(org.apache.hadoop.fs.Path path, String snapshotOldName, String snapshotNewName) throws IOException - Overrides:
renameSnapshot
in classorg.apache.hadoop.fs.FileSystem
- Throws:
IOException
-
deleteSnapshot
- Overrides:
deleteSnapshot
in classorg.apache.hadoop.fs.FileSystem
- Throws:
IOException
-
modifyAclEntries
public void modifyAclEntries(org.apache.hadoop.fs.Path path, List<org.apache.hadoop.fs.permission.AclEntry> aclSpec) throws IOException - Overrides:
modifyAclEntries
in classorg.apache.hadoop.fs.FileSystem
- Throws:
IOException
-
removeAclEntries
public void removeAclEntries(org.apache.hadoop.fs.Path path, List<org.apache.hadoop.fs.permission.AclEntry> aclSpec) throws IOException - Overrides:
removeAclEntries
in classorg.apache.hadoop.fs.FileSystem
- Throws:
IOException
-
removeDefaultAcl
- Overrides:
removeDefaultAcl
in classorg.apache.hadoop.fs.FileSystem
- Throws:
IOException
-
removeAcl
- Overrides:
removeAcl
in classorg.apache.hadoop.fs.FileSystem
- Throws:
IOException
-
setAcl
public void setAcl(org.apache.hadoop.fs.Path path, List<org.apache.hadoop.fs.permission.AclEntry> aclSpec) throws IOException - Overrides:
setAcl
in classorg.apache.hadoop.fs.FileSystem
- Throws:
IOException
-
getAclStatus
public org.apache.hadoop.fs.permission.AclStatus getAclStatus(org.apache.hadoop.fs.Path path) throws IOException - Overrides:
getAclStatus
in classorg.apache.hadoop.fs.FileSystem
- Throws:
IOException
-
getWorkingDirectory
public org.apache.hadoop.fs.Path getWorkingDirectory()- Specified by:
getWorkingDirectory
in classorg.apache.hadoop.fs.FileSystem
-
append
public org.apache.hadoop.fs.FSDataOutputStream append(org.apache.hadoop.fs.Path f, int bufferSize, org.apache.hadoop.util.Progressable progress) throws IOException - Specified by:
append
in classorg.apache.hadoop.fs.FileSystem
- Throws:
IOException
-
concat
public void concat(org.apache.hadoop.fs.Path trg, org.apache.hadoop.fs.Path[] psrcs) throws IOException - Overrides:
concat
in classorg.apache.hadoop.fs.FileSystem
- Throws:
IOException
-
getReplication
Deprecated.- Overrides:
getReplication
in classorg.apache.hadoop.fs.FileSystem
- Throws:
IOException
-
setReplication
- Overrides:
setReplication
in classorg.apache.hadoop.fs.FileSystem
- Throws:
IOException
-
mkdirs
public boolean mkdirs(org.apache.hadoop.fs.Path f, org.apache.hadoop.fs.permission.FsPermission permission) throws IOException - Specified by:
mkdirs
in classorg.apache.hadoop.fs.FileSystem
- Throws:
IOException
-
copyFromLocalFile
public void copyFromLocalFile(org.apache.hadoop.fs.Path src, org.apache.hadoop.fs.Path dst) throws IOException - Overrides:
copyFromLocalFile
in classorg.apache.hadoop.fs.FileSystem
- Throws:
IOException
-
moveFromLocalFile
public void moveFromLocalFile(org.apache.hadoop.fs.Path[] srcs, org.apache.hadoop.fs.Path dst) throws IOException - Overrides:
moveFromLocalFile
in classorg.apache.hadoop.fs.FileSystem
- Throws:
IOException
-
moveFromLocalFile
public void moveFromLocalFile(org.apache.hadoop.fs.Path src, org.apache.hadoop.fs.Path dst) throws IOException - Overrides:
moveFromLocalFile
in classorg.apache.hadoop.fs.FileSystem
- Throws:
IOException
-
copyFromLocalFile
public void copyFromLocalFile(boolean delSrc, org.apache.hadoop.fs.Path src, org.apache.hadoop.fs.Path dst) throws IOException - Overrides:
copyFromLocalFile
in classorg.apache.hadoop.fs.FileSystem
- Throws:
IOException
-
copyFromLocalFile
public void copyFromLocalFile(boolean delSrc, boolean overwrite, org.apache.hadoop.fs.Path[] srcs, org.apache.hadoop.fs.Path dst) throws IOException - Overrides:
copyFromLocalFile
in classorg.apache.hadoop.fs.FileSystem
- Throws:
IOException
-
copyFromLocalFile
public void copyFromLocalFile(boolean delSrc, boolean overwrite, org.apache.hadoop.fs.Path src, org.apache.hadoop.fs.Path dst) throws IOException - Overrides:
copyFromLocalFile
in classorg.apache.hadoop.fs.FileSystem
- Throws:
IOException
-
copyToLocalFile
public void copyToLocalFile(org.apache.hadoop.fs.Path src, org.apache.hadoop.fs.Path dst) throws IOException - Overrides:
copyToLocalFile
in classorg.apache.hadoop.fs.FileSystem
- Throws:
IOException
-
moveToLocalFile
public void moveToLocalFile(org.apache.hadoop.fs.Path src, org.apache.hadoop.fs.Path dst) throws IOException - Overrides:
moveToLocalFile
in classorg.apache.hadoop.fs.FileSystem
- Throws:
IOException
-
copyToLocalFile
public void copyToLocalFile(boolean delSrc, org.apache.hadoop.fs.Path src, org.apache.hadoop.fs.Path dst) throws IOException - Overrides:
copyToLocalFile
in classorg.apache.hadoop.fs.FileSystem
- Throws:
IOException
-
copyToLocalFile
public void copyToLocalFile(boolean delSrc, org.apache.hadoop.fs.Path src, org.apache.hadoop.fs.Path dst, boolean useRawLocalFileSystem) throws IOException - Overrides:
copyToLocalFile
in classorg.apache.hadoop.fs.FileSystem
- Throws:
IOException
-
startLocalOutput
public org.apache.hadoop.fs.Path startLocalOutput(org.apache.hadoop.fs.Path fsOutputFile, org.apache.hadoop.fs.Path tmpLocalFile) throws IOException - Overrides:
startLocalOutput
in classorg.apache.hadoop.fs.FileSystem
- Throws:
IOException
-
completeLocalOutput
public void completeLocalOutput(org.apache.hadoop.fs.Path fsOutputFile, org.apache.hadoop.fs.Path tmpLocalFile) throws IOException - Overrides:
completeLocalOutput
in classorg.apache.hadoop.fs.FileSystem
- Throws:
IOException
-
close
- Specified by:
close
in interfaceAutoCloseable
- Specified by:
close
in interfaceCloseable
- Overrides:
close
in classorg.apache.hadoop.fs.FileSystem
- Throws:
IOException
-
getUsed
- Overrides:
getUsed
in classorg.apache.hadoop.fs.FileSystem
- Throws:
IOException
-
getBlockSize
Deprecated.- Overrides:
getBlockSize
in classorg.apache.hadoop.fs.FileSystem
- Throws:
IOException
-
getDefaultBlockSize
Deprecated.- Overrides:
getDefaultBlockSize
in classorg.apache.hadoop.fs.FileSystem
-
getDefaultBlockSize
public long getDefaultBlockSize(org.apache.hadoop.fs.Path f) - Overrides:
getDefaultBlockSize
in classorg.apache.hadoop.fs.FileSystem
-
getDefaultReplication
Deprecated.- Overrides:
getDefaultReplication
in classorg.apache.hadoop.fs.FileSystem
-
getDefaultReplication
public short getDefaultReplication(org.apache.hadoop.fs.Path path) - Overrides:
getDefaultReplication
in classorg.apache.hadoop.fs.FileSystem
-
mkdirs
- Overrides:
mkdirs
in classorg.apache.hadoop.fs.FileSystem
- Throws:
IOException
-
create
public org.apache.hadoop.fs.FSDataOutputStream create(org.apache.hadoop.fs.Path f, org.apache.hadoop.fs.permission.FsPermission permission, EnumSet<org.apache.hadoop.fs.CreateFlag> flags, int bufferSize, short replication, long blockSize, org.apache.hadoop.util.Progressable progress, org.apache.hadoop.fs.Options.ChecksumOpt checksumOpt) throws IOException - Overrides:
create
in classorg.apache.hadoop.fs.FileSystem
- Throws:
IOException
-
createNonRecursive
@Deprecated public org.apache.hadoop.fs.FSDataOutputStream createNonRecursive(org.apache.hadoop.fs.Path f, boolean overwrite, int bufferSize, short replication, long blockSize, org.apache.hadoop.util.Progressable progress) throws IOException Deprecated.- Overrides:
createNonRecursive
in classorg.apache.hadoop.fs.FileSystem
- Throws:
IOException
-
createNonRecursive
@Deprecated public org.apache.hadoop.fs.FSDataOutputStream createNonRecursive(org.apache.hadoop.fs.Path f, org.apache.hadoop.fs.permission.FsPermission permission, boolean overwrite, int bufferSize, short replication, long blockSize, org.apache.hadoop.util.Progressable progress) throws IOException Deprecated.- Overrides:
createNonRecursive
in classorg.apache.hadoop.fs.FileSystem
- Throws:
IOException
-
createNonRecursive
@Deprecated public org.apache.hadoop.fs.FSDataOutputStream createNonRecursive(org.apache.hadoop.fs.Path f, org.apache.hadoop.fs.permission.FsPermission permission, EnumSet<org.apache.hadoop.fs.CreateFlag> flags, int bufferSize, short replication, long blockSize, org.apache.hadoop.util.Progressable progress) throws IOException Deprecated.- Overrides:
createNonRecursive
in classorg.apache.hadoop.fs.FileSystem
- Throws:
IOException
-
createNewFile
- Overrides:
createNewFile
in classorg.apache.hadoop.fs.FileSystem
- Throws:
IOException
-
append
public org.apache.hadoop.fs.FSDataOutputStream append(org.apache.hadoop.fs.Path f) throws IOException - Overrides:
append
in classorg.apache.hadoop.fs.FileSystem
- Throws:
IOException
-
append
public org.apache.hadoop.fs.FSDataOutputStream append(org.apache.hadoop.fs.Path f, int bufferSize) throws IOException - Overrides:
append
in classorg.apache.hadoop.fs.FileSystem
- Throws:
IOException
-
create
public org.apache.hadoop.fs.FSDataOutputStream create(org.apache.hadoop.fs.Path f, org.apache.hadoop.fs.permission.FsPermission permission, boolean overwrite, int bufferSize, short replication, long blockSize, org.apache.hadoop.util.Progressable progress) throws IOException - Specified by:
create
in classorg.apache.hadoop.fs.FileSystem
- Throws:
IOException
-
create
public org.apache.hadoop.fs.FSDataOutputStream create(org.apache.hadoop.fs.Path f, org.apache.hadoop.fs.permission.FsPermission permission, EnumSet<org.apache.hadoop.fs.CreateFlag> flags, int bufferSize, short replication, long blockSize, org.apache.hadoop.util.Progressable progress) throws IOException - Overrides:
create
in classorg.apache.hadoop.fs.FileSystem
- Throws:
IOException
-
listStatus
public org.apache.hadoop.fs.FileStatus[] listStatus(org.apache.hadoop.fs.Path f) throws FileNotFoundException, IOException - Specified by:
listStatus
in classorg.apache.hadoop.fs.FileSystem
- Throws:
FileNotFoundException
IOException
-
listCorruptFileBlocks
public org.apache.hadoop.fs.RemoteIterator<org.apache.hadoop.fs.Path> listCorruptFileBlocks(org.apache.hadoop.fs.Path path) throws IOException - Overrides:
listCorruptFileBlocks
in classorg.apache.hadoop.fs.FileSystem
- Throws:
IOException
-
listStatus
public org.apache.hadoop.fs.FileStatus[] listStatus(org.apache.hadoop.fs.Path f, org.apache.hadoop.fs.PathFilter filter) throws FileNotFoundException, IOException - Overrides:
listStatus
in classorg.apache.hadoop.fs.FileSystem
- Throws:
FileNotFoundException
IOException
-
listStatus
public org.apache.hadoop.fs.FileStatus[] listStatus(org.apache.hadoop.fs.Path[] files) throws FileNotFoundException, IOException - Overrides:
listStatus
in classorg.apache.hadoop.fs.FileSystem
- Throws:
FileNotFoundException
IOException
-
listStatus
public org.apache.hadoop.fs.FileStatus[] listStatus(org.apache.hadoop.fs.Path[] files, org.apache.hadoop.fs.PathFilter filter) throws FileNotFoundException, IOException - Overrides:
listStatus
in classorg.apache.hadoop.fs.FileSystem
- Throws:
FileNotFoundException
IOException
-
globStatus
public org.apache.hadoop.fs.FileStatus[] globStatus(org.apache.hadoop.fs.Path pathPattern) throws IOException - Overrides:
globStatus
in classorg.apache.hadoop.fs.FileSystem
- Throws:
IOException
-
globStatus
public org.apache.hadoop.fs.FileStatus[] globStatus(org.apache.hadoop.fs.Path pathPattern, org.apache.hadoop.fs.PathFilter filter) throws IOException - Overrides:
globStatus
in classorg.apache.hadoop.fs.FileSystem
- Throws:
IOException
-
listLocatedStatus
public org.apache.hadoop.fs.RemoteIterator<org.apache.hadoop.fs.LocatedFileStatus> listLocatedStatus(org.apache.hadoop.fs.Path f) throws FileNotFoundException, IOException - Overrides:
listLocatedStatus
in classorg.apache.hadoop.fs.FileSystem
- Throws:
FileNotFoundException
IOException
-
listFiles
public org.apache.hadoop.fs.RemoteIterator<org.apache.hadoop.fs.LocatedFileStatus> listFiles(org.apache.hadoop.fs.Path f, boolean recursive) throws FileNotFoundException, IOException - Overrides:
listFiles
in classorg.apache.hadoop.fs.FileSystem
- Throws:
FileNotFoundException
IOException
-
getHomeDirectory
public org.apache.hadoop.fs.Path getHomeDirectory()- Overrides:
getHomeDirectory
in classorg.apache.hadoop.fs.FileSystem
-
setWorkingDirectory
public void setWorkingDirectory(org.apache.hadoop.fs.Path new_dir) This method should never be used onDrillFileSystem
sinceDrillFileSystem
is immutable.- Specified by:
setWorkingDirectory
in classorg.apache.hadoop.fs.FileSystem
- Throws:
UnsupportedOperationException
- when called.
-
rename
public boolean rename(org.apache.hadoop.fs.Path src, org.apache.hadoop.fs.Path dst) throws IOException - Specified by:
rename
in classorg.apache.hadoop.fs.FileSystem
- Throws:
IOException
-
delete
Deprecated.- Overrides:
delete
in classorg.apache.hadoop.fs.FileSystem
- Throws:
IOException
-
delete
- Specified by:
delete
in classorg.apache.hadoop.fs.FileSystem
- Throws:
IOException
-
deleteOnExit
- Overrides:
deleteOnExit
in classorg.apache.hadoop.fs.FileSystem
- Throws:
IOException
-
cancelDeleteOnExit
public boolean cancelDeleteOnExit(org.apache.hadoop.fs.Path f) - Overrides:
cancelDeleteOnExit
in classorg.apache.hadoop.fs.FileSystem
-
exists
- Overrides:
exists
in classorg.apache.hadoop.fs.FileSystem
- Throws:
IOException
-
isDirectory
- Overrides:
isDirectory
in classorg.apache.hadoop.fs.FileSystem
- Throws:
IOException
-
isFile
- Overrides:
isFile
in classorg.apache.hadoop.fs.FileSystem
- Throws:
IOException
-
getLength
Deprecated.- Overrides:
getLength
in classorg.apache.hadoop.fs.FileSystem
- Throws:
IOException
-
getContentSummary
public org.apache.hadoop.fs.ContentSummary getContentSummary(org.apache.hadoop.fs.Path f) throws IOException - Overrides:
getContentSummary
in classorg.apache.hadoop.fs.FileSystem
- Throws:
IOException
-
getUri
- Specified by:
getUri
in classorg.apache.hadoop.fs.FileSystem
-
getCanonicalServiceName
- Specified by:
getCanonicalServiceName
in interfaceorg.apache.hadoop.security.token.DelegationTokenIssuer
- Overrides:
getCanonicalServiceName
in classorg.apache.hadoop.fs.FileSystem
-
getName
Deprecated.- Overrides:
getName
in classorg.apache.hadoop.fs.FileSystem
-
makeQualified
public org.apache.hadoop.fs.Path makeQualified(org.apache.hadoop.fs.Path path) - Overrides:
makeQualified
in classorg.apache.hadoop.fs.FileSystem
-
getDelegationToken
@Private public org.apache.hadoop.security.token.Token<?> getDelegationToken(String renewer) throws IOException - Specified by:
getDelegationToken
in interfaceorg.apache.hadoop.security.token.DelegationTokenIssuer
- Overrides:
getDelegationToken
in classorg.apache.hadoop.fs.FileSystem
- Throws:
IOException
-
addDelegationTokens
@LimitedPrivate({"HDFS","MapReduce"}) public org.apache.hadoop.security.token.Token<?>[] addDelegationTokens(String renewer, org.apache.hadoop.security.Credentials credentials) throws IOException - Specified by:
addDelegationTokens
in interfaceorg.apache.hadoop.security.token.DelegationTokenIssuer
- Throws:
IOException
-
getChildFileSystems
@LimitedPrivate("HDFS") public org.apache.hadoop.fs.FileSystem[] getChildFileSystems()- Overrides:
getChildFileSystems
in classorg.apache.hadoop.fs.FileSystem
-
getFileBlockLocations
public org.apache.hadoop.fs.BlockLocation[] getFileBlockLocations(org.apache.hadoop.fs.FileStatus file, long start, long len) throws IOException - Overrides:
getFileBlockLocations
in classorg.apache.hadoop.fs.FileSystem
- Throws:
IOException
-
getFileBlockLocations
public org.apache.hadoop.fs.BlockLocation[] getFileBlockLocations(org.apache.hadoop.fs.Path p, long start, long len) throws IOException - Overrides:
getFileBlockLocations
in classorg.apache.hadoop.fs.FileSystem
- Throws:
IOException
-
getServerDefaults
Deprecated.- Overrides:
getServerDefaults
in classorg.apache.hadoop.fs.FileSystem
- Throws:
IOException
-
getServerDefaults
public org.apache.hadoop.fs.FsServerDefaults getServerDefaults(org.apache.hadoop.fs.Path p) throws IOException - Overrides:
getServerDefaults
in classorg.apache.hadoop.fs.FileSystem
- Throws:
IOException
-
resolvePath
- Overrides:
resolvePath
in classorg.apache.hadoop.fs.FileSystem
- Throws:
IOException
-
truncate
- Overrides:
truncate
in classorg.apache.hadoop.fs.FileSystem
- Throws:
IOException
-
listStatusIterator
public org.apache.hadoop.fs.RemoteIterator<org.apache.hadoop.fs.FileStatus> listStatusIterator(org.apache.hadoop.fs.Path p) throws FileNotFoundException, IOException - Overrides:
listStatusIterator
in classorg.apache.hadoop.fs.FileSystem
- Throws:
FileNotFoundException
IOException
-
access
public void access(org.apache.hadoop.fs.Path path, org.apache.hadoop.fs.permission.FsAction mode) throws org.apache.hadoop.security.AccessControlException, FileNotFoundException, IOException - Overrides:
access
in classorg.apache.hadoop.fs.FileSystem
- Throws:
org.apache.hadoop.security.AccessControlException
FileNotFoundException
IOException
-
getFileChecksum
public org.apache.hadoop.fs.FileChecksum getFileChecksum(org.apache.hadoop.fs.Path f, long length) throws IOException - Overrides:
getFileChecksum
in classorg.apache.hadoop.fs.FileSystem
- Throws:
IOException
-
setXAttr
- Overrides:
setXAttr
in classorg.apache.hadoop.fs.FileSystem
- Throws:
IOException
-
setXAttr
public void setXAttr(org.apache.hadoop.fs.Path path, String name, byte[] value, EnumSet<org.apache.hadoop.fs.XAttrSetFlag> flag) throws IOException - Overrides:
setXAttr
in classorg.apache.hadoop.fs.FileSystem
- Throws:
IOException
-
getXAttr
- Overrides:
getXAttr
in classorg.apache.hadoop.fs.FileSystem
- Throws:
IOException
-
getXAttrs
- Overrides:
getXAttrs
in classorg.apache.hadoop.fs.FileSystem
- Throws:
IOException
-
getXAttrs
public Map<String,byte[]> getXAttrs(org.apache.hadoop.fs.Path path, List<String> names) throws IOException - Overrides:
getXAttrs
in classorg.apache.hadoop.fs.FileSystem
- Throws:
IOException
-
listXAttrs
- Overrides:
listXAttrs
in classorg.apache.hadoop.fs.FileSystem
- Throws:
IOException
-
removeXAttr
- Overrides:
removeXAttr
in classorg.apache.hadoop.fs.FileSystem
- Throws:
IOException
-
openPossiblyCompressedStream
Returns an InputStream from a Hadoop path. If the data is compressed, this method will return a compressed InputStream depending on the codec.- Parameters:
path
- Input file path- Returns:
- InputStream of opened file path
- Throws:
IOException
- If the file is unreachable, unavailable or otherwise unreadable
-
getCodec
public org.apache.hadoop.io.compress.CompressionCodec getCodec(org.apache.hadoop.fs.Path path) Returns theCompressionCodec
for a given file. This can be used to determine the type of compression (if any) which was used. Returns null if the file is not compressed.- Parameters:
path
- The file of unknown compression- Returns:
- CompressionCodec used by the file. Null if the file is not compressed.
-
fileOpened
Description copied from interface:OpenFileTracker
Add new file location andDrillFSDataInputStream
to list.- Specified by:
fileOpened
in interfaceOpenFileTracker
-
fileClosed
Description copied from interface:OpenFileTracker
Remove the givenDrillFSDataInputStream
from opened file list.- Specified by:
fileClosed
in interfaceOpenFileTracker
-