Package org.apache.drill.exec.util
Class DrillFileSystemUtil
java.lang.Object
org.apache.drill.exec.util.DrillFileSystemUtil
In Drill file system all directories and files that start with dot or underscore is ignored.
This helper class that delegates all work to list directory and file statuses to
FileSystemUtil
class,
only adding Drill file system filter first.-
Field Summary
Modifier and TypeFieldDescriptionstatic final org.apache.hadoop.fs.PathFilter
Path filter that skips all files and folders that start with dot or underscore. -
Constructor Summary
-
Method Summary
Modifier and TypeMethodDescriptionstatic List<org.apache.hadoop.fs.FileStatus>
anyFile
(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path path, org.apache.hadoop.fs.PathFilter... filters) Returns the status of any file present in given path applying custom filters if present.static org.apache.hadoop.fs.Path
createPathSafe
(String path) Safely creates Hadoop Path for null and empty String pathsstatic List<org.apache.hadoop.fs.FileStatus>
listAll
(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path path, boolean recursive, org.apache.hadoop.fs.PathFilter... filters) Returns statuses of all directories and files present in given path applying custom filters if present.static List<org.apache.hadoop.fs.FileStatus>
listAllSafe
(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path path, boolean recursive, org.apache.hadoop.fs.PathFilter... filters) Returns statuses of all directories and files present in given path applying custom filters if present.static List<org.apache.hadoop.fs.FileStatus>
listDirectories
(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path path, boolean recursive, org.apache.hadoop.fs.PathFilter... filters) Returns statuses of all directories present in given path applying custom filters if present.static List<org.apache.hadoop.fs.FileStatus>
listDirectoriesSafe
(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path path, boolean recursive, org.apache.hadoop.fs.PathFilter... filters) Returns statuses of all directories present in given path applying custom filters if present.static List<org.apache.hadoop.fs.FileStatus>
listFiles
(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path path, boolean recursive, org.apache.hadoop.fs.PathFilter... filters) Returns statuses of all files present in given path applying custom filters if present.static List<org.apache.hadoop.fs.FileStatus>
listFilesSafe
(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path path, boolean recursive, org.apache.hadoop.fs.PathFilter... filters) Returns statuses of all files present in given path applying custom filters if present.
-
Field Details
-
DRILL_SYSTEM_FILTER
public static final org.apache.hadoop.fs.PathFilter DRILL_SYSTEM_FILTERPath filter that skips all files and folders that start with dot or underscore.
-
-
Constructor Details
-
DrillFileSystemUtil
public DrillFileSystemUtil()
-
-
Method Details
-
listDirectories
public static List<org.apache.hadoop.fs.FileStatus> listDirectories(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path path, boolean recursive, org.apache.hadoop.fs.PathFilter... filters) throws IOException Returns statuses of all directories present in given path applying custom filters if present. Directories that start with dot or underscore are skipped. Will also include nested directories if recursive flag is set to true.- Parameters:
fs
- current file systempath
- path to directoryrecursive
- true if nested directories should be includedfilters
- list of custom filters (optional)- Returns:
- list of matching directory statuses
- Throws:
IOException
-
listDirectoriesSafe
public static List<org.apache.hadoop.fs.FileStatus> listDirectoriesSafe(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path path, boolean recursive, org.apache.hadoop.fs.PathFilter... filters) Returns statuses of all directories present in given path applying custom filters if present. Directories that start with dot or underscore are skipped. Will include nested directories if recursive flag is set to true. Will ignore all exceptions during listing if any.- Parameters:
fs
- current file systempath
- path to directoryrecursive
- true if nested directories should be includedfilters
- list of custom filters (optional)- Returns:
- list of matching directory statuses
-
listFiles
public static List<org.apache.hadoop.fs.FileStatus> listFiles(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path path, boolean recursive, org.apache.hadoop.fs.PathFilter... filters) throws IOException Returns statuses of all files present in given path applying custom filters if present. Files and nested directories that start with dot or underscore are skipped. Will also include files from nested directories if recursive flag is set to true.- Parameters:
fs
- current file systempath
- path to file or directoryrecursive
- true if files in nested directories should be includedfilters
- list of custom filters (optional)- Returns:
- list of matching file statuses
- Throws:
IOException
-
listFilesSafe
public static List<org.apache.hadoop.fs.FileStatus> listFilesSafe(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path path, boolean recursive, org.apache.hadoop.fs.PathFilter... filters) Returns statuses of all files present in given path applying custom filters if present. Files and nested directories that start with dot or underscore are skipped. Will include files from nested directories if recursive flag is set to true. Will ignore all exceptions during listing if any.- Parameters:
fs
- current file systempath
- path to file or directoryrecursive
- true if files in nested directories should be includedfilters
- list of custom filters (optional)- Returns:
- list of matching file statuses
-
anyFile
public static List<org.apache.hadoop.fs.FileStatus> anyFile(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path path, org.apache.hadoop.fs.PathFilter... filters) throws IOException Returns the status of any file present in given path applying custom filters if present. Files and nested directories that start with dot or underscore are skipped. Will also include files from nested directories.- Parameters:
fs
- current file systempath
- path to file or directoryrecursive
- true if files in nested directories should be includedfilters
- list of custom filters (optional)- Returns:
- list of at most one matching file status
- Throws:
IOException
-
listAll
public static List<org.apache.hadoop.fs.FileStatus> listAll(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path path, boolean recursive, org.apache.hadoop.fs.PathFilter... filters) throws IOException Returns statuses of all directories and files present in given path applying custom filters if present. Directories and files that start with dot or underscore are skipped. Will also include nested directories and their files if recursive flag is set to true.- Parameters:
fs
- current file systempath
- path to file or directoryrecursive
- true if nested directories and their files should be includedfilters
- list of custom filters (optional)- Returns:
- list of matching directory and file statuses
- Throws:
IOException
-
listAllSafe
public static List<org.apache.hadoop.fs.FileStatus> listAllSafe(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path path, boolean recursive, org.apache.hadoop.fs.PathFilter... filters) Returns statuses of all directories and files present in given path applying custom filters if present. Directories and files that start with dot or underscore are skipped. Will include nested directories and their files if recursive flag is set to true. Will ignore all exceptions during listing if any.- Parameters:
fs
- current file systempath
- path to file or directoryrecursive
- true if nested directories and their files should be includedfilters
- list of custom filters (optional)- Returns:
- list of matching directory and file statuses
-
createPathSafe
Safely creates Hadoop Path for null and empty String paths- Parameters:
path
- String path, which can be null or empty- Returns:
- Hadoop Path. Root - for empty or null path
-