Package org.apache.drill.exec.util
Class FileSystemUtil
java.lang.Object
org.apache.drill.exec.util.FileSystemUtil
Helper class that provides methods to list directories or file or both statuses.
Can list statuses recursive and apply custom filters.
-
Field Summary
Modifier and TypeFieldDescriptionstatic final org.apache.hadoop.fs.PathFilter
Filter that will accept all files and directories.static final String
-
Constructor Summary
-
Method Summary
Modifier and TypeMethodDescriptionstatic List<org.apache.hadoop.fs.FileStatus>
anyFile
(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path path, org.apache.hadoop.fs.PathFilter... filters) Returns the statuses of at least one file present in given path applying custom filters if present.static List<org.apache.hadoop.fs.FileStatus>
listAll
(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path path, boolean recursive, org.apache.hadoop.fs.PathFilter... filters) Returns statuses of all directories and files present in given path applying custom filters if present.static List<org.apache.hadoop.fs.FileStatus>
listAllSafe
(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path path, boolean recursive, org.apache.hadoop.fs.PathFilter... filters) Returns statuses of all directories and files present in given path applying custom filters if present.static List<org.apache.hadoop.fs.FileStatus>
listDirectories
(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path path, boolean recursive, org.apache.hadoop.fs.PathFilter... filters) Returns statuses of all directories present in given path applying custom filters if present.static List<org.apache.hadoop.fs.FileStatus>
listDirectoriesSafe
(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path path, boolean recursive, org.apache.hadoop.fs.PathFilter... filters) Returns statuses of all directories present in given path applying custom filters if present.static List<org.apache.hadoop.fs.FileStatus>
listFiles
(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path path, boolean recursive, org.apache.hadoop.fs.PathFilter... filters) Returns statuses of all files present in given path applying custom filters if present.static List<org.apache.hadoop.fs.FileStatus>
listFilesSafe
(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path path, boolean recursive, org.apache.hadoop.fs.PathFilter... filters) Returns statuses of all files present in given path applying custom filters if present.static org.apache.hadoop.fs.PathFilter
mergeFilters
(org.apache.hadoop.fs.PathFilter... filters) Will merge given array of filters into one.static org.apache.hadoop.fs.PathFilter
mergeFilters
(org.apache.hadoop.fs.PathFilter filter, org.apache.hadoop.fs.PathFilter[] filters) Merges given filter with array of filters.
-
Field Details
-
RECURSIVE_LISTING_PROP_NAME
- See Also:
-
DUMMY_FILTER
public static final org.apache.hadoop.fs.PathFilter DUMMY_FILTERFilter that will accept all files and directories.
-
-
Constructor Details
-
FileSystemUtil
public FileSystemUtil()
-
-
Method Details
-
listDirectories
public static List<org.apache.hadoop.fs.FileStatus> listDirectories(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path path, boolean recursive, org.apache.hadoop.fs.PathFilter... filters) throws IOException Returns statuses of all directories present in given path applying custom filters if present. Will also include nested directories if recursive flag is set to true.- Parameters:
fs
- current file systempath
- path to directoryrecursive
- true if nested directories should be includedfilters
- list of custom filters (optional)- Returns:
- list of matching directory statuses
- Throws:
IOException
-
listDirectoriesSafe
public static List<org.apache.hadoop.fs.FileStatus> listDirectoriesSafe(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path path, boolean recursive, org.apache.hadoop.fs.PathFilter... filters) Returns statuses of all directories present in given path applying custom filters if present. Will also include nested directories if recursive flag is set to true. Will ignore all exceptions during listing if any.- Parameters:
fs
- current file systempath
- path to directoryrecursive
- true if nested directories should be includedfilters
- list of custom filters (optional)- Returns:
- list of matching directory statuses
-
listFiles
public static List<org.apache.hadoop.fs.FileStatus> listFiles(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path path, boolean recursive, org.apache.hadoop.fs.PathFilter... filters) throws IOException Returns statuses of all files present in given path applying custom filters if present. Will also include files from nested directories if recursive flag is set to true.- Parameters:
fs
- current file systempath
- path to file or directoryrecursive
- true if files in nested directories should be includedfilters
- list of custom filters (optional)- Returns:
- list of matching file statuses
- Throws:
IOException
-
listFilesSafe
public static List<org.apache.hadoop.fs.FileStatus> listFilesSafe(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path path, boolean recursive, org.apache.hadoop.fs.PathFilter... filters) Returns statuses of all files present in given path applying custom filters if present. Will also include files from nested directories if recursive flag is set to true.- Parameters:
fs
- current file systempath
- path to file or directoryrecursive
- true if files in nested directories should be includedfilters
- list of custom filters (optional)- Returns:
- list of matching file statuses
-
anyFile
public static List<org.apache.hadoop.fs.FileStatus> anyFile(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path path, org.apache.hadoop.fs.PathFilter... filters) throws IOException Returns the statuses of at least one file present in given path applying custom filters if present.- Parameters:
fs
- current file systempath
- path to file or directoryfilters
- list of custom filters (optional)- Returns:
- list of at most one matching file status
- Throws:
IOException
-
listAll
public static List<org.apache.hadoop.fs.FileStatus> listAll(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path path, boolean recursive, org.apache.hadoop.fs.PathFilter... filters) throws IOException Returns statuses of all directories and files present in given path applying custom filters if present. Will also include nested directories and their files if recursive flag is set to true.- Parameters:
fs
- current file systempath
- path to file or directoryrecursive
- true if nested directories and their files should be includedfilters
- list of custom filters (optional)- Returns:
- list of matching directory and file statuses
- Throws:
IOException
-
listAllSafe
public static List<org.apache.hadoop.fs.FileStatus> listAllSafe(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path path, boolean recursive, org.apache.hadoop.fs.PathFilter... filters) Returns statuses of all directories and files present in given path applying custom filters if present. Will also include nested directories and their files if recursive flag is set to true. Will ignore all exceptions during listing if any.- Parameters:
fs
- current file systempath
- path to file or directoryrecursive
- true if nested directories and their files should be includedfilters
- list of custom filters (optional)- Returns:
- list of matching directory and file statuses
-
mergeFilters
public static org.apache.hadoop.fs.PathFilter mergeFilters(org.apache.hadoop.fs.PathFilter filter, org.apache.hadoop.fs.PathFilter[] filters) Merges given filter with array of filters. If array of filters is null or empty, will return given filter.- Parameters:
filter
- given filterfilters
- array of filters- Returns:
- one filter that combines all given filters
-
mergeFilters
public static org.apache.hadoop.fs.PathFilter mergeFilters(org.apache.hadoop.fs.PathFilter... filters) Will merge given array of filters into one. If given array of filters is empty, will returnDUMMY_FILTER
.- Parameters:
filters
- array of filters- Returns:
- one filter that combines all given filters
-