The fastest Powershell #2 : Count all files in a large network share

By | February 11, 2015

Updated :  May 21, 2016

Question : What is the fastest solution to count the number of files in a large network share ?


Answer : To answer this question, I will compare 4 different commands:

  1. [System.IO.Directory]::EnumerateFiles
  2. [System.IO.Directory]::GetFiles
  3. Get-ChildItem
  4. cmd /c dir
  5. robocopy (I need to include in this test, will be done later)

I will count all the files in a large network share (27.7 GB : 71 694 files and 8821 folders ).

Important : in this test there are no long paths, access denied, etc. I will write another article about how to deal with exceptions.

the-fastest-4-final-size

First, I just confirm that all these commands return the same result before measuring that.

We can start the measure as we have the same value (71690 files).

the-fastest-4-results

Result:

the-fastest-search-files

Conclusion : In this scenario, the fastest was :
[System.IO.Directory]::EnumerateFiles($path,'*.*','AllDirectories')

Note 1: The EnumerateFiles and GetFiles methods differ as follows: When you use EnumerateFiles, you can start enumerating the collection of names before the whole collection is returned; when you use GetFiles, you must wait for the whole array of names to be returned before you can access the array. Therefore, when you are working with many files and directories, EnumerateFiles can be more efficient.

Note 2 : I recommend to use multithreading search when working with large amount of data to speed up. I will write another article about that.

Note 3 : If you have a faster solution, feel free to comment below so I can update my article.


I have received an email about someone asking me how to deal with exceptions.
Here it is a temporary solution (which needs some minor improvements) to parse directories dealing with exceptions.
It requires the module AlphaFS but I will start to work to create a function using FindFirstFile and FindNextFile.


previous-button

2 thoughts on “The fastest Powershell #2 : Count all files in a large network share

  1. Pingback: # The fastest Powershell 3 : Search users in Active Directory

  2. Yomodo

    Hi, the next version of AlphaFS will allow for reporting of access denied messages AND resumes enumerating.

    Reply

Leave a Reply

Your email address will not be published. Required fields are marked *