Skip to content
Menu
ScriptingNerd
  • Home
  • Quick Tips
  • GitHub
ScriptingNerd

Running ForEach in parallel on Windows Powershell 5 (and older)

Posted on November 27, 2021April 20, 2022 by Patrik

What we are looking at today is something I have seen people ask for time and time again but never seen a good pre-built solution for. I used to have my own "manual" and clunky way of doing this but then when looking for something completely different I just happened to stumble upon this little beauty of a function being used internally in the ChocoOneGet module. And let me tell you it is just amazing in its simplicity!

https://www.powershellgallery.com/packages/ChocoOneGet/0.4.0/Content/Internal%5CForEach-Parallel.ps1

Internally this function uses runspaces but you really don't need to know anything about how it does it's magic to use it so that is all I am going to say about that. If you are interested in finding out exactly how it works let me know and I might do another post explaining the code inside the function.

So how does it work from a user perspective then?

Actually it is really simple, it works pretty much the same as Foreach-Object but much much faster. Lets take an example. Below you have two loops that run the Test-Connection command 16 times, I will get back to that number later on. One of these loops use regular old ForEach-Object and the other one is using ForEach-Parallel. Then in the box below you have the output of this little experiment.

#Load the function
.\ForEach-Parallel.ps1


"Regular ForEach"
Measure-Command {
    1..16| ForEach-Object {
        Test-Connection "scriptingnerd.com"
    }
}| Select -Expand TotalSeconds


"`nForEach-Parallel"
Measure-Command {
    1..16| ForEach-Parallel -ScriptBlock {
        Test-Connection "scriptingnerd.com"
    }
}| Select -Expand TotalSeconds
Regular ForEach
52,3000472

ForEach-Parallel
3,4803696

Wow, you see that? The regular ForEach took about 52 seconds while the Parallel one took just under 3.5 seconds. That's amazing and just imagine how much this could speed up your already existing scripts. Instead of having to wait for all tasks one by one your waiting time would always be that of the slowest task. That is, unless you do 17 or more tasks.

Wait, what do you mean?
Simple, by default the function will only start 16 tasks at a time, once one of them is finished it will run task 17 and so forth. In the example above this would lead to a doubling of the parallel time. The limit is there to avoid overwhelming your system but if you think it can handle the load you can change how many tasks should be allowed to run at the same time by passing in the parameter -MaxRunspaces. Like this

# Load the function
.\ForEach-Parallel.ps1

"ForEach-Parallel (Default)"
Measure-Command {
    1..17| ForEach-Parallel -ScriptBlock {
        Test-Connection "scriptingnerd.com"
    }
}

"`nForEach-Parallel (Extra runspaces)"
Measure-Command {
    1..17| ForEach-Parallel -ScriptBlock {
        Test-Connection "scriptingnerd.com"
    } -MaxRunspaces 17
}
ForEach-Parallel (Default)
6,6795728

ForEach-Parallel (Extra runspaces)
3,5955956

Ok, this sound great, so what's the catch?
Honestly, I haven't found anything that stopped me from using it yet, should I find one in the future I will come back and update this. That said there is one thing you should be aware of, if you are used to the parallel flag of Powershell 7 then know that this behaves a bit different. Instead of running everything and throw it back to you as soon as it finish this function will wait until everything is finished before giving it back, this also means that it will be returned in the same order you sent it in, not in the order it finished.

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

  • Patrik Johansson
    Senior System Administrator

    Working with a wide flora of systems but with a focus on Configuration Manager (MEMCM/SCCM).

    Passionate about Powershell and building tools that are easy to use for everyone.
    In my spare time I work on other coding projects as well, mainly Booksonic

    LinkedIn GitHub Blog

Recent Posts

  • How to download CMTrace from Microsoft
  • Powershell counting to $null instead of 1
  • Running ForEach in parallel on Windows Powershell 5 (and older)
  • Use Powershell to create a "fake" program in the programs and features list for indexing purposes
  • How to read the manifest of an appx package file using Powershell

Archives

  • February 2023
  • April 2022
  • November 2021
  • March 2020
  • December 2019
  • February 2019

Categories

  • Powershell
  • Quick Tips
  • Below are affiliate links, if you click on them and buy something I may earn a small amount of money without any additional cost to you. Any links found here are for products I am using myself and can vouch for.
  • InterServer
    Very cheap SSD/HDD based virtual servers (VPS). Starting at $6/month for 1TB. Use coupon code BOOKSONIC and get the first month for just 1 cent. I have recently started using intersever for personal projects and so far I am very happy with them.
  • Contabo
    Cheap HDD/SSD/NVMe based Virtual servers (VPS)
    I have used Contabo for years both for my private needs as well as to host the booksonic demo server
©2023 ScriptingNerd | Powered by WordPress & Superb Themes