PowerShell » Get-Blog : Ryan's PowerShell Blog

True Multithreading in PowerShell

Jun 242014

As with all things, the longer you play with something the more you learn about it. It has been nearly 5 years since I wrote my original article on multithreading where I used PowerShell jobs to run multiple items at a time. In the conversations following that post I had a reader submit a fairly nasty note saying that multithreading with jobs was in fact not real multithreading. He also selected some choice words for me making sure I couldn’t post his comments. It hurt my very fragile feelings and, as a result, I started looking at how I could really, truly multithread with PowerShell. The result of that effort is this script. This is a true multithreading script no “ifs”, “ands” or “buts”.

It runs MUCH faster than my other rendition and includes a much more advanced feature set from my other script. The down side is that the script is very difficult to understand if you are new to PowerShell. If you would like to have a script that is more visible and easier to understand, please refer to the version that uses Jobs.

Okay, so the big addition for this script is the ability to either run a script or a cmdlet that’s built in. As well, you can run this within the pipeline! To do that I had to include the begin, process and end blocks. It makes the script a bit more complex, but really pays off when you pipe your custom script into Out-Gridview or pipe your advanced filtering script into a multithreaded one! I have even pulled my SCCM collections via Get-WmiObject and piped them into a multithreaded script! Cool stuff!

Okay, so on to the breakdown.

First we need to get all of our parameters. If this doesn’t make since, please find my post on parameters!

Param($Command = $(Read-Host "Enter the script file"), 
    [Parameter(ValueFromPipeline=$true,ValueFromPipelineByPropertyName=$true)]$ObjectList,
    $InputParam = $Null,
    $MaxThreads = 20,
    $SleepTimer = 200,
    $MaxResultTime = 120,
    [HashTable]$AddParam = @{},
    [Array]$AddSwitch = @()
)

Param($Command = $(Read-Host "Enter the script file"),

[Parameter(ValueFromPipeline=$true,ValueFromPipelineByPropertyName=$true)]$ObjectList,

$InputParam = $Null,

$MaxThreads = 20,

$SleepTimer = 200,

$MaxResultTime = 120,

[HashTable]$AddParam = @{},

[Array]$AddSwitch = @()

)

All of these are defined as follows:
Command
This is where you provide the powershell Commandlet / Script file that you want to multithread. You can also choose a built in cmdlet. Keep in mind that your script. This script is read into a scriptblock, so any unforeseen errors are likely caused by the conversion to a script block.

ObjectList
The objectlist represents the arguments that are provided to the child script. This is an open ended argument and can take a single object from the pipeline, an array, a collection, or a file name. The multithreading script does it’s best to find out which you have provided and handle it as such. If you would like to provide a file, then the file is read with one object on each line and will be provided as is to the script you are running as a string. If this is not desired, then use an array.

InputParam
This allows you to specify the parameter for which your input objects are to be evaluated. As an example, if you were to provide a computer name to the Get-Process cmdlet as just an argument, it would attempt to find all processes where the name was the provided computer name and fail. You need to specify that the parameter that you are providing is the “ComputerName”.

AddParam
This allows you to specify additional parameters to the running command. For instance, if you are trying to find the status of the “BITS” service on all servers in your list, you will need to specify the “Name” parameter. This command takes a hash pair formatted as follows:

@{"ParameterName" = "Value"}
@{"ParameterName" = "Value" ; "ParameterTwo" = "Value2"}

1 2	@{"ParameterName" = "Value"} @{"ParameterName" = "Value" ; "ParameterTwo" = "Value2"}

AddSwitch
This allows you to add additional switches to the command you are running. For instance, you may want to include “RequiredServices” to the “Get-Service” cmdlet. This parameter will take a single string, or an aray of strings as follows:

"RequiredServices"
@("RequiredServices", "DependentServices")

1 2	"RequiredServices" @("RequiredServices", "DependentServices")

MaxThreads
This is the maximum number of threads to run at any given time. If resources are too congested try lowering this number. The default value is 20.

SleepTimer
This is the time between cycles of the child process detection cycle. The default value is 200ms. If CPU utilization is high then you can consider increasing this delay. If the child script takes a long time to run, then you might increase this value to around 1000 (or 1 second in the detection cycle).

Now we need to set everything up. This stuff needs to execute outside of the pipeline, so we place it in the “begin” block of the script.

Begin{
    $ISS = [system.management.automation.runspaces.initialsessionstate]::CreateDefault()
    $RunspacePool = [runspacefactory]::CreateRunspacePool(1, $MaxThreads, $ISS, $Host)
    $RunspacePool.Open()
        
    If ($(Get-Command | Select-Object Name) -match $Command){
        $Code = $Null
    }Else{
        $OFS = "`r`n"
        $Code = [ScriptBlock]::Create($(Get-Content $Command))
        Remove-Variable OFS
    }
    $Jobs = @()
}

Begin{

$ISS = [system.management.automation.runspaces.initialsessionstate]::CreateDefault()

$RunspacePool = [runspacefactory]::CreateRunspacePool(1, $MaxThreads, $ISS, $Host)

$RunspacePool.Open()

If ($(Get-Command | Select-Object Name) -match $Command){

$Code = $Null

}Else{

$OFS = "`r`n"

$Code = [ScriptBlock]::Create($(Get-Content $Command))

Remove-Variable OFS

}

$Jobs = @()

}

Okay, so to break this down, first we need to make our ISS, or initial session state. This is basically the session state to be used when we open our Runspace. Next we create our Runspaces. The RunspacePool is really what’s going to do the multithreading. It will handle starting our threads and continuously start new ones as required. It is the operating environment for our command pipeline. Finally we open the RunSpacePool. Note that “.Open()” opens the RunSpacePool synchronously, creating a Windows PowerShell execution environment.

Now I am running a detection on what the user provided for the $command parameter. First I will look at all of the currently loaded cmdlets. If it is one of those, then we continue. Otherwise we assume it is a script file. If it is a script file then we need to read the file in to a script block that we can pass to our future threads. To do this we need to change the default $OFS (Object Field Separator), for more understanding here, please read my other post!

Okay, so the next step is to start receiving items from the pipeline. We can do this by starting the process block. Note that the process block is executed for each item we find in the pipeline. Meanwhile, if you did not execute in the pipeline it is executed once for the script as a whole. What this means is that we need to assume that $ObjectList will either be a single item or multiple items. The best way to do that is to use a ForEach Loop.

Process{
    Write-Progress -Activity "Preloading threads" -Status "Starting Job $($jobs.count)"
    ForEach ($Object in $ObjectList){
        If ($Code -eq $Null){
            $PowershellThread = [powershell]::Create().AddCommand($Command)
        }Else{
            $PowershellThread = [powershell]::Create().AddScript($Code)
        }
        If ($InputParam -ne $Null){
            $PowershellThread.AddParameter($InputParam, $Object.ToString()) | out-null
        }Else{
            $PowershellThread.AddArgument($Object.ToString()) | out-null
        }
        ForEach($Key in $AddParam.Keys){
            $PowershellThread.AddParameter($Key, $AddParam.$key) | out-null
        }
        ForEach($Switch in $AddSwitch){
            $Switch
            $PowershellThread.AddParameter($Switch) | out-null
        }
        $PowershellThread.RunspacePool = $RunspacePool
        $Handle = $PowershellThread.BeginInvoke()
        $Job = "" | Select-Object Handle, Thread, object
        $Job.Handle = $Handle
        $Job.Thread = $PowershellThread
        $Job.Object = $Object.ToString()
        $Jobs += $Job
    }
        
}

Process{

Write-Progress -Activity "Preloading threads" -Status "Starting Job $($jobs.count)"

ForEach ($Object in $ObjectList){

If ($Code -eq $Null){

$PowershellThread = [powershell]::Create().AddCommand($Command)

}Else{

$PowershellThread = [powershell]::Create().AddScript($Code)

}

If ($InputParam -ne $Null){

$PowershellThread.AddParameter($InputParam, $Object.ToString()) | out-null

}Else{

$PowershellThread.AddArgument($Object.ToString()) | out-null

}

ForEach($Key in $AddParam.Keys){

$PowershellThread.AddParameter($Key, $AddParam.$key) | out-null

}

ForEach($Switch in $AddSwitch){

$Switch

$PowershellThread.AddParameter($Switch) | out-null

}

$PowershellThread.RunspacePool = $RunspacePool

$Handle = $PowershellThread.BeginInvoke()

$Job = "" | Select-Object Handle, Thread, object

$Job.Handle = $Handle

$Job.Thread = $PowershellThread

$Job.Object = $Object.ToString()

$Jobs += $Job

}

So now we have to build the thread that we are going to execute. We do this by adding either the command, or the script. The first IF block is to determine which. A thread either takes an existing PowerShell command, or a Scriptblock. If you remember we built this out in the Begin statement above. Once that is done we need to start giving the user the power to control the item we are calling.

First things first we look at $InputParam. This is what allows the user to execute the child script not just with the argument provided, but also specify the parameter. We see this is useful with the Get-Process cmdlet. Let’s say that you want to see the processes running on 20 different servers. If you just ran Get-Process ServerName you would be looking at your local machine for any processes with the name “ServerName” and you would get no return (probably). Instead you would want to run Get-Process –ComputerName ServerName. The trick here is that when you do this you’ve actually changed things! When an item is just hanging at the end of the statement, it is called an Argument. When you pair the item you are setting with the setting it is called a Parameter. So if the user wants to specify the parameter name, we are actually adding a different item to our thread!

Now we need to see if the user wanted to add some extra parameters. For this I decided that a hash table was perfect. This is because they are built much like a parameter, as they are name / value pairs. The user can provide as many as they want in a single hash table, and we can easily run a ForEach to evaluate this. Again we are going to use the .AddParameter() statement to evaluate.

Finally we need to add any Switches that the user wants to add. A good example of this is the –Force switch on cmdlets like Get-ChildItem. We can do this with an array of string which again allows the user to put in as many various switches as they like. One interesting not here is that I had to use .AddParameter() for this as well. Instead of creating a method called .AddSwitch(), Microsoft simply chose to add an overloaded definition of .AddParameter(). What’s peculiar is that even in their documentation (link below) it does in fact say that providing just a string adds a switch instead of parameter.

Now that we have out thread all set up we simply attach our RunspacePool that we setup in the begin statement, and then tell it to execute our thread!

To avoid having a thread hanging around that we lose track of we need to try to do some tracking here. To do this we first catch the thread by creating the output of the command in a $Handle variable. This will provide the link back to the handle in our “End” block. I then go ahead and create a custom object which I have named $Job to hold all these little gems of knowledge. Then I add my custom object to the array for tracking called $Jobs. This method of creating objects was taught to me by my reader from this post!

At this point all of the code to start the jobs is complete! Now we just have to grab all of the jobs back. That calls for the “End” block which is executed once per script run. Since that is the case, we need one main loop to ensure that it stay running while we need it to. Within the “End” block we will watch for jobs to finish and provide that output as they complete.

End{
    $ResultTimer = Get-Date
    While (@($Jobs | Where-Object {$_.Handle -ne $Null}).count -gt 0)  {
    
        $Remaining = "$($($Jobs | Where-Object {$_.Handle.IsCompleted -eq $False}).object)"
        If ($Remaining.Length -gt 60){
            $Remaining = $Remaining.Substring(0,60) + "..."
        }
        Write-Progress `
            -Activity "Waiting for Jobs - $($MaxThreads - $($RunspacePool.GetAvailableRunspaces())) of $MaxThreads threads running" `
            -PercentComplete (($Jobs.count - $($($Jobs | Where-Object {$_.Handle.IsCompleted -eq $False}).count)) / $Jobs.Count * 100) `
            -Status "$(@($($Jobs | Where-Object {$_.Handle.IsCompleted -eq $False})).count) remaining - $remaining" 

        ForEach ($Job in $($Jobs | Where-Object {$_.Handle.IsCompleted -eq $True})){
            $Job.Thread.EndInvoke($Job.Handle)
            $Job.Thread.Dispose()
            $Job.Thread = $Null
            $Job.Handle = $Null
            $ResultTimer = Get-Date
        }
        If (($(Get-Date) - $ResultTimer).totalseconds -gt $MaxResultTime){
            Write-Error "Child script appears to be frozen, try increasing MaxResultTime"
            Exit
        }
        Start-Sleep -Milliseconds $SleepTimer
        
    } 
    $RunspacePool.Close() | Out-Null
    $RunspacePool.Dispose() | Out-Null
}

End{

$ResultTimer = Get-Date

While (@($Jobs | Where-Object {$_.Handle -ne $Null}).count -gt 0) {

$Remaining = "$($($Jobs | Where-Object {$_.Handle.IsCompleted -eq $False}).object)"

If ($Remaining.Length -gt 60){

$Remaining = $Remaining.Substring(0,60) + "..."

}

Write-Progress `

-Activity "Waiting for Jobs - $($MaxThreads - $($RunspacePool.GetAvailableRunspaces())) of $MaxThreads threads running" `

-PercentComplete (($Jobs.count - $($($Jobs | Where-Object {$_.Handle.IsCompleted -eq $False}).count)) / $Jobs.Count * 100) `

-Status "$(@($($Jobs | Where-Object {$_.Handle.IsCompleted -eq $False})).count) remaining - $remaining"

ForEach ($Job in $($Jobs | Where-Object {$_.Handle.IsCompleted -eq $True})){

$Job.Thread.EndInvoke($Job.Handle)

$Job.Thread.Dispose()

$Job.Thread = $Null

$Job.Handle = $Null

$ResultTimer = Get-Date

}

If (($(Get-Date) - $ResultTimer).totalseconds -gt $MaxResultTime){

Write-Error "Child script appears to be frozen, try increasing MaxResultTime"

Exit

}

Start-Sleep -Milliseconds $SleepTimer

}

$RunspacePool.Close() | Out-Null

$RunspacePool.Dispose() | Out-Null

}

The first part of this is simply to provide some form of pretty output. First, I evaluate what jobs are still running and creating a string (truncated) that names them. I added this so that if the child script is failing on just one or two servers, you will known which they are to fix them. Then a rather complex write-progress statement which I’ll let you look into. For More info on write-progress you can read my blog articles over write-progress or getting the progress of a child job.

Okay, so now we look for jobs that are completed. We can do this by looking into our array of objects where our handle (that we captured above) has .IsCompleted set to $true! We then run a ForEach on each of these to stop it running and dispose of it. Keep in mind that dispose returns all of the output from the thread. What this means is that the script will actually write all of the output as jobs are finished instead of having to wait for all jobs to finish!

Next I added some protection to the script. I had a couple of scripts in my arsenal that would lock up and never finish. When this happened the multithreading script would continue to run and loop forever and ever. To stop this I added a maximum time to wait for additional jobs to finish. To do this I simply look at the system clock for when the last job completed and look at the time gap. If it is greater than out parameter for $MaxResultTime then we’ll throw an error and exit. Note that until PowerShell actually closes those threads will continue to hang around!

As a very last step we clean up our $RunspacePool.

Well that’s all folks! I really hope that this script provide many time saving events for my wonderful readers. I know it has saved me hundreds of man hours!

Following is the full script with the comment block intact for your cutting and pasting pleasure! Note that you can use the advanced controls here to pop out to a new window or show plain code for copy and paste.

#.Synopsis
#    This is a quick and open-ended script multi-threader searcher
#    
#.Description
#    This script will allow any general, external script to be multithreaded by providing a single
#    argument to that script and opening it in a seperate thread.  It works as a filter in the 
#    pipeline, or as a standalone script.  It will read the argument either from the pipeline
#    or from a filename provided.  It will send the results of the child script down the pipeline,
#    so it is best to use a script that returns some sort of object.
#
#    Authored by Ryan Witschger - http://www.Get-Blog.com
#    
#.PARAMETER Command
#    This is where you provide the PowerShell Cmdlet / Script file that you want to multithread.  
#    You can also choose a built in cmdlet.  Keep in mind that your script.  This script is read into 
#    a scriptblock, so any unforeseen errors are likely caused by the conversion to a script block.
#    
#.PARAMETER ObjectList
#    The objectlist represents the arguments that are provided to the child script.  This is an open ended
#    argument and can take a single object from the pipeline, an array, a collection, or a file name.  The 
#    multithreading script does it's best to find out which you have provided and handle it as such.  
#    If you would like to provide a file, then the file is read with one object on each line and will 
#    be provided as is to the script you are running as a string.  If this is not desired, then use an array.
#    
#.PARAMETER InputParam
#    This allows you to specify the parameter for which your input objects are to be evaluated.  As an example, 
#    if you were to provide a computer name to the Get-Process cmdlet as just an argument, it would attempt to 
#    find all processes where the name was the provided computername and fail.  You need to specify that the 
#    parameter that you are providing is the "ComputerName".
#
#.PARAMETER AddParam
#    This allows you to specify additional parameters to the running command.  For instance, if you are trying
#    to find the status of the "BITS" service on all servers in your list, you will need to specify the "Name"
#    parameter.  This command takes a hash pair formatted as follows:  
#
#    @{"ParameterName" = "Value"}
#    @{"ParameterName" = "Value" ; "ParameterTwo" = "Value2"}
#
#.PARAMETER AddSwitch
#    This allows you to add additional switches to the command you are running.  For instance, you may want 
#    to include "RequiredServices" to the "Get-Service" cmdlet.  This parameter will take a single string, or 
#    an aray of strings as follows:
#
#    "RequiredServices"
#    @("RequiredServices", "DependentServices")
#
#.PARAMETER MaxThreads
#    This is the maximum number of threads to run at any given time.  If resources are too congested try lowering
#    this number.  The default value is 20.
#    
#.PARAMETER SleepTimer
#    This is the time between cycles of the child process detection cycle.  The default value is 200ms.  If CPU 
#    utilization is high then you can consider increasing this delay.  If the child script takes a long time to
#    run, then you might increase this value to around 1000 (or 1 second in the detection cycle).
#
#    
#.EXAMPLE
#    Both of these will execute the script named ServerInfo.ps1 and provide each of the server names in AllServers.txt
#    while providing the results to the screen.  The results will be the output of the child script.
#    
#    gc AllServers.txt | .\Run-CommandMultiThreaded.ps1 -Command .\ServerInfo.ps1
#    .\Run-CommandMultiThreaded.ps1 -Command .\ServerInfo.ps1 -ObjectList (gc .\AllServers.txt)
#
#.EXAMPLE
#    The following demonstrates the use of the AddParam statement
#    
#    $ObjectList | .\Run-CommandMultiThreaded.ps1 -Command "Get-Service" -InputParam ComputerName -AddParam @{"Name" = "BITS"}
#    
#.EXAMPLE
#    The following demonstrates the use of the AddSwitch statement
#    
#    $ObjectList | .\Run-CommandMultiThreaded.ps1 -Command "Get-Service" -AddSwitch @("RequiredServices", "DependentServices")
#
#.EXAMPLE
#    The following demonstrates the use of the script in the pipeline
#    
#    $ObjectList | .\Run-CommandMultiThreaded.ps1 -Command "Get-Service" -InputParam ComputerName -AddParam @{"Name" = "BITS"} | Select Status, MachineName
#


Param($Command = $(Read-Host "Enter the script file"), 
    [Parameter(ValueFromPipeline=$true,ValueFromPipelineByPropertyName=$true)]$ObjectList,
    $InputParam = $Null,
    $MaxThreads = 20,
    $SleepTimer = 200,
    $MaxResultTime = 120,
    [HashTable]$AddParam = @{},
    [Array]$AddSwitch = @()
)

Begin{
    $ISS = [system.management.automation.runspaces.initialsessionstate]::CreateDefault()
    $RunspacePool = [runspacefactory]::CreateRunspacePool(1, $MaxThreads, $ISS, $Host)
    $RunspacePool.Open()
        
    If ($(Get-Command | Select-Object Name) -match $Command){
        $Code = $Null
    }Else{
        $OFS = "`r`n"
        $Code = [ScriptBlock]::Create($(Get-Content $Command))
        Remove-Variable OFS
    }
    $Jobs = @()
}

Process{
    Write-Progress -Activity "Preloading threads" -Status "Starting Job $($jobs.count)"
    ForEach ($Object in $ObjectList){
        If ($Code -eq $Null){
            $PowershellThread = [powershell]::Create().AddCommand($Command)
        }Else{
            $PowershellThread = [powershell]::Create().AddScript($Code)
        }
        If ($InputParam -ne $Null){
            $PowershellThread.AddParameter($InputParam, $Object.ToString()) | out-null
        }Else{
            $PowershellThread.AddArgument($Object.ToString()) | out-null
        }
        ForEach($Key in $AddParam.Keys){
            $PowershellThread.AddParameter($Key, $AddParam.$key) | out-null
        }
        ForEach($Switch in $AddSwitch){
            $Switch
            $PowershellThread.AddParameter($Switch) | out-null
        }
        $PowershellThread.RunspacePool = $RunspacePool
        $Handle = $PowershellThread.BeginInvoke()
        $Job = "" | Select-Object Handle, Thread, object
        $Job.Handle = $Handle
        $Job.Thread = $PowershellThread
        $Job.Object = $Object.ToString()
        $Jobs += $Job
    }
        
}

End{
    $ResultTimer = Get-Date
    While (@($Jobs | Where-Object {$_.Handle -ne $Null}).count -gt 0)  {
    
        $Remaining = "$($($Jobs | Where-Object {$_.Handle.IsCompleted -eq $False}).object)"
        If ($Remaining.Length -gt 60){
            $Remaining = $Remaining.Substring(0,60) + "..."
        }
        Write-Progress `
            -Activity "Waiting for Jobs - $($MaxThreads - $($RunspacePool.GetAvailableRunspaces())) of $MaxThreads threads running" `
            -PercentComplete (($Jobs.count - $($($Jobs | Where-Object {$_.Handle.IsCompleted -eq $False}).count)) / $Jobs.Count * 100) `
            -Status "$(@($($Jobs | Where-Object {$_.Handle.IsCompleted -eq $False})).count) remaining - $remaining" 

        ForEach ($Job in $($Jobs | Where-Object {$_.Handle.IsCompleted -eq $True})){
            $Job.Thread.EndInvoke($Job.Handle)
            $Job.Thread.Dispose()
            $Job.Thread = $Null
            $Job.Handle = $Null
            $ResultTimer = Get-Date
        }
        If (($(Get-Date) - $ResultTimer).totalseconds -gt $MaxResultTime){
            Write-Error "Child script appears to be frozen, try increasing MaxResultTime"
            Exit
        }
        Start-Sleep -Milliseconds $SleepTimer
        
    } 
    $RunspacePool.Close() | Out-Null
    $RunspacePool.Dispose() | Out-Null
}

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

134

135

136

137

138

139

140

141

142

143

144

145

146

147

148

149

150

151

152

153

154

155

156

157

158

159

160

161

162

163

164

165

166

#.Synopsis

# This is a quick and open-ended script multi-threader searcher

#.Description

# This script will allow any general, external script to be multithreaded by providing a single

# argument to that script and opening it in a seperate thread. It works as a filter in the

# pipeline, or as a standalone script. It will read the argument either from the pipeline

# or from a filename provided. It will send the results of the child script down the pipeline,

# so it is best to use a script that returns some sort of object.

# Authored by Ryan Witschger - http://www.Get-Blog.com

#.PARAMETER Command

# This is where you provide the PowerShell Cmdlet / Script file that you want to multithread.

# You can also choose a built in cmdlet. Keep in mind that your script. This script is read into

# a scriptblock, so any unforeseen errors are likely caused by the conversion to a script block.

#.PARAMETER ObjectList

# The objectlist represents the arguments that are provided to the child script. This is an open ended

# argument and can take a single object from the pipeline, an array, a collection, or a file name. The

# multithreading script does it's best to find out which you have provided and handle it as such.

# If you would like to provide a file, then the file is read with one object on each line and will

# be provided as is to the script you are running as a string. If this is not desired, then use an array.

#.PARAMETER InputParam

# This allows you to specify the parameter for which your input objects are to be evaluated. As an example,

# if you were to provide a computer name to the Get-Process cmdlet as just an argument, it would attempt to

# find all processes where the name was the provided computername and fail. You need to specify that the

# parameter that you are providing is the "ComputerName".

#.PARAMETER AddParam

# This allows you to specify additional parameters to the running command. For instance, if you are trying

# to find the status of the "BITS" service on all servers in your list, you will need to specify the "Name"

# parameter. This command takes a hash pair formatted as follows:

# @{"ParameterName" = "Value"}

# @{"ParameterName" = "Value" ; "ParameterTwo" = "Value2"}

#.PARAMETER AddSwitch

# This allows you to add additional switches to the command you are running. For instance, you may want

# to include "RequiredServices" to the "Get-Service" cmdlet. This parameter will take a single string, or

# an aray of strings as follows:

# "RequiredServices"

# @("RequiredServices", "DependentServices")

#.PARAMETER MaxThreads

# This is the maximum number of threads to run at any given time. If resources are too congested try lowering

# this number. The default value is 20.

#.PARAMETER SleepTimer

# This is the time between cycles of the child process detection cycle. The default value is 200ms. If CPU

# utilization is high then you can consider increasing this delay. If the child script takes a long time to

# run, then you might increase this value to around 1000 (or 1 second in the detection cycle).

#.EXAMPLE

# Both of these will execute the script named ServerInfo.ps1 and provide each of the server names in AllServers.txt

# while providing the results to the screen. The results will be the output of the child script.

# gc AllServers.txt | .\Run-CommandMultiThreaded.ps1 -Command .\ServerInfo.ps1

# .\Run-CommandMultiThreaded.ps1 -Command .\ServerInfo.ps1 -ObjectList (gc .\AllServers.txt)

#.EXAMPLE

# The following demonstrates the use of the AddParam statement

# $ObjectList | .\Run-CommandMultiThreaded.ps1 -Command "Get-Service" -InputParam ComputerName -AddParam @{"Name" = "BITS"}

#.EXAMPLE

# The following demonstrates the use of the AddSwitch statement

# $ObjectList | .\Run-CommandMultiThreaded.ps1 -Command "Get-Service" -AddSwitch @("RequiredServices", "DependentServices")

#.EXAMPLE

# The following demonstrates the use of the script in the pipeline

# $ObjectList | .\Run-CommandMultiThreaded.ps1 -Command "Get-Service" -InputParam ComputerName -AddParam @{"Name" = "BITS"} | Select Status, MachineName

Param($Command = $(Read-Host "Enter the script file"),

[Parameter(ValueFromPipeline=$true,ValueFromPipelineByPropertyName=$true)]$ObjectList,

$InputParam = $Null,

$MaxThreads = 20,

$SleepTimer = 200,

$MaxResultTime = 120,

[HashTable]$AddParam = @{},

[Array]$AddSwitch = @()

)

Begin{

$ISS = [system.management.automation.runspaces.initialsessionstate]::CreateDefault()

$RunspacePool = [runspacefactory]::CreateRunspacePool(1, $MaxThreads, $ISS, $Host)

$RunspacePool.Open()

If ($(Get-Command | Select-Object Name) -match $Command){

$Code = $Null

}Else{

$OFS = "`r`n"

$Code = [ScriptBlock]::Create($(Get-Content $Command))

Remove-Variable OFS

}

$Jobs = @()

}

Process{

Write-Progress -Activity "Preloading threads" -Status "Starting Job $($jobs.count)"

ForEach ($Object in $ObjectList){

If ($Code -eq $Null){

$PowershellThread = [powershell]::Create().AddCommand($Command)

}Else{

$PowershellThread = [powershell]::Create().AddScript($Code)

}

If ($InputParam -ne $Null){

$PowershellThread.AddParameter($InputParam, $Object.ToString()) | out-null

}Else{

$PowershellThread.AddArgument($Object.ToString()) | out-null

}

ForEach($Key in $AddParam.Keys){

$PowershellThread.AddParameter($Key, $AddParam.$key) | out-null

}

ForEach($Switch in $AddSwitch){

$Switch

$PowershellThread.AddParameter($Switch) | out-null

}

$PowershellThread.RunspacePool = $RunspacePool

$Handle = $PowershellThread.BeginInvoke()

$Job = "" | Select-Object Handle, Thread, object

$Job.Handle = $Handle

$Job.Thread = $PowershellThread

$Job.Object = $Object.ToString()

$Jobs += $Job

}

End{

$ResultTimer = Get-Date

While (@($Jobs | Where-Object {$_.Handle -ne $Null}).count -gt 0) {

$Remaining = "$($($Jobs | Where-Object {$_.Handle.IsCompleted -eq $False}).object)"

If ($Remaining.Length -gt 60){

$Remaining = $Remaining.Substring(0,60) + "..."

}

Write-Progress `

-Activity "Waiting for Jobs - $($MaxThreads - $($RunspacePool.GetAvailableRunspaces())) of $MaxThreads threads running" `

-PercentComplete (($Jobs.count - $($($Jobs | Where-Object {$_.Handle.IsCompleted -eq $False}).count)) / $Jobs.Count * 100) `

-Status "$(@($($Jobs | Where-Object {$_.Handle.IsCompleted -eq $False})).count) remaining - $remaining"

ForEach ($Job in $($Jobs | Where-Object {$_.Handle.IsCompleted -eq $True})){

$Job.Thread.EndInvoke($Job.Handle)

$Job.Thread.Dispose()

$Job.Thread = $Null

$Job.Handle = $Null

$ResultTimer = Get-Date

}

If (($(Get-Date) - $ResultTimer).totalseconds -gt $MaxResultTime){

Write-Error "Child script appears to be frozen, try increasing MaxResultTime"

Exit

}

Start-Sleep -Milliseconds $SleepTimer

}

$RunspacePool.Close() | Out-Null

$RunspacePool.Dispose() | Out-Null

}

Further Reading:
Initial Session State:
http://msdn.microsoft.com/en-us/library/system.management.automation.runspaces.initialsessionstate%28v=vs.85%29.aspx
Runspaces:
http://msdn.microsoft.com/en-us/library/System.Management.Automation.Runspaces.Runspace(v=vs.85).aspx
PowerShell Class:
http://msdn.microsoft.com/en-us/library/system.management.automation.powershell%28v=vs.85%29.aspx
PowerShell AddParameter Method:
http://msdn.microsoft.com/en-us/library/system.management.automation.powershell.addparameter%28v=vs.85%29.aspx

Quick Tips: Counting in PowerShell

PowerShell, Quick Tips No Responses »

Jan 302013

If you’re like me, sometimes you just feel like counting. Well, in PowerShell you can count easier than ever before! PowerShell gives you ability to not only easily increment, but also run code for each increment because you can pipe! First, let’s take a look at the basic code:

PS C:> 1..5
1
2
3
4
5

PS C:> 1..5

PS C:> 5..7
5
6
7

PS C:> 5..7

And why limit yourself to counting up?! You can even decrement those numbers:

PS C:> 6..3
6
5
4
3

PS C:> 6..3

So, how can you use this? Well, remember the wonderful thing called a pipeline. You can potentially ping an entire subnet with the following command:

1..254 | % {ping 192.168.1.$_}

1	1..254 \| % {ping 192.168.1.$_}

Remember, if that command takes too long, you could consider multithreading it with one of the scripts on this site!

Quick Tip: Newlines when Typecasting Arrays to Strings

PowerShell, Quick Tips 3 Responses »

Aug 302012

Converting arrays into strings can cause a couple different problems, and I am starting to see more and more people struggling. The problem is that it doesn’t always act like you think it should. By default, converting an array into a string will place every item on one line of text separated by a space. This is particularly true with arrays made by get-content that are typecast into a string.

The answer is very simple. When you type cast an array into a string every field is separated by a secret value (not so secret anymore!) called the Object Field Separator, or OFS. This value is keep in a variable called $OFS and you can change it! For instance, if you want there to be a new line when you typecast the array then set $OFS = "`r`n" which will make a newline between every field. If you want to comma separate the values, then set $OFS = ", "

Just remember to clean up when you are done! You can set the value back to it’s default (a single space) by removing the variable: Remove-Variable OFS Don’t worry, PowerShell will remake it.

C:> $Test = "Ryan", "John", "Mark"
C:> [String]$Test
Ryan Mark John

C:> $OFS = "`r`n"
C:> [String]$Test
Ryan
Mark
John

C:> $OFS = ", "
C:> [String]$Test
Ryan, Mark, John

C:> Remove-Variable OFS
C:> [String]$Test
Ryan Mark John

C:> $Test = "Ryan", "John", "Mark"

C:> [String]$Test

Ryan Mark John

C:> $OFS = "`r`n"

C:> [String]$Test

Ryan

Mark

John

C:> $OFS = ", "

C:> [String]$Test

Ryan, Mark, John

C:> Remove-Variable OFS

C:> [String]$Test

Ryan Mark John

NOTE: The ‘ that you are seeing should be the Backtick key found to the left of your numbers on a US keyboard.

Enjoy!

AD Without Quest: Script to Query AD (In The Pipeline)

Active Directory, PowerShell No Responses »

Aug 302012

I really don’t like using tools like Quest for several reasons, primary among them being that it isn’t portable, or installed by default. I started the AD without Quest segment some time ago and I wanted to continue it again, but in a new direction. I think making scripts that can work in the pipeline is a better approach over making functions that port to each new script. That said, I wanted to make a generic AD searcher that could be used without modification for most queries. After I created it, I found that I prefer it over the UI based tools for information gathering purposes, it really is pretty neat.

Let’s get started!

First, we need to create our variables. As with all of my recent scripts, I have placed all of my variables in the Param block so that I can modify them at run time. I really like this practice since I can now change the way the script works at run time and I can even use “tab complete” to remember what I was thinking if I run the script 7 months from now. Notice that I setup a “default” value for each of these parameters so that it doesn’t need to be provided for the script to run, you are simply modifying the way the script runs with each.

Param(
    [String]$ObjectType = "*",
    [Parameter(ValueFromPipeline=$true)][String]$SearchTerm = "*",
    [String]$SearchProperty = "cn",
    [String]$SearchOperator = "=",
    [Array]$PropertiesToLoad
)

Param(

[String]$ObjectType = "*",

[Parameter(ValueFromPipeline=$true)][String]$SearchTerm = "*",

[String]$SearchProperty = "cn",

[String]$SearchOperator = "=",

[Array]$PropertiesToLoad

)

Notice that the only value that I have defined as “from the pipeline” is the search term. This is because I think it’s the only thing that might change if it is coming from the pipeline. I don’t think, for instance that you would have a single search string, but then search on different properties. As well, I have forced the “PropertiesToLoad” to be an array. This is because I will use the .count property later, and if only one property is provided then PowerShell will make the variable a string instead of an array of strings. Strings do not have the .count property.

Now, I wanted to script to work in the pipeline, and there are several semantics that must be used to do this “properly” the first is that you have a “begin”, “process” and “end” block. The begin block will be run once, it is where you can setup any persistent objects or other complete environment items. In my case, I don’t want to create a new .NET AD searcher for every single item in the pipeline, the one will serve for everything, so I create it in the “begin” block.

Begin{
    $Searcher=New-Object System.DirectoryServices.DirectorySearcher
    $Searcher.PageSize = 1000

Begin{

$Searcher=New-Object System.DirectoryServices.DirectorySearcher

$Searcher.PageSize = 1000

The page size of the searcher will break the search into chunks returning 1000 objects at a time. The end result of a search with 8000 users is the same, but AD actually limits the number of items that can be returned in a search (by default 1000) so in the background we will perform eight inquiries to get our one return.

Now, I wanted a way get back only certain data. What if I only want to send the name down the pipeline? What if I want a quick way to see users with a logon count of greater than 1000 and their user names? I decided to let the user pass an array of attribute names to be loaded and sent down the pipeline. More on this later, but for now I am going to load them all into the searcher. Again, this is only done once, not for every single item since it’s in the begin block.

    If ($PropertiesToLoad -ne $Null){
        ForEach ($Property in $PropertiesToLoad){
            $Searcher.PropertiesToLoad.Add($Property.ToLower())  | Out-Null
        }
    }
}

If ($PropertiesToLoad -ne $Null){

ForEach ($Property in $PropertiesToLoad){

$Searcher.PropertiesToLoad.Add($Property.ToLower()) | Out-Null

}

Now, we can move to the “process” portion of the script. This will get executed for each item that enters the pipeline, or a single time if executing outside of the pipeline.

Process{

Process{

The first thing that we need to do is setup the search string for the AD object. I am going to base this whole thing off of the parameters we created at the start. Those familiar with AD search string will recognize this. This is also where the heart of the script resides.

$Searcher.Filter="(&(objectcategory=$ObjectType($SearchProperty$SearchOperator$SearchTerm))"
$Results=$Searcher.FindAll()

1 2	$Searcher.Filter="(&(objectcategory=$ObjectType($SearchProperty$SearchOperator$SearchTerm))" $Results=$Searcher.FindAll()

All this does is make a string that makes a typical, quick search. The default value of the parameters would may the string "(&(objectcategory=*)(cn=*))" and return every single AD object. Any modification of the parameters narrows it down from there. If the script is executed with –ObjectType user then the string would become "(&(objectcategory=user)(cn=*))" and return every single user. I think you probably get the idea. The second line simply executes the query and places the results in our variable with the same name.

For the last part of the script, I wanted to format the data for use in the most intelligent way. Basically, depending on what the user provides for the “PropertiesToLoad” parameter we will have the script give a slightly different output. I decided that if the user gave nothing, then we would send the native .NET object that the searcher returns, unmodified. That would allow the most open-ended use of the return down the pipeline because all of the properties are included. If the user gives a single property, then they are probably trying to send a single piece of information down the pipeline, and so we can return a basic array of strings. Finally, if they are providing several, specific pieces, then perhaps they are trying to gather information, and we can make an array of objects with a property for each item requested and provide it, this makes the output compatible with cmdlets like Out-Gridview.

To start off we will see if the user provided input for “PropertiesToLoad” and either show the direct results, or start some post processing. If we are going to start post processing we need to make our array of objects and all associated properties. If there was only one object loaded, we’ll convert the array of objects into an array of strings.

    If ($PropertiesToLoad -ne $Null){
        ForEach ($Result in $Results){
            $Return = "" | Select-Object $PropertiesToLoad
            ForEach ($Property in $PropertiesToLoad){
                $Return.$Property = [STRING]$Result.properties.$($Property.ToLower())
            }
            If ($PropertiesToLoad.Count -eq 1){
                $Return | %{$_.$PropertiesToLoad}
            }Else{
                $Return
            }
        }
    }Else{
        $Results
    }

If ($PropertiesToLoad -ne $Null){

ForEach ($Result in $Results){

$Return = "" | Select-Object $PropertiesToLoad

ForEach ($Property in $PropertiesToLoad){

$Return.$Property = [STRING]$Result.properties.$($Property.ToLower())

}

If ($PropertiesToLoad.Count -eq 1){

$Return | %{$_.$PropertiesToLoad}

}Else{

$Return

}

}Else{

$Results

}

To break it down more in case that’s a bit confusing. The first step is to make an object called Return and give it the properties that the user wants. This is a quick, easy way to do just that.

            $Return = "" | Select-Object $PropertiesToLoad

1	$Return = "" \| Select-Object $PropertiesToLoad

Next we will loop through all of the properties that they would like to have and move it from our results to our return object. We have to do this so that we can pick and choose only the items the user wanted. Even though the native .NET result was limited to the properties we told it above, it will always contain certain things like the distinguished name. Making our own object removes those items and allows PowerShell to display it in a pretty manner.

            If ($PropertiesToLoad.Count -eq 1){
                $Return | %{$_.$PropertiesToLoad}

1 2	If ($PropertiesToLoad.Count -eq 1){ $Return \| %{$_.$PropertiesToLoad}

Here we are checking to see if only one property was provided. If so then we will quickly convert the array of objects into an array of strings. This is the best way I know how to do that. We are simply sending the object down the pipeline and running a ForEach-Object (% is a built-in alias) and printing the property.

}Else{
                $Return
            }
}

}Else{

$Return

}

If there is more than one property then we will print the whole object.

That’s it. Once you start to play around with the script you will start to see all of the possibilities. The following code block has the entire thing as one with a help block for usage information. This block will make the script compatible with Get-Help as well!

#.Synopsis
#    This is a quick and open-ended Active Directory searcher
#   
#.Description
#    This script will perform an Active directory search based off of the criteria provided.  It
#    is capabale of reading items from the pipeline and delivering items to the pipeline.  The output
#    format is dependant on parameters provided.
#
#    Authored by Ryan Witschger - http://Ryan.Witschger.net
#   
#.PARAMETER ObjectType
#    This is the type of object to search for.  Typical valid options are "user" and "computer"; however,
#    it will take any valid entry for "objectCategory".  This limits the objects returned to this
#    type.  The default for this option is "*" and will search for all objects.
#   
#.PARAMETER SearchTerm
#    This parameter takes the query to search for including wilcards.  Examples are "*Ryan*" or "Server1".
#    Any valid AD search term can be accpeted.  The default search term is "*" and will search for all objects.
#    This is the only parameter that will be read from th pipeline.
#   
#.PARAMETER SearchProperty
#    The is the property to search against.  Examples are "cn" and "whenCreated".  This will work for
#    any entry to your AD schema. the default entry is "cn" to allow a search for object names
#   
#.PARAMETER SearchOperator
#    This paramter takes the logical operator to provide with the AD search.  Examples are "=", "<=" and ">=".
#    You may use any valid AD search operator here.  The default for this entry is "=" and will allow
#    for a wildcard search of a string.
#   
#.PARAMETER PropertiesToLoad
#    A list, comma seperated, or all properties that you would like to return for the objects found.  This
#    parameter also determines the type of output for the script.  If not provided the default is to return
#    the .NET object typical provided by an AD searcher.  A single entry (example "Name") will provide an
#    array of strings with that property.  Finally, an array of properties will provide and array of objects
#    with those properties present.  This output may be used in the pipeline.
#   
#.EXAMPLE
#    Return all AD objects.
#   
#    ./Get-ADObject.ps1
#   
#.EXAMPLE
#    Return all users named Steve.
#   
#    ./Get-ADObject.ps1 -ObjectType user -SearchTerm "*Steve*"
#   
#.EXAMPLE
#    Read computers from a file and return thier last logon time.
#   
#    Get-Content c:Computers.txt | Get-ADObject.ps1 -ObjectType computer -SearchProperty name -PropertiesToLoad lastlogon
#   
#.EXAMPLE
#    Get local user profiles, lookup their AD properties and display them in Out-Gridview
#   
#    GCI c:users | % {$_.Name} | .Get-ADObject.ps1 -SearchProperty sAMAccountName -PropertiesToLoad GivenName, ln, initials, whenCreated | Out-Gridview
 
Param(
    [String]$ObjectType = "*",
    [Parameter(ValueFromPipeline=$true)][String]$SearchTerm = "*",
    [String]$SearchProperty = "cn",
    [String]$SearchOperator = "=",
    [Array]$PropertiesToLoad
)
 
Begin{
    $Searcher=New-Object System.DirectoryServices.DirectorySearcher
    $Searcher.PageSize = 1000
   
    If ($PropertiesToLoad -ne $Null){
        ForEach ($Property in $PropertiesToLoad){
            $Searcher.PropertiesToLoad.Add($Property.ToLower())  | Out-Null
        }
    }
}
 
Process{
    $Searcher.Filter="(&(objectcategory=$ObjectType)($SearchProperty$SearchOperator$SearchTerm))"
    $Results=$Searcher.FindAll()
 
    If ($PropertiesToLoad -ne $Null){
        ForEach ($Result in $Results){
            $Return = "" | Select-Object $PropertiesToLoad
            ForEach ($Property in $PropertiesToLoad){
                $Return.$Property = [STRING]$Result.properties.$($Property.ToLower())
            }
            If ($PropertiesToLoad.Count -eq 1){
                $Return | %{$_.$PropertiesToLoad}
            }Else{
                $Return
            }
        }
    }Else{
        $Results
    }
}

#.Synopsis

# This is a quick and open-ended Active Directory searcher

#.Description

# This script will perform an Active directory search based off of the criteria provided. It

# is capabale of reading items from the pipeline and delivering items to the pipeline. The output

# format is dependant on parameters provided.

# Authored by Ryan Witschger - http://Ryan.Witschger.net

#.PARAMETER ObjectType

# This is the type of object to search for. Typical valid options are "user" and "computer"; however,

# it will take any valid entry for "objectCategory". This limits the objects returned to this

# type. The default for this option is "*" and will search for all objects.

#.PARAMETER SearchTerm

# This parameter takes the query to search for including wilcards. Examples are "*Ryan*" or "Server1".

# Any valid AD search term can be accpeted. The default search term is "*" and will search for all objects.

# This is the only parameter that will be read from th pipeline.

#.PARAMETER SearchProperty

# The is the property to search against. Examples are "cn" and "whenCreated". This will work for

# any entry to your AD schema. the default entry is "cn" to allow a search for object names

#.PARAMETER SearchOperator

# This paramter takes the logical operator to provide with the AD search. Examples are "=", "<=" and ">=".

# You may use any valid AD search operator here. The default for this entry is "=" and will allow

# for a wildcard search of a string.

#.PARAMETER PropertiesToLoad

# A list, comma seperated, or all properties that you would like to return for the objects found. This

# parameter also determines the type of output for the script. If not provided the default is to return

# the .NET object typical provided by an AD searcher. A single entry (example "Name") will provide an

# array of strings with that property. Finally, an array of properties will provide and array of objects

# with those properties present. This output may be used in the pipeline.

#.EXAMPLE

# Return all AD objects.

# ./Get-ADObject.ps1

#.EXAMPLE

# Return all users named Steve.

# ./Get-ADObject.ps1 -ObjectType user -SearchTerm "*Steve*"

#.EXAMPLE

# Read computers from a file and return thier last logon time.

# Get-Content c:Computers.txt | Get-ADObject.ps1 -ObjectType computer -SearchProperty name -PropertiesToLoad lastlogon

#.EXAMPLE

# Get local user profiles, lookup their AD properties and display them in Out-Gridview

# GCI c:users | % {$_.Name} | .Get-ADObject.ps1 -SearchProperty sAMAccountName -PropertiesToLoad GivenName, ln, initials, whenCreated | Out-Gridview

Param(

[String]$ObjectType = "*",

[Parameter(ValueFromPipeline=$true)][String]$SearchTerm = "*",

[String]$SearchProperty = "cn",

[String]$SearchOperator = "=",

[Array]$PropertiesToLoad

)

Begin{

$Searcher=New-Object System.DirectoryServices.DirectorySearcher

$Searcher.PageSize = 1000

If ($PropertiesToLoad -ne $Null){

ForEach ($Property in $PropertiesToLoad){

$Searcher.PropertiesToLoad.Add($Property.ToLower()) | Out-Null

}

Process{

$Searcher.Filter="(&(objectcategory=$ObjectType)($SearchProperty$SearchOperator$SearchTerm))"

$Results=$Searcher.FindAll()

If ($PropertiesToLoad -ne $Null){

ForEach ($Result in $Results){

$Return = "" | Select-Object $PropertiesToLoad

ForEach ($Property in $PropertiesToLoad){

$Return.$Property = [STRING]$Result.properties.$($Property.ToLower())

}

If ($PropertiesToLoad.Count -eq 1){

$Return | %{$_.$PropertiesToLoad}

}Else{

$Return

}

}Else{

$Results

}

Dynamic Write-Progress Percentages

PowerShell No Responses »

Feb 162012

I have had problems in my very long scripts with Write-Progress. When writing the script you add in several Write-Progress statements and give them a percentage, the problem is that the percentage you give it is likely a number that you are typing in and hard coding. This is a problem because if you add to the code or move the code around you have to go back and change every one of these items.

In a recent script I wrote that was just over 300 lines I had to repeat this process 5 times which was really frustrating, so I thought up a way to make sure I don’t have to do it again.

Now, I should state beforehand that this process only works if each write-progress is executed once and only once. Any changes to that means you may need to add some other intelligence, but in general I think this is a good starting point.

The first thing you want to do is read the script file that you are executing to get a count of the number of lines with a “Write-Progress” in them. Then store a tracking number as a global variable so that all functions can see it. Do this at the start of the script.

Here is my single line to do all of the above:

$ProgressIndicators = $($Global:I=0; $(GC $MyInvocation.InvocationName | ? {$_ -Match "Write-Progress"}).count -1)

So, there is a whole lot of learning packed into a small area here. Let me cover the confusing sections. First naming a variable $Global:I will create a global variable called “I” that is accessible from all PowerShell scopes. As well, there is a “;” after this portion to execute this as a separate line of code. Since there is no output, this will not become part of the variable when run like this. For more information on scopes visit http://technet.microsoft.com/en-us/library/dd315289.aspx.

The command GC is a built-in alias for Get-Content.

The object property “$MyInvocation.InvocationName will provide the file path of the script that is currently being executed. So, in effect, we are reading the entire script that is currently running.

Then we pipe the script contents to ? which is a built-in alias for Where-Object. Then we look for a match on “Write-Progress”. You’ll notice that this whole section is surrounded in $() which “Instantiates” a variable for this this output. Then we can find the number of lines with the .count property. After that, I subtract 1 so it doesn’t count itself.

All of this means that the variable $ProgressIndicators will now contain the number of times “Write-Progress” appears in the script, no matter what script you place it in.

So now all we need is for Write-Progress to read these two values when it run and increment our global tracking variable.

Write-Progress -Activity "Making Progress" -Status "Test 1" -PercentComplete $($($Global:I++;$Global:I) / $ProgressIndicators * 100)
Sleep 1
Write-Progress -Activity "Making Progress" -Status "Test 2" -PercentComplete $($($Global:I++;$Global:I) / $ProgressIndicators * 100)
Sleep 1
Write-Progress -Activity "Making Progress" -Status "Test 3" -PercentComplete $($($Global:I++;$Global:I) / $ProgressIndicators * 100)
Sleep 1

So, as you can see I use the same techniques described above to increment the variable and then do some basic match to get a percentage of total progress.

You can throw this code into any script to give you a dynamically updating progress indicator. There are two things to keep in mind. If you have a loop that runs a write-progress command over and over then it will continue to increment and break this. You could remove the $Global:I++ portion or make a sub progress bar to correctly display the progress of the loop. Also, you might skip a write-progress line due to an IF statement. If so maybe you could increment the value of $Global:I in an ELSE statement.

AD without Quest: Getting and Using ADSPath

Active Directory, PowerShell 3 Responses »

Jul 032011

This is the second post in my AD without Quest series. I am covering individual functions that can be combine to produce a wide variety of scripts. Today I am going to be covering how to connect to AD to read an object ADSPath. The ADSPath is basically the LDAP string to connect to that object. Once you have an object’s LDAP path it is very easy to work with the object.

When searching in AD all you need to do is use the directory searcher object and continue to narrow down the filter. There are some slight changes between searching for a user, computer and group, so I’ll cover each.

Function Get-UserADSPathbyName{
Param([STRING]$User)

$Searcher=New-Object System.DirectoryServices.DirectorySearcher
$Searcher.Filter="(&(SamAccountName=$User)(objectcategory=user))"
$Searcher.PageSize = 1000
$Results=$Searcher.FindAll()
If(!$?){"AD searcher failed on $User"}

If ($Results.Count -eq 1){
$Results[0].properties.adspath
}Else{
Return $false
}
}

On the line with $Results[0].properties.adspath it is important to note that the “adspath” portion is case sensitive. I really have no clue why that is the case. If you know, drop me a comment to help me out!

Function Get-ComputerADSPathbyName{
Param([STRING]$Computer)

$Searcher=New-Object System.DirectoryServices.DirectorySearcher
$Searcher.Filter=”(&(CN=$Computer)(objectcategory=computer))”
$Searcher.PageSize = 1000
$Results=$Searcher.FindAll()
If(!$?){"AD searcher failed on $Computer"}

If ($Results.Count -eq 1){
$Results[0].properties.adspath
}Else{
Return $false
}
}

So as you can see, a couple of small changes will convert the function for use with computers instead of users. One final function converts this again for use with groups.

Function Get-GroupADSPathbyName{
Param([STRING]$Group)

$Searcher=New-Object System.DirectoryServices.DirectorySearcher
$Searcher.Filter=”(&(CN=$Group)(objectcategory=group))”
$Searcher.PageSize = 1000
$Results=$Searcher.FindAll()
If(!$?){"AD searcher failed on $Group"}

If ($Results.Count -eq 1){
$Results[0].properties.adspath
}Else{
Return $false
}
}

So there they are, but what can you do with them? There is a great variable type in PowerShell defined with the ADSI tag. Once a LDAP string is defined as and ADSI variable you can interact directly with that object by reading any property you want, and even changing them.

$UserLDAP = [ADSI]$(Get-UserADSPathbyName “MyUserName”)
$UserLDAP.extensionAttribute1
>MyOldValue
$UserLDAP.Put(“extensionAttribute1”,”MyNewValue”)
$UserLDAP.SetInfo()
$UserLDAP.extensionAttribute1
>MyNewValue

AD Without Quest: Listing Servers and Users

Active Directory, PowerShell No Responses »

Jul 022011

I’ve decided to start an Active Directory series. I have discovered that when you search online for any AD related code everything points back to the Quest tools. I prefer to do everything natively and not rely on a third party tool kits that have to be installed. As an admin, I understand that we do rely on our workstations, but when you start needing to install third party tools on server you are entering dangerous ground. I wouldn’t want to explain that Exchange went down because I installed some tool that locked the server.

I have been using PowerShell since its inception and I still haven’t found an instance where I need to use the Quest tools. Sometimes it might be easier, but once you’ve created a well written function, it’s easy to copy and paste it into your next script. So I am going to base this series off of individual functions that can be added to scripts to work with AD.

We’ll start with the basic ones. The following functions can be used to pull all the server names and user name out of Active Directory. This can be modified to pull whatever property you’d like, but I find that either account name or ADSPath (the LDAP string) are the most useful. I use some variation of these in almost every script I make.

First, let’s look at pulling all AD servers. Now these are only for the domain that you are running it from. There is some extra work to do to change domains, but I’ll cover that in a later blog post. The basic idea is to create a directory searcher, then set a filter (or you will return every object in AD). Then you need to set the page size. That, basically, tells the searcher how many objects it can return in a single search, so if you set it to 1000 and you have 8000 total objects, it will run a total of 8 searches. This step can be skipped if you have a very small domain, but AD has a built in limit to how many objects can be returned in a single search. This value can be set by the admin, but 1000 works with default domains. Finally, you run the search and then return the property you want from the function.

Function Get-ServerList{
$Searcher=New-Object System.DirectoryServices.DirectorySearcher
$Searcher.Filter="(&(objectcategory=computer)(operatingSystem=*Server*))"
$Searcher.PageSize = 1000
$Results=$Searcher.FindAll()

$Results | % {$_.Properties.name}
}

So, that’s it. You can drop this into your script and then run it with $servers = Get-ServerList to get a list of all your server names. This function runs very quickly and can be put into any script to make things really easily.

However, part of what I am trying to get across is that these functions can be quickly changed to suite your needs. So this is the function very slightly changed to pull all the users instead. In this case the PageSize parameter really comes into use.

Function Get-UserList{
$Searcher=New-Object System.DirectoryServices.DirectorySearcher
$Searcher.Filter="(objectcategory=user)"
$Searcher.PageSize = 1000
$Results=$Searcher.FindAll()

$Results | % {$_.Properties.samaccountname}
}

More AD entries to come! You can find them via my blogs tags to the right.

Adding Help to a PowerShell Script

PowerShell No Responses »

Mar 152011

With the scripting games coming up, I noticed that they will be adding stars to your work based on the addition of script help blocks that are compatible with the PowerShell “Comment Based Help”. Basically, this is a form of writing a comment block that works in conjunction with the get-help cmdlet that is included with PowerShell.

With this post I am hoping spread the information that this is possible just as much as the how to do it. I think that most script writers don’t even know it’s possible, and I’ll be honest that while I’ve know that you can do it, I don’t include in all of my own scripts.

So, you can add comment based help either to a script or to a function. It is my opinion that you do not need to add the help to a function within a script as no one will be able to run a get-help against it; however, using the format provided to label your functions is probably a good idea.

So, there are two ways to make a comment block and three places to put them.

You can start all the lines in a block with # or you can surround all the lines with <# ...comments... #>. Either form is fine, but the later might be easier for large block; however, the former might be easier to follow if browsing the code. I typically like to start my lines with # as I can quickly see where all the comments start and stop.

As far as where to place them, you can put them at the start of a function or script, at the end of a function or script,or just before the start of a function. Line breaks become very important as spaces between comments and their attached objects will detach them.

Now the comments need to follow the format:

# .KEYWORD
# Content
#
# .SYNOPSIS
# This is an example of comment based help.

So what kind of keywords are there? This is taken from the help file:

.SYNOPSIS
A brief description of the function or script. This keyword can be used
only once in each topic.

.DESCRIPTION
A detailed description of the function or script. This keyword can be
used only once in each topic.

.PARAMETER <Parameter-Name>
The description of a parameter. You can include a Parameter keyword for
each parameter in the function or script syntax.

The Parameter keywords can appear in any order in the comment block,but
the function or script syntax determines the order in which the parameters
(and their descriptions) appear in Help topic. To change the order,
change the syntax.

You can also specify a parameter description by placing a comment in the
function or script syntax immediately before the parameter variable name.
If you use both a syntax comment and a Parameter keyword, the description
associated with the Parameter keyword is used, and the syntax comment is
ignored.

.EXAMPLE
A sample command that uses the function or script, optionally followed
by sample output and a description. Repeat this keyword for each example.

.INPUTS
The Microsoft .NET Framework types of objects that can be piped to the
function or script. You can also include a description of the input
objects.

.OUTPUTS
The .NET Framework type of the objects that the cmdlet returns. You can
also include a description of the returned objects.

.NOTES
Additional information about the function or script.

.LINK
The name of a related topic. Repeat this keyword for each related topic.

This content appears in the Related Links section of the Help topic.

The Link keyword content can also include a Uniform Resource Identifier
(URI) to an online version of the same Help topic. The online version
opens when you use the Online parameter of Get-Help. The URI must begin
with "http" or "https".

There are even more past that, but I think that covers the main group. If you want to see them all and get more detailed information go to PowerShell and run Get-Help about_Comment_Based_Help.

As well, there are several built in sections that are generated automatically: Name, Syntax, Parameter List, Common Parameters, Parameter Attribute Table, Remarks.

To get more information and some good examples go to PowerShell and run Get-Help about_Comment_Based_Help.

Getting the Progress of a Child Thread in PowerShell

PowerShell 3 Responses »

Feb 042011

My most popular article has thus far been the post over multithreading found here. As such I felt that my audience might like to see some more entries covering that topic. After all, it is a vast topic with a lot of very advanced things you can do. In this post I would like to talk about tracking the status of your various threads. In other words, how can you see a progress of what your child script is doing?

In exploring this myself, I worked with passing custom objects back to the parent which held various status messages that the parent could read in with the “receive-job” command. That actually worked, but it took a lot of code in both the parent and the child to generate and read the status from the child threads.

While working with this, though, I found that the Job object from “get-job”, a System.Management.Automation.Job, has a property called “Progress”. I then decided to look into how this actually gets filled out. And the answer eluded my research until I managed to do it, completely by accident. As it turns out, this very useful property is filled in automatically by a “write-progress” command run within the child thread. It is actually quite logical that the progress field is filled in by “write-progress”.

So, that is enough of a background on how I came about it. Let’s actually look at some code. First,I want to create a script block to run within my child job.

## Write some code for the child threads to execute
$ThreadCode = {
$x = 0
$time = 200
While ($x -lt 100){
$x++

Write-Progress -Activity "Testing"
-Status "Testing"
-PercentComplete $X
-completed
Start-Sleep -Milliseconds $time
}
Write-Progress -Activity "Testing"
-Status "Testing"
-PercentComplete 100
-Completed
"Annnd we're done."
}

So here you’ll notice that I am basically creating a loop that will count 1 to 100 and then close. Along the way it will fill out its progress with the “write-progress” cmdlet. The one important note is that I am adding the “-completed” parameter to the “write-progress” cmdlet. That may look like a mistake in the code,but it isn’t. If you play around with this manner of getting progress from a child you will notice that when you run a “receive-job” your host thread will read all output from the child, including flashing all of the “write-progress” bars that were in the child. If you actually read the help files for “write-progress”, you’ll see that actual description of the functionality of “-complete” states that is simply hides the output bar. We need to do this so that when the parent thread runs a “receive-job” there is no odd behavior for the end user.

## Kill any current jobs in the sessions and get rid of them
Get-Job | Stop-Job
Get-Job | Remove-Job

## Start the child job and assign to a variable to save some code later
$Job = Start-Job -ScriptBlock $ThreadCode

So for this quick snippet we are just closing an remove any jobs our current session may have, and then starting our job. I assign it to a variable so we don’t have to track it down later, not that it’s hard.

While ($Job.State -eq "Running"){
## Get the Child job object
$Child = $Job.ChildJobs[0]

## Get the newest Progress Object in the child job
$Progress = $Child.Progress[$Child.Progress.Count - 1]

## If the progress object is not null
If ($Progress.Activity -ne $Null){
Write-Progress -Activity $Job.Name
-Status $Progress.StatusDescription
-PercentComplete $Progress.PercentComplete
-ID $Job.ID
}

Start-Sleep -Milliseconds 200
}

So this is the bread and butter of the parent script. It simply loops around as long as our job is running. First we need to find the child job to our parent job. I am not sure why Microsoft chose to design it like this, but each job you start actually runs commands in a child job. I have a feeling it has something to do with functionality with Windows HPC, but that is neither here nor there. Once we have our child job, we want to find the latest progress object that has been passed. After that I just check to see if the object is empty, because the first check may not have given the child thread enough time to fill this out. If there is data, then we can fill out a “write-progress” bar on our host thread. From there we just repeat until the job is done.

Get-Job | Receive-Job

At the end we just want to read anything the child script may actually have sent. So that covers it. It is really easy to get custom progress bars for a child process because there is a good, built in method to do it.

Following is a full example of a script which will launch three child threads at a time until there have been 10 opened total. It will show you the output of each thread as it completes and the progress bars produced by each. I even included a random speed that each child will complete so you can see some bells and whistles going.

## Write some code for the child threads to execute
$ThreadCode = {
$x = 0
$time = Get-Random -Maximum 200 -SetSeed $(Get-Date).Millisecond
While ($x -lt 100){
$x++

Write-Progress -Activity "Testing" -Status "Testing" -PercentComplete $X -completed
Start-Sleep -Milliseconds $time
}
Write-Progress -Activity "Testing" -Status "Testing" -PercentComplete 100 -Completed
"Annnd we're done."
}

## Kill any current jobs in the sessions and get rid of them
Get-Job | Stop-Job
Get-Job | Remove-Job

## Loop control
$x = 0

## While we are still launching threads or wait for them to close
While (($x -lt 10) -or ($(Get-Job -State "Running").count -gt 0)){
## While there are less than three jobs running and less than 10 have started
While (($(Get-Job -State "Running").count -lt 3) -and ($x -lt 10)){
Start-Job -ScriptBlock $ThreadCode | Out-Null
$x++
}

## Main portion - Read all jobs
ForEach ($Job in Get-Job) {
## Read all children of all jobs
ForEach ($Child in $Job.ChildJobs){
## Get the latest progress object of the job
$Progress = $Child.Progress[$Child.Progress.Count - 1]

## If there is a progress object returned write progress
If ($Progress.Activity -ne $Null){
Write-Progress -Activity $Job.Name
-Status $Progress.StatusDescription
-PercentComplete $Progress.PercentComplete
-ID $Job.ID
}

## If this child is complete then stop writing progress
If ($Progress.PercentComplete -eq 100){
Write-Progress -Activity $Job.Name
-Status $Progress.StatusDescription
-PercentComplete $Progress.PercentComplete
-ID $Job.ID
-Complete
## Clear all progress entries so we don't process it again
$Child.Progress.Clear()
}
}
}

Get-Job -State "Completed" | Receive-Job

## Setting for loop processing speed
Start-Sleep -Milliseconds 200
}

Get-Job | Wait-Job | Out-Null
Get-Job | Receive-Job

Quick Tip: Tab Complete in PowerShell

Quick Tips No Responses »

Feb 022011

As you are typing in any given PowerShell console or in PowerShell ISE you can finish typing many things simply by hitting the “Tab” key while you are halfway through a word. This practice is refered to as “Tab Complete”. If you aren’t in the practice of using this all the time, then you really need to start utilizing it heavily for your own sake. Not only can it help you type and create code faster, but it can be useful in remembering cmdlet names or parameters when you are typing.

Things I know you can tab complete are: cmdlet names, file and directory names, third party programs, custom scripts, cmdlet parameters, variable names, object members, functions, and many more.

Another cool thing is that if you hit it too early in the word, simply hit it again to keep going through all possible entries. If you type “Get-” and then start hitting tab,you will see the names of all the cmdlets until you get the one you want. Or after you’ve typed the name of a cmdlet,type “-” and then tab to see all possible parameter names, and it even works with your own scripts!

Skipped by the command you wanted? Shift-Tab will start you back in the other direction.

Older Entries