Clear-Script

Verify Your Module's Help with Pester v5

2020-07-08T00:00:00-04:00

If anyone's not yet aware, Pester recently released a new major version: 5.0. This comes with a slew of breaking changes and some fancy new functionality, and some pretty solid performance improvements to boot. A user in the PowerShell Discord server recently came across some tests written for Pester v4 that they wanted to refactor for use in Pester v5. I decided to take a thorough look, and it proved to be quite a bit more complicated than I'd initially expected, mainly due to the way the tests were written.

What Worked in Pester v4

The original code we'll be hacking away at comes fom François-Xavier Cat (aka lazywinadmin) and looks like this:

Describe "AdsiPS Module" -Tags "Module" {

    # Import Module
    #import-module C:\Test\AdsiPS\AdsiPS.psd1

    #$FunctionsList = (get-command -Module ADSIPS).Name
    $FunctionsList = (get-command -Module ADSIPS | Where-Object -FilterScript { $_.CommandType -eq 'Function' }).Name

    FOREACH ($Function in $FunctionsList)
    {
        # Retrieve the Help of the function
        $Help = Get-Help -Name $Function -Full

        $Notes = ($Help.alertSet.alert.text -split '\n')

        # Parse the function using AST
        $AST = [System.Management.Automation.Language.Parser]::ParseInput((Get-Content function:$Function), [ref]$null, [ref]$null)

        Context "$Function - Help"{

            It "Synopsis"{ $help.Synopsis | Should not BeNullOrEmpty }
            It "Description"{ $help.Description | Should not BeNullOrEmpty }
            It "Notes - Author" { $Notes[0].trim() | Should Be "Francois-Xavier Cat" }
            It "Notes - Site" { $Notes[1].trim() | Should Be "Lazywinadmin.com" }
            It "Notes - Twitter" { $Notes[2].trim() | Should Be "@lazywinadmin" }
            It "Notes - Github" { $Notes[3].trim() | Should Be "github.com/lazywinadmin" }

            # Get the parameters declared in the Comment Based Help
            $RiskMitigationParameters = 'Whatif', 'Confirm'
            $HelpParameters = $help.parameters.parameter | Where-Object name -NotIn $RiskMitigationParameters

            # Get the parameters declared in the AST PARAM() Block
            $ASTParameters = $ast.ParamBlock.Parameters.Name.variablepath.userpath

            $FunctionsList = (get-command -Module $ModuleName | Where-Object -FilterScript { $_.CommandType -eq 'Function' }).Name



            It "Parameter - Compare Count Help/AST" {
                $HelpParameters.name.count -eq $ASTParameters.count | Should Be $true
            }

            # Parameter Description
            If (-not [String]::IsNullOrEmpty($ASTParameters)) # IF ASTParameters are found
            {
                $HelpParameters | ForEach-Object {
                    It "Parameter $($_.Name) - Should contains description"{
                        $_.description | Should not BeNullOrEmpty
                    }
                }

            }

            # Examples
            it "Example - Count should be greater than 0"{
                $Help.examples.example.code.count | Should BeGreaterthan 0
            }

            # Examples - Remarks (small description that comes with the example)
            foreach ($Example in $Help.examples.example)
            {
                it "Example - Remarks on $($Example.Title)"{
                    $Example.remarks | Should not BeNullOrEmpty
                }
            }
        }
    }
}

There's a lot going on here, but there are a few things I'd specifically like to point out that will cause us some particular difficulty. First on the list is the very frequent shifting of scope and shared information between Describe/Context blocks and the It blocks which make up the actual tests. This can be an issue in Pester v4 as well, but you'll generally only notice that if the code being used generates errors.

For a concrete example, look at how $Help is used in the above code – it's created in a Describe block and used again in both Context and It blocks. It's unlikely, but if that Get-Help call generates an error, the entire Describe test would fail, halting all subsequent tests. With the way these tests are set up, whichever functions have yet to be tested would simple not get tested. That isn't a huge issue in this specific case, but it's definitely something to keep in mind.

When writing Pester tests, you generally want to follow a couple very broad rules (among others):

As much code as possible should be inside It or a BeforeAll/BeforeEach/AfterAll/AfterEach block. This ensures that when errors occur, they register as a test failure without necessarily breaking the rest of the tests completely.
Code used to generate test cases should utilize -TestCases wherever possible. This lets you offload all the looping logic to Pester itself, and tends to make tests run a little more smoothly.

While you can (yes, even in v5) utilise a standard loop to generate It test cases, you will very quickly run into issues doing that in v5. Part of that is due to the split behaviour of Discovery and Run in Pester v5, so let's cover that real quick.

Testing Phases in Pester v5

In Pester v5, tests happen in two very distinct phases. First, there's a Discovery phase:

During Discovery, code in the main body of the test file, as well as code in Describe and Context blocks, is executed. This allows you to generate test cases, iteratively create It blocks directly, and gives Pester a chance to evaluate its test plan ahead of time.

Second, there's the Run phase:

During the Run phase, only code inside an It block will be run. This does include It blocks and test cases that were generated by code run during Discovery, but variables created during Discovery are typically not available during Run. For that reason, I'd recommend moving all your test case generation to proper -TestCases usage first — it's the only supported way to make data from Discovery available during Run.

Limitations of `-TestCases`

At the time of writing, it's not yet possible replicate fully what the above code is doing by using -TestCases. If you examine the code, you can see it's actually generating Context blocks. This is still doable in Pester v5, but passing data from there into the Run phase can get a bit tricky. However, we'll do our best!

Prepare for Pester v5

The initial code is working from an assumption that all public functions for our module are contained in individual function files under the Public folder in the module. We can break away from this assumption by simply querying the imported module for its exported commands, which will include compiled cmdlets as well.

The other side of things is that in order to pass data from the Discovery phase to the test runs, we'll need to utilize -TestCases quite heavily. This could be done purely with -TestCases, by including the command names and all other information as part of the test case. Doing things that way works quite well, but you will lose the ability to have each command in its own Context block. If that doesn't bother you, feel free to go that route! Personally, I appreciate the extra clarity of having a complete block for each command.

Below is a rewritten version of the above script, which should be fully compatible with Pester v5. There are some design changes here and there, but the vast majority of the above script has been retained. It includes a couple of variables for a module name, which should let you substitute any module name in to test against that module.

You'll notice I have a habit of placing code explicitly in regions marked Discovery. This is purely something I do for clarity; Pester doesn't require it, and won't recognise it differently to anything else. I'm hopeful that at some point in future Pester will provide an explicit BeforeDiscovery block for this purpose, but I'm unsure as to when that's likely to be implemented.

#region Discovery

$ModuleName = 'MyModule'

#endregion Discovery

BeforeAll {
    $ModuleName = 'MyModule'
    Import-Module $ModuleName
}

Describe "$ModuleName Sanity Tests - Help Content" -Tags 'Module' {

    #region Discovery

    # The module will need to be imported during Discovery since we're using it to generate test cases / Context blocks
    Import-Module $ModuleName

    $ShouldProcessParameters = 'WhatIf', 'Confirm'

    # Generate command list for generating Context / TestCases
    $Module = Get-Module $ModuleName
    $CommandList = @(
        $Module.ExportedFunctions.Keys
        $Module.ExportedCmdlets.Keys
    )

    #endregion Discovery

    foreach ($Command in $CommandList) {
        Context "$Command - Help Content" {

            #region Discovery

            $Help = @{ Help = Get-Help -Name $Command -Full | Select-Object -Property * }
            $Parameters = Get-Help -Name $Command -Parameter * -ErrorAction Ignore |
                Where-Object { $_.Name -and $_.Name -notin $ShouldProcessParameters } |
                ForEach-Object {
                    @{
                        Name        = $_.name
                        Description = $_.Description.Text
                    }
                }
            $Ast = @{
                # Ast will be $null if the command is a compiled cmdlet
                Ast        = (Get-Content -Path "function:/$Command" -ErrorAction Ignore).Ast
                Parameters = $Parameters
            }
            $Examples = $Help.Help.Examples.Example | ForEach-Object { @{ Example = $_ } }

            #endregion Discovery

            It "has help content for $Command" -TestCases $Help {
                $Help | Should -Not -BeNullOrEmpty
            }

            It "contains a synopsis for $Command" -TestCases $Help {
                $Help.Synopsis | Should -Not -BeNullOrEmpty
            }

            It "contains a description for $Command" -TestCases $Help {
                $Help.Description | Should -Not -BeNullOrEmpty
            }

            It "lists the function author in the Notes section for $Command" -TestCases $Help {
                $Notes = $Help.AlertSet.Alert.Text -split '\n'
                $Notes[0].Trim() | Should -BeLike "Author: *"
            }

            # This will be skipped for compiled commands ($Ast.Ast will be $null)
            It "has a help entry for all parameters of $Command" -TestCases $Ast -Skip:(-not ($Parameters -and $Ast.Ast)) {
                @($Parameters).Count | Should -Be $Ast.Body.ParamBlock.Parameters.Count -Because 'the number of parameters in the help should match the number in the function script'
            }

            It "has a description for $Command parameter -<Name>" -TestCases $Parameters -Skip:(-not $Parameters) {
                $Description | Should -Not -BeNullOrEmpty -Because "parameter $Name should have a description"
            }

            It "has at least one usage example for $Command" -TestCases $Help {
                $Help.Examples.Example.Code.Count | Should -BeGreaterOrEqual 1
            }

            It "lists a description for $Command example: <Title>" -TestCases $Examples {
                $Example.Remarks | Should -Not -BeNullOrEmpty -Because "example $($Example.Title) should have a description!"
            }
        }
    }
}

ScriptBlocks and GetNewClosure()

2020-05-30T00:00:00-04:00

Before we even begin, let's just take a moment to remember something very important. In PowerShell, everything is an object. Although you'll normally see scriptblocks used to declare functions, or perhaps as a command parameter, and so on… they are themselves just an object.

That means you can do funny things like just arbitrarily assigning them to variables so you can use them later:

$path = 'C:\'

$myScript = {
    Get-ChildItem -Path $path
}

& $myScript

$path = 'C:\Users'
& $myScript

If you run that code, you'll some some interesting results. The $path variable is never defined in the scriptblock, so it actually retrieves the variable from the parent scope when it's invoked. Handy! Well, sometimes, anyway. This is how scriptblocks tend to work for the most part, but it's not always what you want.

Capturing State

Let's take a look at how we might want to process a JSON blob into a more usable PowerShell object. Special thanks to user @MTG on the PowerShell Slack workspace for the fantastic test case here. Our JSON blob looks a bit like this:

{
    "name":  "server1",
    "id":  28361,
    "active":  true,
    "items":  [
        {
            "itemId":  71418,
            "itemValue":  "dk09230",
            "fieldId":  320,
            "fieldName":  "servicetag",
            "fieldDescription":  ""
        },
        {
            "itemId":  71318,
            "itemValue":  "city",
            "fieldId":  300,
            "fieldName":  "location",
            "fieldDescription":  ""
        }
    ]
}

So, we're looking to get out a flat object, which contains the following properties:

name
id
active
All the "items" in items as their own individual properties, where fieldName is the property name, and itemValue is the property value.

There's probably a decent number of ways to handle this, but it's difficult to handle it well if you have a handful of these JSON objects, and everything has a different set of items that you need as individual properties. Without hard-coding handling for each type of item you might happen across in the real world, how do you (reasonably neatly) handle the variant properties you might be getting back here?

There are probably quite a few rabbit holes we could go down to help sort this out, and I'm sure somewhere in there we'd have some folks using Invoke-Expression sooner or later. I'd rather not, so let's take a bit of a look at what scriptblocks can do for us.

Returning to our earlier PowerShell example, let's see how we can modify the scriptblock behaviour a bit to make things work for us.

`GetNewClosure()`

$path = 'C:\'

$myScript = {
    Get-ChildItem -Path $path
}.GetNewClosure()

& $myScript

$path = 'C:\Users'
& $myScript

If you try to run this code, you'll something a bit interesting. Unlike the first example, where changing the value of $path before invoking the scriptblock the second time changes the output, this scriptblock does the exact same things both times. Ignoring the changed value of $path completely and just using the original value that it had when the scriptblock was created. More properly, the value as it was when the GetNewClosure() method was used to create a copy of the scriptblock (we're discarding the original un-enclosed copy of the scriptblock in this instance).

That doesn't look super useful here, but it will help us, since it allows us to use the same code to create a collection of scriptblocks that behave slightly differently. Here's how I ended up framing the code:

$object = $json | ConvertFrom-Json

$Properties = @(
    'Name'
    'Id'
    'Active'
    foreach ($field in $object.items) {
        @{ Name = $field.fieldName; Expression = { $field.itemValue }.GetNewClosure() }
    }
)

$object | Select-Object -Property $Properties

To explain a little… If you weren't aware, Select-Object takes strings, scriptblocks, and/or hashtables as input for -Property, and I'm building the collection of properties ahead of time here. That lets me use slightly neater syntax, but if you really wanted to you could pass that array literal @( ... ) directly to the command as well. First, we simply add the direct properties we want from the original object as strings. Then, using a foreach loop I can build the hashtables necessary to add a calculated property to the final object for every item that was attached to the original object.

The GetNewClosure() here lets us use the value of $field in the scriptblock and have it refer to the correct item from the original object by capturing the state of $field when it was created and always using that. This is important, because without it, Select-Object will just invoke that scriptblock as-is, and it will always be referring to the same $field value for each of the properties, which is not what we want.

There might be other ways to handle this, but probably not many ways that are quite this elegant. The only one that really comes to mind is perhaps looping over $object.items and repeatedly calling Select-Object… which is much less readable, and probably a lot slower overall.

It's not often I see a good use case for it, but I'm always glad I'm aware of it when I need it!

Handling PowerShell's Versatile Subexpressions

2020-02-24T00:00:00-05:00

I've always been a fan of using subexpressions to tidy up my code. They tend to help me collect and store output where I need it, without requiring that I add complex loops and delve into .NET collection types. The pattern seems to be a bit of a rarity among a lot of other scripters I see, so I thought I'd write a post on subexpressions and all the ways you can use them!

I'll also be covering use of scriptblocks as anonymous functions — invoking them immediately as you define them — since that's rather similar to how subexpressions work.

Subexpressions and Anonymous Functions

Subexpressions are kind of like an inline function that always gets invoked immediately. You define a set of commands that all get invoked one after another, and then PowerShell processes all the output and either stores or outputs the results. The only thing you can't do with them is store the commands inside and execute them later. A subexpression is always evaluated at the point in the script where it's declared.

True anonymous functions in PowerShell are also available in the form of inline scriptblock invocations. If you choose, you can actually store the scriptblock itself — the function definition, essentially — and invoke it multiple times.

Something to note that's shared between all subexpressions (as well as scriptblocks) is that the end of a line does not terminate the subexpression. Only the matching closing parenthesis closes off the subexpression. This actually includes explicit line endings with ; as well, and this is why ( ) is not a true subexpression:

# This line will throw a parse error; the expression ends before the closing parenthesis
$Values = ( Get-Item -Path $Path; Get-ChildItem -Path $Path )

# Conversely, this line is perfectly fine; the subexpression allows the use of line endings within it
$Values = $( Get-Item -Path $Path; Get-ChildItem -Path $Path )

Let's take a closer look at the types of subexpression PowerShell has available.

Types of Subexpressions

Without lumping in anonymous functions as a kind of subexpression (which they sort of are, but we'll look at that in a minute), there are two types of subexpression in PowerShell:

Syntax	Name	Example
`$( )`	Subexpression	`$String = "This script has $(Get-Random) functions."`
`@( )`	Array Subexpression	`$Values = @(1, 2, 3, 4)`

Regular Subexpressions

I think most folks will typically see subexpressions used to insert data into a string, just like in the above example. This is a great use for it, and it's a function exclusive to this type of subexpression.

But it's far from all you can do with it. Notice I mentioned a moment ago that you can have line breaks in subexpressions. That opens up a few interesting possibilities here, especially because each line is considered a totally separate statement:

$SomeValue = 4
$Values = $(
    Test-Connection 1.1.1.1

    if ($SomeValue -lt 10) {
        10
    }
    else {
        15
    }

    1..20
)

What are you looking at me like that for? Yes, it's perfectly legal (and quite useful on occasion!) syntax to drop an entire statement block into a subexpression. Ifs, switches, loops, you name it. Pretty much any language statement is a legal entry in a subexpression, and though I'd advise against it you can do odd things like embedding an entire if statement into a string. More to the point, though, what kind of result would you expect to find in $Values when all's said and done?

It might not be immediately obvious if you're not accustomed to seeing this, but you'll get a single flat array stored in $Values when the subexpression completes. Each statement within the subexpression executes in order and submits its output to the pipeline.

For Test-Connection (at least in PowerShell 6.2+) this means that 4 PingStatus objects are output and subsequently stored into $Values.
The if/else statement is evaluated, and the value 10 is output and subsequently stored.
The expression 1..20 is evaluated, and an array with values 1 through 20 are output. Note that this output is still over a pipeline, so the array is ultimately split up again and sent to output one item at a time. This will become important later.

So, our end result, if we'd written it all out manually, would look a bit like this:

# Final array size: 25
$Values = [object[]]::new(25)

# First four values -> Test-Connection Output (if using PS Core 6.2.x / 7)
$Values[0..3] = Test-Connection 1.1.1.1

# Fifth value -> result of the if statement
$Values[4] = if ($SomeValue -lt 10) { 10 } else { 15 }

# Remaining values -> contents of array
$Values[5..24] = 1..20

Notice that it doesn't matter here whether we drop a scalar value or an array value into the subexpression. As long as we leave a line-break between our expressions, each array is unwrapped as it goes over the pipeline, and then all the results are tossed into a single array at the end. This is part of the essentials of pipeline behaviour, but I'm sure quite a few folks won't expect to see any pipeline behaviour arise from the above example.

💡 Remember

Every statement in PowerShell that doesn't itself store or redirect output elsewhere uses the standard output pipeline. This includes statements within subexpressions.

And yes, before you ask… all of this still applies when you have a subexpression inside a string. The only additional thing to note in that case is that you're always going to end up converting to a string value at the end. This can occasionally cause some unintended behaviour, since the behaviour when converting an array to a string depends on the value of $OFS. Usually $OFS is unset, and a space is used to separate the items in the array. if you set $OFS to, say, a comma, this behaviour changes.

PS> "The values are $(1..10)."
The values are 1 2 3 4 5 6 7 8 9 10

PS> $OFS = ','
PS> "The values are $(1..10)."
The values are 1,2,3,4,5,6,7,8,9,10.

Just one little detail to keep in mind. 🙂

Array Subexpressions

Array subexpressions work almost exactly the same was as regular subexpressions. They have two key differences:

You cannot directly insert an array subexpression into a string.
The result of an array subexpression is always (as the name implies) an array. (Specifically, it will always be an [object[]] array.)

📝 Note

Array subexpressions will only wrap the output in an array if the output is not already an array. In other words, it guarantees the output will ultimately be an [object[]] array. If the result is a single item, it will be boxed into an [object[]] array, and if it's an array of another type, it will box each item and hand back an object[] array. In most cases, PowerShell will already have boxed the output as [object[]] due to its pipeline processing.

In every other way, they work identically to standard subexpressions. You'll most often see these used as an "explicit" variant of defining an array.

$array = @( 1, 2, 3, 4, 5 )

While this can be used for clarity, it is in effect somewhat redundant. The , comma operator is what's actually creating the array in this instance. The subexpression operator simply ensures the result is in fact an array.

Inline Scriptblock Invocations

Scriptblocks can be defined and used directly in a script, much like a subexpression. Unlike subexpressions, however, scriptblocks are usually used to store code — they won't be invoked unless you use one of the invocation operators. A scriptblock, as the name implies, is just a block of script. Any valid PowerShell code is valid here, with only a scant few exceptions.

# `&` or `.` must be used to invoke the scriptblock
$Results = & {
    $folders = Get-ChildItem -Directory
    $folders | Get-ChildItem -File
}

This has a similar effect to using a subexpression. However, using a scriptblock creates a new scope — any new variables or other scoped data will not be available outside the scriptblock. In the above example, the $folders variable becomes unavailable as soon as you step outside the scriptblock once again.

The other option is to dot-source the scriptblock.

$Results = . {
    $folders = Get-ChildItem -Directory
    $folders | Get-ChildItem -File
}

In this instance, both the $Results variable and the $folders variable would become available in the same scope after the scriptblock finishes executing.

ℹ Note

Dot-sourcing a scriptblock is more expensive in terms of performance than simply invoking it. Generally speaking, invoking the scriptblock should be the typical case, and dot-sourcing should be an exception to the rule.

`{}.GetNewClosure()`

Something to note about scriptblocks is that you can store them for later invocation, like an impromptu function. Those of you coming from a C# background are probably already familiar with lambda expressions — this is something similar.

$Value = 12
$Script = {
    $Value + 12
}

$Value = & $Script
$Value # output: 24

& $Script # output: 36

As you can see, although the scriptblock is indeed a stored action, its effect still depends on the code around it. In some cases, you won't want this context-sensitive behaviour as much. For these cases, you can call .GetNewClosure() on the scriptblock to effectively tie it to the context where it was defined.

If we apply that to the above example, you can see it in action:

$Value = 12
$Script = {
    $Value + 12
}.GetNewClosure()

$Value = & $Script
$Value # output: 24

& $Script # output: 24

That is, no matter where you call the scriptblock from, it behaves the same way. It's hard to see how this is useful here, but it comes in very handy if you ever need to handle events in PowerShell.

Events in C# are pretty easy to work with. In PowerShell, they can be problematic, since there's no guarantee an event will be invoked in the same context that it was registered in. It can very easily end up being called from a completely separate thread. This means that the scriptblock has to be effectively self-contained — if you need to reference anything that's defined or created outside the scriptblock itself, GetNewClosure() comes in very handy.

function Get-Collection {
    [CmdletBinding()]
    param() end {
        $collection = [System.Management.Automation.PSDataCollection[int]]::new()
        $sb = {
            if ($null -eq $collection) {
                [Console]::WriteLine("COLLECTION IS NULL OH NO")
                return
            }

            [Console]::WriteLine("Collection length is now $($collection.Count + 1).")
        }.GetNewClosure()

        $collection.add_DataAdding($sb)
        $PSCmdlet.WriteObject($collection, $false)
    }
}

$col = Get-Collection
$col.Add(10)

Credit to Patrick Meinecke for the lovely example here.

Essentially what's going on here is we're creating a PSDataCollection, and then registering a scriptblock to its DataAdding event. This event triggers whenever you add items to the collection. Here, GetNewClosure() allows the scriptblock to reference its own parent collection directly, without fear that the $collection variable won't exist when the event fires.

Calling .Add() causes the event to fire as the new data is added, and the reference to $collection is maintained. You won't always need this capability, but it's a very handy tool to keep on hand for when you do need it!

Putting it All Together

I've often found that this kind of thing is best illustrated by example, so I'll try to make this part a bit more on the side of showing than telling.

Examples

Generating a List of Arguments for a Native Command

$command = 'docker'
$args = @(
    'rm'
    if ($Force) {
        '-f'
    }
    $Name
)

& $command @args | Out-Null

In this example, we're invoking an rm command with Docker, and we want to wrap it in a PowerShell function. We can expose a -Force parameter that it respects, and altering the arguments is as simple as adding the -f argument or not. Rather than complicate things by trying to add it in later, or duplicating the code completely and having two paths (one with -f and one without), we can just add an if statement directly in the subexpression.

Personally, I prefer this kind of methodology over manually manipulating strings any day, especially since that usually leads to Invoke-Expression.

Creating Match Patterns

$DomainPattern = @(
    'domain.com'
    'domain.co.uk'
    'domain.de'
).ForEach{ [regex]::Escape($_) } -join '|'

Get-ExoMailbox -ResultSize Unlimited | Where-Object UserPrincipalName -match $DomainPattern

Here, we put together an array of domain names, call [regex]::Escape() on each of them so we can use them as literals in a regex expression, and then join on | which is a logical OR separator in regex. The end result is that we can filter by the regular expression and get any of the objects where the UserPrincipalName contains any of the designated domain names. It's a great example of when you can use the multiline syntax for an array subexpression. Using a multi-line statement is perfectly optional here; the above $DomainPattern declaration is identical to the following:

$DomainPattern = @('domain.com', 'domain.co.uk', 'domain.de').ForEach{ [regex]::Escape($_) } -join '|'

There are a couple of reasons I tend to prefer the multiline syntax:

Shorter line lengths tend to be a bit easier for me to read; I prefer to scan primarily down a script rather than both down and across.
Breaking the statement over a couple of lines actually makes collaborative coding easier. In a line-by-line diff (the sort you'll see with Git and also with pull requests on Github and similar platforms), the multiline version is much easier on the eyes when you're adding or removing items from the array.

Collating Output from Multiple Sources

$FileList = @(
    Get-ChildItem -Path $Dir1 -File
    Get-ChildItem -Path $Dir2 -File -Recurse
)

foreach ($File in $FileList) {
    Rename-Item -Path $File.FullName -NewName "$($File.Name).old"
}

Here we're able to collate the output from multiple discrete commands and put them into a single array. This example is a little contrived, since you can do a similar thing in most cases just by supplying multiple items to -Path. However, if you ever have a case where you need to search through each path a little differently (using different filters, for example), this can come in handy.

I also quite like that you can effectively join two separate pipelines together and continue them in a third pipeline!

$(
    Get-ChildItem -Path $Dir1 -File | Where-Object Name -notmatch '\d'
    Get-ChildItem -Path $Dir2 -Recurse -File | Where-Object Length -gt 4kb
) |
    Sort-Object -Property Length |
    Select-Object -Property Name, Length, FullName |
    Export-Csv -Path $CsvFile

The syntax can at times be a bit awkward, so feel free to play with it a little until it looks somewhat acceptable to you. I haven't needed this more than once or twice, but I rather like the idea that I can sort of funnel many things into a single pipeline if I need to.

Familiarity

I think a lot of users will likely be confused at first if they see a subexpression pushed to its limits. You can go quite a ways into the weeds with them if you really have a need to.

$List = 1..20

$Items = @(
    $Assertions = switch ($List) {
        { $_ % 3 -eq 0 } {
            "$_ is divisible by 3"
        }
        { $_ / 2 -gt 8 } {
            "$_ is more than 16"
        }
        7 {
            "Lucky day!"
        }
        default {
            "Today's not very lucky for you..."
        }
    }

    "Your Lucky Numbers are..."
    $(
        1..50 | Get-Random -Count (Get-Random -Min 1 -Max 10)
    ) -join ', '

    $Assertions | Get-Random

    # Push any errors back to the regular output stream here
    & { Resolve-DnsName google.com } 2>&1
)

$Items -join ' | '

Would you ever need to? Probably not. Clearly this is quite a contrived example, but it serves to show just how much you can throw into a single subexpression. Yes, if you wanted to, you could use $() the same way @() is used here, if you needed to.

Sometimes you'll find the alternative is significantly longer, more convoluted, and quite a bit slower. There are advantages to be had in being able to contain so many different constructs inside a single statement, from time to time.

I would definitely be wary of trying to encapsulate too many complicated statements into a single subexpression, to be sure. Knowing it's possible just ensures you have the option open to you, should you ever need it.

Building Arrays and Collections in PowerShell

2020-02-15T00:00:00-05:00

Well, it's certainly been a while since I posted on here. Trying to get back into the swing of things for 2020 now that I'm mostly settled in at my new job. So, let's take a look at a classic problem that never fails to confuse folks just getting into PowerShell. I've also seen plenty of more experienced folks trip over it quite a lot, so here we are.

Before we begin, many thanks to Chris Dent for asking me to put this together and the basis for the recommendations at the end!

Background Information
- Background - Collections in .NET
- ICollection
Collections in PowerShell
- Common Patterns
- Working With Collections
  - New-Object v.s. ::new() v.s. Casting
  - Adding Many Items to an Existing Collection
General Recommendations

Background Information

The term collection is often bandied about, so let's define it concretely before we get started, shall we? In general parlance, a collection is generally considered to be a pretty broad term. In its simplest usage, it just refers to a set of individual items. Oftentimes, these items may be related to one another in some way.

Background - Collections in .NET

In the .NET ecosystem, the term collection has a pretty well-defined meaning. A "collection" is any type of object that implements the ICollection interface. Seems a bit circular, right? Let's look at what that actually means and take a quick refresher on what an interface actually is in .NET for those of you who're not familiar. From the C# Programming Guide's page on interfaces, we get this definition:

An interface contains definitions for a group of related functionalities that a non-abstract class or a struct must implement.

By using interfaces, you can, for example, include behavior from multiple sources in a class.

That's a bit of a wordy definition, but it does the job. In practical terms, an interface is a way of defining a required set of properties or methods that must be defined by any other type that wants to inherit from or implement this interface. It's a way of being able to define standard ways of interacting with more than one kind of object.

ICollection

ICollection is a fairly simple interface that has a few demands:

Methods
- CopyTo(Array array, int index)
Properties
- Count
- IsSynchronized
- SyncRoot

Not a whole lot, huh? So for a type to be an ICollection, it needs to guarantee these things:

It has a CopyTo() method that you can use to copy its items into an array of your choosing starting from the requested index.
It has a Count property that will give you the current number of items in that collection.
It has the IsSynchronized and SyncRoot properties. For collections that utilise this functionality, these can be used to determine whether a collection's state is synchronised across threads. One example of a collection that allows you to create synchronised versions, you can check out Hashtable and its static method Synchronized().

Fairly straightforward on the whole, I suppose. Interfaces can get quite complex, and there are two main extensions of ICollection in use in .NET Core: IList and IDictionary IList types represent a "flat" collection, where it's simply an ordered list of items. On the other side of things, IDictionary represents a less-strictly-ordered set of key/value pairs where a given key (often a string such as a name) is used to access the relevant item directly.

Collections in PowerShell

The main type of collection in use in PowerShell is the humble Array, at least in terms of what is often available directly from a script. Behind the murky curtain, PowerShell actually utilises the System.Collections.ArrayList type quite heavily in its pipeline processor.

Partially as a result of that, arrays (and more specifically object[]) are the backbone of PowerShell scripting. In the vast majority of cases, you're going to be handling arrays unless you deliberately choose to use another type of collection to handle output. Let's take a look at some fairly common methods of handling collections of items in PowerShell, shall we?

Common Patterns

Use of `+=` to Build Arrays

$array = @()
foreach ($value in 1..5) {
    $array += $value
}

Seems pretty simple and sound, at least at first glance. So… spot anything amiss?

The pattern here is "create the collection object, and then add each item to it one at a time". This works quite well for some collection types, but not so well for others. In fact, for arrays specifically it's technically not possible to do this — arrays are actually fixed-size collections. If you call $array.Add(10) you'll get an error stating as much. But this still works, somehow, and it's all thanks to some ✨ PowerShell magic! ✨

ℹ Behind the Scenes

PowerShell's + and += operators are designed to work with arrays in a relatively unusual way. When you try to add items to an array like this, what actually happens goes something like this:

PowerShell checks the size of the collection in $array and the number of items being added to it (in this case, just one each time).

PowerShell creates a completely different array of the correct size.

The original array is copied into this new array, along with the new item(s).

This is also why it's perfectly possible to join two arrays together with the + or += operators.

What's Wrong with this Picture

The missing piece here, and "what's wrong" in some ways is that this operation can become prohibitively expensive. For smaller arrays, it's not a concern at all. The main problem is that we won't always know ahead of time how big our collection might become.

If $array is quite large (say somewhere in the realm of 5,000-10,000 items), then generating new arrays every single time you add just a couple of items becomes very expensive to do. A new array, even if it's empty, requires allocation of memory so that the array has enough space to exist in a single region of memory. Adding a single item to a 10,000 item array means reserving a new block of memory, enough to hold another 10,001 items, then copying everything across.

Notice that I didn't mention anything about deleting the original array. This is because PowerShell can't necessarily assume you're definitely wanting to completely remove the original array. You might have another variable storing a reference to that same array in a completely different scope, after all. Checking for that every time you want to add things to an array would also be prohibitively expensive.

So what happens, then? The original array will sit around in memory until the .NET garbage collection routines realize that it's not needed by anything anymore. In some cases, that can mean it sits around for quite a while before that memory is freed up again.

For smaller collections, of course, this isn't really an issue at all. However, this is a very easy pattern to fall into, so it can come back to bite you when you least expect it. Let's look at a few other ways to approach this task.

Using an Expandable List

.NET contains quite a few types of lists. If you look at the types that implement IList you'll see there are quite a few available options. Oftentimes older blog posts or code examples will have ArrayList as their collection of choice. Personally, I've never liked using ArrayList directly. As I mentioned above, PowerShell uses it behind the scenes extensively — but I'd be willing to bet that if they had the time available, more than a few people on the PowerShell team would love to replace it with something more modern. I know I would; ArrayList comes from the very old days of .NET, hailing all the way back to .NET v1 if I recall correctly.

In fact, the documentation page for ArrayList warns against its continued use:

ℹ Important

We don't recommend that you use the ArrayList class for new development. Instead, we recommend that you use the generic List<T> class. The ArrayList class is designed to hold heterogeneous collections of objects. However, it does not always offer the best performance. Instead, we recommend the following:

For a heterogeneous collection of objects, use the List<Object> (in C#) or List(Of Object) (in Visual Basic) type.

For a homogeneous collection of objects, use the List<T> class. See Performance Considerations in the List<T> reference topic for a discussion of the relative performance of these classes. See Non-generic collections shouldn't be used on GitHub for general information on the use of generic instead of non-generic collection types.

So, for completeness' sake, I'll include an example of what using an ArrayList looks like — but I wouldn't recommend ever using it.

# There are multiple ways to create the original ArrayList, but this is one of the simplest and easiest to use.
[System.Collections.ArrayList]$list = @()
foreach ($value in 1..5) {
    # Redirection to null is necessary as ArrayList outputs an index number for each added item.
    $list.Add($value) > $null
}

So, let's say we follow that recommendation from the ArrayList documentation, and want to use a generic List instead. What even is List<T> anyway? List<T> is a generic type, which in simple terms just means it has a bit of a fluid definition. You can define a List<T> in terms of what type of object you want it to work with or (in the case of List, at least) store, where the T is the type of object you're storing in it. For a list comprised entirely of numbers, we might use List<int> to store them.

Why would we use this over ArrayList? In PowerShell there isn't a massive difference here, but for me personally it comes down to a few things.

You can create a List that only stores a specific type, which means you can catch mistakes earlier. Adding other types of items to that list will either have them converted to the List's designated type, or will throw an error stating that it was the wrong type.
Using the Add() method on List<T> doesn't create incidental output like ArrayList's Add() method does, so you have less chance to miss things and pollute your output stream.
Given the assertion in the documentation, no less, that ArrayList is recommended against, I would assume two things:
- There is the possibility at any point that the .NET / .NET Core team will eventually deprecate it completely and possibly even remove it from .NET at that point or afterwards.
- No further work is being done on ArrayList, and if there are any improvements to be had you will likely only find them in List<T>.

Given that, I'd choose to use List<T> instead of ArrayList in every single case. When using it, there's one question you need to ask before using it:

What kind of objects am I storing in this list?

If you know ahead of time that you plan on storing more than one type of item in the list, you have two choices:

Use a common parent type or interface, for example System.Exception is the parent type of any Exception-typed object in .NET, so using that base type allows you to store any other Exception object.
Use the object type, which is the parent type for everything in .NET.

In our previous examples, we're simply storing numbers. Let's see how that looks with a generic List, shall we?

[System.Collections.Generic.List[int]]$list = @()
foreach ($value in 1..5) {
    # No redirection needed here, List<T> does not emit data from Add()
    $list.Add($value)
}

That certainly looks a little tidier. The long type name is a bit much at times, but at least in PowerShell 5.1 and up, you can use using namespace System.Collections.Generic to let you just use [List[int]] if you prefer.

📝 Note

While the .NET documentation frequently uses the C# syntax List<T>, when we get to PowerShell we actually use square brackets to indicate the generic type parameter instead. List<T> instead becomes [List[T]], and in practical usage T is replaced with another type name. This is purely a syntactical difference between C# and PowerShell — in both cases we're referring to the same .NET type. In F# you'd write List<'T> and in Visual Basic it looks instead like List(Of T).

Why Use a List

Lists are fantastic for cases where you don't know the collection size ahead of time. If you knew the size ahead of time, you can simply pre-allocate an array of the correct type and size anyway:

$array = [int[]]::new(5)
for ($index = 0; $index -lt 5; $index++) {
    $array[$index] = $index * 2
}

In PowerShell, they're best used in cases where you need to be adding to the collection from multiple places in your code, and you want the order of items to stay the same as the order you put them into the collection. They're also the most effective way to build multiple collections in a single loop statement, if that's something you need to do.

Using the Pipeline

I mentioned earlier that PowerShell already heavily uses ArrayList in its pipeline processor. We can actually take advantage of this without ever needing to use ArrayList ourselves directly. This style of building collections looks a lot more like coding in a functional language than the other options, so it may confuse some folks who aren't accustomed to coding this way. However, it's perfectly valid and quite flexible in nature.

The above examples coded this way would look a little more like this:

$array = foreach ($value in 1..5) {
    # This value is not stored anywhere and becomes part of the "output" from the loop statement.
    $value + 4
}

If you're not familiar with this kind of pattern, you're probably wondering how on Earth that even works. In PowerShell, unlike most .NET languages (and indeed many other programming languages in general), allows you to assign a variable to the "result" of a statement, regardless of whether that statement is a keyword-based loop or a complete pipeline. Any uncaptured output will then be funneled into that variable and stored as [object[]] (an array of objects) in the variable.

Because this utilises the built-in PowerShell pipeline (which, remember, uses ArrayList to handle output), there is no additional overhead of creating another type of collection here. We're just using what PowerShell has always been ready to provide for us. This is also one of the simplest ways to keep your code highly compatible across PowerShell versions with minimal effort.

ℹ Compatibility

This pattern works with all released versions of PowerShell, even back to version 1. For compatibility, it's the absolute best option. And thankfully, it's one of the most effective options as well!

This pattern works with all PowerShell control-flow statements, which include switch, foreach, for, do..until, do..while, while, and try..catch..finally (and there's probably a couple I forgot to mention, but will still work as well!) It also works seamlessly with pipelines, no matter how lengthy they get:

$pictures = Get-ChildItem -Filter *.png -Recurse -File |
    Where-Object Length -gt 1MB |
    Select-Object -Property Name, Length, FullName

Working With Collections

`New-Object` v.s. `::new()` v.s. Casting

This is more of a general point to make about creating new objects in PowerShell, but if you're not aware it bears mention here too. New-Object is, not to put too fine a point on it, quite slow. At least compared to alternatives.

Syntax like $list = [System.Collections.Generic.List[int]]::new() is fairly new, first available in PowerShell 5.1. If compatibility is your main concern, ::new() is not the first choice, to be sure. However, New-Object should still be the last choice, generally being reserved for COM objects and similar otherwise difficult to construct items.

Syntax $list = [System.Collections.Generic.List[int]]@() or [System.Collections.Generic.List[int]] $list = @() will work all the way back to version 2 of PowerShell. For a compromise between compatibility and speed I'd go for one of those.

Adding Many Items to an Existing Collection

If you have a List collection and want to add multiple items to it quickly, rather than using any kind of loop you can actually add them in directly. List<T> has an AddRange() method that allows you to add multiple items at once:

$List = [System.Collections.Generic.List[object]]@(1..5)
$List.AddRange(5..10)

# Note - for strongly-typed lists you will need to cast the array you're adding to the relevant array type explicitly:

$List = [System.Collections.Generic.List[int]]@(1..5)
$List.AddRange([int[]](5..10))

General Recommendations

Taking all the above into account, we can establish some broad recommendations when handling collections of items in PowerShell.

If you can simply assign the statement to a variable to build your collection, do so. It's the most compatible approach and one of the simplest ways to handle it.
If for some reason you cannot assign the statement or need to build multiple collections in the same loop, use a List<T>.
If you have a scenario where you need to add multiple items to a collection, use a List<T> and make short work of it with AddRange().

I went into this post thinking it'd be short and sweet, but I really can't help but explain things to the nth degree, can I? 😂

That's all I have for now, I think. If you have any questions or comments, feel free to leave a comment below.

I'm also keeping an eye out for some more content to cover to get back into regular blog posts, so if there's something I haven't already covered that you're needing some explanation on and think I can help, give me a shout on Twitter or one of the community channels. Have a great day! 😊

Working with PowerShell's -replace Operator

2019-03-20T00:00:00-04:00

In case you couldn't already tell, I rather like playing around with regex in PowerShell. There's something very enjoyable about poking it in just the right way to get it to do exactly what you want. I've already talked a bit about how we can use it to create PSCustomObjects.

Working with `-replace`

-replace is a very handy operator to have quick access to in PowerShell. In its most basic usage, it allows you to swap out bits and pieces of text, or to remove unwanted pieces from a string.

PS> $string = 'Glass half empty, glass half full.'
PS> $string -replace 'glass','cup'
cup half empty, cup half full.
PS> $string -replace '[^a-z]'
Glasshalfemptyglasshalffull

Using Match Groups

With the use of parentheses to create a match group, -replace becomes even more powerful. Stripping out everything from a string except the one or two things we want is extremely simple.

PS> $string = 'You lost your mind in the sound.'
PS> $string -replace '.*(\w+)\.','$1'
sound

There are two things to note in the above example. First of all, that pattern would match the entire string, so it would normally just remove the string. Second, we don't get the characters $1 back in our result.

While that syntax might look familiar to you from PowerShell's variables, don't be fooled. This isn't PowerShell, at least not where it counts here. PowerShell isn't processing that $ symbol, and we don't want it to; that's why I used single quotes to create a literal string.

Instead, this string is given to the underlying regex method, which does recognise it as a variable. In this context, that variable refers to "the first match group". Why is it 1-indexed, you might ask, when literally everything else in PowerShell is 0-indexed? Because $0 is quietly already taken.

You might recall from my previous post on regex that PowerShell has a $matches variable, where $matches[0] refers to "the complete matched string". It's the same thing here.

PS> 'hello world' -replace '(\w+) \w+','$1 $0'
hello hello world

Note that the match group is respected, pulling the first word, and then the entire match is retrieved again and reinserted. As you can imagine, this can be made incredibly complex and strange very quickly. With a bit of tinkering and a careful eye, you can do some wonderful stuff with -replace!

Named Match Groups

Bit of a theme with me, isn't it? I love anything that can lend a little extra verbosity to a complicated regex pattern, a bit of self-documentation, if you will.

I was tinkering around with the above syntaxes for -replace (not $0, though, I hadn't realised that part until I started writing this post!) the other week, and got to wondering. Is there a syntax for inserting named match groups in a replace string? It stands to reason that there ought to be, doesn't it? Nobody I asked knew a way, but then again, nobody could say for sure that it couldn't be done, either.

So, I got to tinkering. My first attempt looked a bit like this:

PS> $string = 'Hello darkness, my old friend.'
PS> $string -replace '(hello) (?<Item>Darkness)','$Item says $1'
$Item says Hello, my old friend.

Hmm. Okay, not quite there. As expected, the unnamed group is noticed, but the named group is ignored. A little bit more fiddling with the syntax, and I landed on the correct way to do it quite by accident.

PS> $string -replace '(hello) (?<Item>Darkness)','${Item} says $1'
darkness says Hello, my old friend.

Yep. That worked, very nicely.

So just remember: the temptation to obscure your code with regex is always there. But you have the tools available to make it that crucial bit more self-documenting, provided you name the bits and pieces effectively. A well-named match string makes it entirely unnecessary to have those four extra lines of comments attempting to explain what the heck is going on.

Summary

When using -replace on a string, the following tokens hold special meaning in the replacement string.

Symbol	Meaning
`$0`	References the complete matched pattern.
`$1` - `$n`	References the unnamed match group from the pattern string, starting from `$1`.
`${Name}`	References the specific named match group from the pattern string.

Keep your code clear and clean. It'll save you a great many headaches, especially when dealing with regex!

Error Handling in PowerShell - Best Practices

2019-01-31T00:00:00-05:00

There are a few different kinds of errors in PowerShell, and it can be a little bit of a minefield on occasion. There are always two sides to consider, too: how you write code that creates errors, and how you handle those errors in your own code. Let's have a look at some ways to effectively utilise the different kinds of errors you can work with in PowerShell, and how to handle them.

Different Types of Errors

Terminating Errors

Terminating errors are mostly an "exactly what it sounds like" package in general use. They're very similar to C#'s Exception system, although PowerShell uses ErrorRecord as its base error object. Their common characteristics are as follows:

Trigger try/catch blocks
Return control to the caller, along with the exception, if they're not handled using a try/catch block.

There are a couple different ways to get a terminating error in PowerShell, and each has its own small differences that make them more useful in a tricky situation.

`Throw`

First is your typical throw statement, which simply takes an exception message as its only input.

throw "Something went wrong"

throw creates an exception and an ErrorRecord (PowerShell's more detailed version of an exception) in a single, simple statement, and wraps them together. Its exceptions display the entered error message, a generic RuntimeException, and the errorID matches the entered exception message. The line number and position displayed from a throw error message point back to the throw statement itself. throw exceptions are affected by -ErrorAction parameters of their advanced functions.

throw should generally be used for simple runtime errors, where you need to halt execution, but typically only if you plan to catch it yourself later. There is no guarantee a user will not elect to simply ignore your error with -ErrorAction Ignore and continue with their lives if it is passed up from your functions.

`ThrowTerminatingError()`

Next, we have a more advanced version of the throw statement. ThrowTerminatingError() is a method that comes directly from PowerShell's PSCmdlet .NET class. It is accessible only in advanced functions via the $PSCmdlet variable.

using namespace System.Management.Automation;
$Exception = [Exception]::new("error message")
$ErrorRecord = [System.Management.Automation.ErrorRecord]::new(
    $Exception,
    "errorID",
    [System.Management.Automation.ErrorCategory]::NotSpecified,
    $TargetObject # usually the object that triggered the error, if possible
)
$PSCmdlet.ThrowTerminatingError($ErrorRecord)

As you can see, these require a little more manual labor. You can bypass some of this by deliberately using the throw statement and then later catching the generated ErrorRecord to pass it into the ThrowTerminatingError() method directly. This saves a decent amount of the code involved, and offloads the work to PowerShell's engine instead. However, this will still involve essentially the same work being done, without necessarily giving you a chance to specify the finer details of the error record.

ThrowTerminatingError() creates a terminating error (as the name implies), however unlike throw its errors are unaffected by -ErrorAction parameters applied to the parent function. It will also never reveal the code around it. When you see an error thrown via this method, it will always display only the line where the command was called, never the code within the command that actually threw the error.

ThrowTerminatingError() should generally be used for serious errors, where continuing to process in spite of the error may lead to corrupted data, and you want any specified -ErrorAction parameter to be ignored.

Non-Terminating Errors

Non-terminating errors are PowerShell's concession demanded by the fact that it is a shell that supports pipelines. Not all errors warrant halting all ongoing actions, and only need to inform the user that a minor error occurred, but continue processing the remainder of the items.

Unlike terminating errors, non-terminating errors:

Do not trigger try/catch blocks.
Do not affect the script's control flow.

`Write-Error`

Write-Error is a bit like the non-terminating version of throw, albeit more versatile. It too creates a complete ErrorRecord, which has a NotSpecified category and some predefined Exception and ErrorCode properties that both refer to Write-Error.

This is a very simple and effective way to write a non-terminating error, but it is somewhat limited in its use. While you can override its predefined default exception type, error code, and essentially everything about the generated ErrorRecord, Write-Error is similar to throw in that it too points directly back to the line where Write-Error itself was called. This can often make the error display very unclear, which is rarely a good thing.

Write-Error is a great way to emit straightforward, non-terminating errors, but its tendency to muddy the error display makes it difficult to recommend for any specific uses. However, as it is itself a cmdlet, it is also possible to make it create terminating errors as well, by using the -ErrorAction Stop parameter argument.

WriteError()

WriteError() is the non-terminating cousin of ThrowTerminatingError(), and is also available as one of the members of the automatic $PSCmdlet variable in advanced functions. It behaves similarly to its cousin in that it too hides the code that implements it and instead points back to the point where its parent function was called as the source point of the error. This is often preferable when dealing with expected errors such as garbled input or arguments, or simple "could not find the requested item" errors in some cases.

$Exception = [Exception]::new("error message")
$ErrorRecord = [System.Management.Automation.ErrorRecord]::new(
    $Exception,
    "errorID",
    [System.Management.Automation.ErrorCategory]::NotSpecified,
    $TargetObject # usually the object that triggered the error, if possible
)
$PSCmdlet.WriteError($ErrorRecord)

Emit Errors Responsibly

There are no true hard-and-fast rules here. If one solution works especially well for what you happen to be putting together, don't let me deter you. However, I would urge you to consider both the impact to your users as well as yourself or your team before selecting a single-minded approach to error creation. Not all error scenarios are created equal, so it's very rare that I would recommend using one type over all others.

WriteError() is generally preferred over its cmdlet form Write-Error, because it offers more control over how you emit the error. Both tend to be used for expected and technically avoidable errors that occur, but the $PSCmdlet method has the additional benefit of not divulging implementation details unnecessarily.

If you expect the error to occur sometimes, I would generally advise using WriteError() unless the error's presence significantly impairs your cmdlet's ability to process the rest of its input, in which case I would use ThrowTerminatingError().

throw is generally nice as a quick way to trigger a try/catch block in your own code, but I would also recommend using it for: those truly unexpected errors, that last-ditch catch clause after several more specific catch blocks have been put in place and bypassed. At this point, a final ending catch block that simply triggers a throw $_ call to re-throw the caught exception into the parent scope and quickly pass execution back to the caller. It's a very quick and very effective way to have the error be noticed, but should usually be kept internal to the function more than anything else, except in the last scenario mentioned. Use of Write-Error -ErrorAction Stop may potentially also be substituted for that purpose if you personally prefer.

Thanks for reading!

Playing Around with System.Drawing in PowerShell

2019-01-17T00:00:00-05:00

Through a bit of fiddling with PSWordCloud I've found some rather neat features of System.Drawing that are a little on the weird side, and also not the most discoverable, so I figured I should talk a little about them.

Disclaimers

As I have myself only recently learned, what I've been able to do here, while available in System.Drawing.Common, do not work especially well in Linux or Mac. Tyler has been road testing them for me just a little bit here and there, and it appears that in both Linux and Mac they have a hard dependency on an external Mono library called libgdiplus that does not usually come packaged with PowerShell or .NET Core by default.

Even after this library is installed:

On Linux, it appears that attempting to instantiate the System.Drawing.Region class completely crashes PowerShell.
On Mac, it seems to work but I cannot guarantee its accuracy; it seems to have trouble with rendering what I asked of it quite properly.

Some Fun Possibilities

So what I've been toying with for PSWordCloud has been the ability to construct complex structures in the abstract, and then my process has basically been to "store" them by adding them to a Region object, and on each iteration I check that any new additions don't interfere or overlap with pre-existing items.

This is sort of what it looks like:

using namespace System.Drawing

Add-Type -AssemblyName System.Drawing # Specify System.Drawing.Common on PS Core

# Init base image and graphics drawing objects
[Image] $Image = [Bitmap]::new(1024, 1024) # Width, Height
[Graphics] $DrawingSurface = [Graphics]::FromImage($Image)

# Initialize an empty region
$FilledSpace = [Region]::new()
$FilledSpace.MakeEmpty()

# Pens for outlines, brushes for fills
$Brush  [SolidBrush]::new([Color]::Black)
$PenWidth = 4.5
$Pen = [Pen]::new([Brush] $Brush, [double] $PenWidth)

foreach ($Word in $WordList) {
    [PointF]::new([double] $x, [double] $y)

    $WordPath = [Drawing2d.GraphicsPath]::new()
    $WordPath.AddString(
        [string] $Word, # The text to render as a path
        [FontFamily] $FontFamily, # [System.Drawing.FontFamily] object, mostly has the font name
        [int] $FontStyle, # Bold, italic, whatever, [System.Drawing.FontStyle] enum value as an int
        [double] $FontSize, # Em-size of the font
        [PointF] $DrawLocation, # Location of the path in the graphics space
        [StringFormat] $Format # [System.Drawing.StringFormat] object with format flags etc.
    )

    $DrawingSurface.DrawPath([Pen] $Pen, $WordPath) # Outline text
    $DrawingSurface.FillPath([Brush] $Brush, $WordPath) # Fill text

    # Add path to region
    $FilledSpace.Union($WordPath)
}

There's a lot going on that I'm going to sort of skip over here, because we can be here for hours before it's all fully explained. Thankfully, the Microsoft docs on System.Drawing are fairly complete, with some useful examples. Explore all you like, and experiment — that's the fun way to learn, in my opinion!

Checking Collisions and Overlaps

Now, you'll notice one interesting thing — a little bit missing in the above example. That's the sort of "collision checking" or intersection checking that I'm doing lots of in PSWordCloud. To do that, we have a fairly simple process we can follow, with one small caveat. When we're drawing our first item, we need to ignore the collision checking, because the libraries do some funny things when trying to check collisions against an empty Region.

With that mentioned, we can check for collisions based on the Region object using the $Region.IsVisible() method. This method has a bunch of overloads, for working with a variety of other graphical objects, both with and without reference to an existing graphics surface object.

Using this and just retrying a bunch of times as we scan across the image space, we can check any given area for empty space. You will, however, notice the somewhat tricky lack of an exact overload for checking the visibility of a GraphicsPath. This stymied me for a moment until I realised that the GraphicsPath object has a $Path.GetBounds() method, which returns a [RectangleF] object. Using this, we can check if a given path's bounds intersect with (are visible when drawn over) the existing Region object.

$WordBounds = $WordPath.GetBounds()

# Returns $true if the rectangle is within the region .IsVisible() is called on
$WordIsColliding = $FilledSpace.IsVisible($WordBounds)

With that, we have the framework for figuring out whether or not objects collide, and there are a whole lot of different objects you can work with in System.Drawing — lines, curves, ellipses, polygons, paths, etc. And with the GraphicsPath objects, you can create literally anything you want or need using its methods.

Some of this stuff is really handy for creating images, and it's also so completely outside the normal bent for PowerShell folks that working with it is utterly alien for myself, and I'm sure for many others out there.

Go wild, have fun, and see what crazy things you can come up with!

Thanks for reading!

PowerShell's Functional Language Features

2019-01-10T00:00:00-05:00

Functional languages differ from typical, more concretely object-oriented languages in a few important ways. One of the simplest to point out and demonstrate is that in most functional languages, every code path returns a value — there's effectively no such thing as a method that exists purely to create some side effect like there often is in object oriented languages.

I've been meaning to write this post for a while now, ever since my post on Anonymous Functions. I've been turning over in my head the similarities PowerShell has to some functional languages, particularly F#, so I thought it was about time I wrote about it!

I'll restrict my comments here to languages I know pretty well myself, so that really only leaves me with C#, F#, and PowerShell at the time of writing. I have some experience in a few others, but they're all pretty thoroughly object-oriented languages, so I'll leave them aside for the time being.

Functional Concepts

Before I even begin, yes, PowerShell breaks a lot of functional programming rules. I'm not here to pretend it doesn't, that would be silly. However, it's worth looking at PowerShell's features from a functional language perspective from time to time, because it can really help you grasp those unfamiliar concepts at times, and it's also quite fun to just use another lens once in a while. It can really change how you approach your code.

I'll be borrowing the core concepts list from this article in the interest of keeping things as straightforward as possible. Functional programming concepts can be a bit weird to wrap your head around for those of us really accustomed to the more object-oriented concepts.

Concept 1: Functions are Pure

A function called multiple times with the same arguments will always return the same value. Always.

No side effects occur throughout the function’s execution.

I'll be honest, I've never actually used a functional programming language where this is strictly true and properly enforceable. F# can certainly write very pure functions, but it's also tied to .NET, which means a whole heap of side effects available for use and abuse at a moment's notice.

Similarly, PowerShell can have pure functions, but mostly… doesn't. Being an automation tool, a great many parts of PowerShell exist precisely because of their side effects, and rightly so.

Concept 2: Functions are First-Class Citizens

However, in some ways its functions are first-class citizens. PowerShell has both script functions and cmdlets, which differ in quite a few ways, but in both cases they are themselves a type of object that can be manipulated and worked with. Cmdlets are their own class type itself, which inherit from the System.Management.Automation.Cmdlet base class.

Functions are themselves also a type of object — each function can be worked with as an instance of the System.Management.Automation.FunctionInfo class, and after storing a FunctionInfo object, you can later execute it. However, you can't simply store a function itself as an object directly, and you do need to use the Get-Item cmdlet to retrieve a function from the current session.

# Declare function
function Test-FunctionInfo { "Testing..." }

# Get FunctionInfo object
$Function = Get-Item -Path 'Function:\Test-FunctionInfo'

# Retrieve just the contents of the function (its script block)
$FunctionScript = ${Function:\Test-FunctionInfo}

# Invoke the function later
& $Function

However, although it's technically possible to create "higher-order" functions in PowerShell, which themselves return some other function, it's generally not particularly feasible or well-supported to program in such a fashion in PowerShell.

Concept 3: Variables are Immutable

For those of us who are not very familiar with functional languages (myself included), this concept seems very odd. Programming? With immutable variables? Say what? F# follows this to an extent — its variables are immutable by default, with an optional modifier that makes them mutable. It also requires that mutable variables be updated with a special operator for clarity in the code in terms of which variables are mutable or not.

// Immutable variables
let x = 1 // Attempting to re-assign a value later will be a compilation error

// Mutable variables
let mutable y = 4 // Initial assignment
y <- 5 // Assigning a new value

PowerShell, however, has very mutable variables by default. However, if you wanted to program in a more functional manner, you could conceivably do so using Set-Variable or New-Variable to create read-only variables or constants.

# Readonly variables can be removed later with Remove-Variable -Force
Set-Variable -Name readonly -Value 5 -Option ReadOnly
$Readonly # Output: 5

# Constant variables can never be removed from the scope they're created in
Set-Variable -Name const -Value 12 -Option Constant
$Const # Output: 12

If you wanted to program with this ideal in mind, I would generally recommend using read-only variables instead of constants. Constants in PowerShell are really hard to work with at times, but I suppose if you really wanted to enforce the functional programming concept of immutable variables you could do so with it. However, I'd still strongly recommend never setting constant variables in the global scope — the only way to wipe the slate clean there is to literally restart the whole PowerShell session.

Because PowerShell is a console and an automation framework, it often won't make that much sense to use too many immutable variables in general use, but I could definitely see that perhaps in a given module you may want to attempt to code in this matter for greater reliability of results. However, it's also important to remember that use of some objects may make it rather difficult to enforce completely — for example, a constant variable may store a hashtable reference, but as long as you're using an actual hashtable there isn't really a way to make it read only.

You could, conceivably, instead opt to create a [System.Collections.ObjectModel.ReadOnlyDictionary[T1,T2]] rather than a hashtable, and use [System.Collections.ObjectModel.ReadOnlyCollection[T]] instead of a standard array, however — so if you would like to attempt to go that route, it may be possible to do so. Personally, I think having a constant or read-only variable that points to a hashtable that remains mutable may still be an interesting concept to play with at times.

Concept 4: Functions Have Referential Transparency

As the linked article details, referential transparency is a bit of a tricky thing to define properly. But given the definition in that article, I think it's pretty safe to say that the vast majority of PowerShell functions and cmdlets are not referentially transparent by nature. Certainly you can opt to make something referentially transparent in PowerShell, but in the majority of places you want to use PowerShell it's probably not a particularly valuable option.

Concept 5: Basis in Lambda Calculus

This really comes down to two things. First, that all functions can be written without a name, and second, that functions are curried.

Nameless Functions

In PowerShell, separating a function from its name or creating a nameless function is quite simple and straightforward. You can simply extract the script block from the function to create what is effectively a nameless function, which can be executed independently as its own object. Script blocks can also be created independently of functions and passed as parameters or executed on their own using the & operator or either the $sb.Invoke() or $sb.InvokeReturnAsIs() methods.

$Scriptblock = Get-Item -Path Function:mkdir | Select-Object -ExpandProperty ScriptBlock
& $Scriptblock 'C:\Test'

$NewScriptblock = {
    param([string] $Path)
    Get-ChildItem -Path $Path
}

$NewScriptBlock.Invoke('C:\Program Files')

Currying Functions

Currying is probably something most people working with PowerShell either haven't heard of or don't really use very often. PowerShell isn't really designed for currying functions, which I would say is mostly because it's designed to function as a shell for administrative and automation purposes. Currying is what a language like F# does naturally, and it's a bit of an unusual concept to those of us accustomed to working with less functionally-minded languages.

In F#, all functions have a single set of parameters that are all mandatory, unlike PowerShell. If you call an F# function with less parameters than it needs to function, it doesn't return any errors or warnings. Instead, it's designed such that doing so actually creates a new version of that function which effectively has some preset parameters compared to the original. It's a bit easier to show than to tell with this sort of thing, so let's see how this tends to work.

// Define the "base" function
let sumNumbers x y z =
    x + y + z

// This function can be called directly, with 3 parameters
sumNumbers 1 2 3 // returns: 6

// Define a curried function
let addToThree x y =
    sumNumbers 3

// Call curried version, with two parameters
addToThree 2 2 // returns: 7

PowerShell typically isn't designed to work like this. However, given that functions can be pretty easily built on the fly with script blocks, you could potentially design a framework to curry functions in PowerShell. In general, though, this would almost certainly become rather complicated with PowerShell's ability to define complex parameter sets and optional parameters. But if you were really determined to make it work, you definitely could do so — and with PowerShell's splatting ability combined with $PSBoundParameters, it is actually relatively easy to do so.

function Add-Numbers {
    param([int] $x, [int] $y)

    $x + $y
}

# "curried" function
function Add-ToSquare {
    param([int] $x)

    $Square = $x * $x
    Add-Numbers @PSBoundParameters $Square
}

As a result, in general usage function currying is, to an extent, almost "baked in" in some ways to the function design itself, depending on how you choose to construct your parameters, set default parameters, and permit optional parameters to be omitted. Working directly with script blocks and the & operator also brings it a bit closer to the functional programming language methodology, as well. However, unlike true functional languages, it doesn't do currying by default with all functions.

Implicit Return

F# has something that I would usually call an "implicit return" — that is, there isn't really a return statement, and anything called on a line on its own will return the value to whatever assignment is "upstream", if there is any. If there is no upstream assignment going on, it simply outputs the variable to the console. I'm not sure if this is something that's super common in functional languages, but it is something that PowerShell also mimics by design.

$Values = foreach ($Number in 1..10) {
    # Note there is no assignent happening here, so it is passed upstream to the $Values variable
    # and stored as an array.
    $Number * 2
}

function Get-Numbers {
    param([int] $Start, [int] $End)

    # Simply outputs the values when the function is called
    $Start..$End
}

Get-Numbers 1 10 # Prints numbers 1 through 10 to the console

PowerShell does still have a return statement, but its primary operation here is purely for flow control. You can pass it a value if you like, but it doesn't require a value to operate. When called, return in PowerShell simply passes control back to the caller of the function or script. This can be a bit confusing for some, who are more accustomed to a language like C#, which doesn't allow for this kind of implicit return, and assume that unless they specify return no values will be output. Because of this, the usual recommendation is to restrict use of return only to places where it is really needed for flow control (to exit a function early) or in classes, where the stricter parsing and JIT compilation modes make it a requirement.

PowerShell is Weird

PowerShell has a lot of object-oriented tendencies, and that's really the core principle of its overall design. But because it's also a functional administrative shell, it also takes a few pages from the functional programming book as it needs them. It's a very interesting combination, and I am rather fond of it myself, though I understand all too well that this unusual combination of concepts can tend to make it a rather confusing language to work with.

I've found that at least for myself, I understood PowerShell pretty readily only because I had prior experience with both F# and C#, and as a result I was able to pretty easily categorise and note the different interplay of concepts which often don't really work together in other languages.

For anyone who wants to get a little more familiar with a more functionally-minded language, I highly recommend a bit of dabbling in F#. When I was first looking for learning materials on it, I discovered Chris Marinos' F# Koans, which I highly recommend for anyone wanting to take a look at F# from a beginner's perspective — they were also the inspiration for PSKoans, after all!

Thanks for reading!

Contributing to Open Source Projects - PowerShell Core

2019-01-03T00:00:00-05:00

It's been a little while now since I started contributing to PowerShell Core, so I thought it's probably an apt time to talk about it a little bit. I've come to the conclusion that sooner or later, we should probably all look to see if we can contribute anything to whichever tools we happen to be using. After all, there really is no one better suited to guide the development of the tools you're using on a daily basis than you. I'll try to break down why I think we should all look to contribute a little something back to the tools we use in our day-to-day lives wherever we can.

Developers often have their own ideas for where the project is headed, and that's fantastic — having a concrete vision for where they want to take the project is essential, if they ever want to get it off the ground and in common use. However, it is also arguably just as important for those of us who actually use the end product to be able to offer our own perspective, experience, and ideas. A project maintained alone can only survive so long, after all.

Without a new perspective every once in a while, many things get ignored or overlooked; it's essential to be able to hear about the areas where the product falls short as well as the areas it does well. Without that, it becomes a never-ending minefield of guessing what will be valuable in the short term as well as what is worthwhile pursuing in the long-term.

Selfish Selflessness: Perspective is Important

While it may not always be voiced, I can't help but feel that all too often I see people take the attitude of "well, it doesn't necessarily help me or what I'm doing; why should I bother?" Certainly, there may not always be a clear reason to contribute to an open source project. It mightn't help you in your day to day immediately with that specific tool, or you mightn't have the necessary knowledge right now to make something happen that you'd like to see.

Even so, I think it's well worth your while to contribute as much as you're able to spare the time for as a hobby. I have at least a few reasons for believing this.

Reason 1: It helps the overall development of the project

You mightn't be able to contribute immensely to the core functionality of a tool, you mightn't even be all that familiar with the language used to create the tool. Even so, you can chip in and help out with a small amount of the "simpler" things, like:

Low-priority issues
Minor bugs
Small quality-of-life improvements
Documentation

I emphasize documentation because it's the easiest to neglect, but also one of the most important things that affects the adoption of products as well as new features.

Helping out with these "less-important" but frequently overlooked "minor" things can really free up other contributors and the project maintainers to be able to dedicate their time to the more flashy and impressive new features or overhauls that always take more time. If they're always finding their time chipped away by having to constantly handle the small stuff, they'll be caught in the uncomfortable place of either having to ignore these issues entirely to get that new feature completed (often to the detriment of documentation), or they'll have to forego the necessary time investment to properly complete or even get any work done at all on the new feature they've been trying to work on.

With even a handful of people helping chip away at the smaller issues, the ones that seem on the surface to be "less important" no longer need take up time for everyone else who is contributing to the project. As a result, the project's development can be accelerated quite a bit just by a few people taking a little time to jot down some documentation or tackle the relatively simpler fixes and minor issues.

Reason 2: It helps you develop your own skills, or learn new ones

My first contribution to PowerShell Core was very much a trial by fire, as Patrick put it. I was grumbling in the PowerShell Discord server about how it was very annoying that you can't specify named parameters for .NET methods in PowerShell, I think.

Enter Patrick, stage left (known widely as @seeminglyscienc on Twitter, and similarly SeeminglyScience on Github) who simply asked me "Why don't you make it happen?"

Frankly, I was surprised. I hadn't even considered doing it myself — and I suspect I am not the only one out there who simply hasn't even considered contributing. That's part of the reason I'm writing this at all, really.

I knew I was out of my depth, but Patrick was keen to help me figure it out. He gave me initial pointers, some places to start, and told me to send him a message if I got stuck. In the interests of transparency, that idea still hasn't really gotten anywhere. It's still out of my depth a bit, but I'm confident now that given enough time to work on it and sufficient guidance, I could absolutely make it happen. It's a pet project that I've not yet found the time to complete just yet, but it's very much doable.

Before I got too far with that, however, someone brought up PowerShell's rather sparse support for different numeric types. Sure, you can apply a type cast, but unlike many other languages — including its close cousin C# — PowerShell really only supported int, long, decimal, and double at the time. (This has since changed.)

I started to wonder about this as well, and before long I asked Patrick if he knew where I should start looking — and of course he did. I've since found that quite a lot of the folks you'll run across in the PowerShell repository (and indeed many of those who maintain or contribute to projects written in PowerShell) are extremely helpful and willing to point you where you need to go.

After several iterations and a couple of pull requests I ended up closing because, frankly, my commit history was an absolute mess, I had a three part project on my hands. The last pull request in this particular saga is still pending review, but if you're interested in checking out the improvements I coded into the tokenizer (including the native ability for PS to parse binary numbers, with a very gracious nod to HumanEquivalentUnit for his impressive work helping me refine and optimize the binary parse method) you can find that ongoing here at the time of writing.

And beyond that, I've since branched out into some extra smaller projects, like improving the type inference for $_ and $PSItem in a catch {} block, once again with some interesting ideas from Patrick's inventive mind.

Now, I started out knowing some C#, but not an awful lot. I would say I knew just enough to be able to comprehend most code bases I would be reading, but in terms of actually being able to write it well, I was pretty rusty at best. I took one year of a C# course back in college, and then got too bored to continue; I wasn't exactly the model student for that sort of thing. However, having a project to puzzle over for a time really helped me get a great many more tools under my belt, and I'm relatively confident that at the moment I could probably work in any codebase I want to, provided I have a little bit of extra time to familiarize myself with the code and general structure.

Reason 3: It prepares you to take on the tougher stuff

As I mentioned above, I made it rather hard for myself in the beginning when I started contributing to PS Core; I can't say I would recommend starting that way, but it certainly has its merits. Even so, it's probably not for everyone. We all learn in different ways, after all.

But no matter where you start, you'll be gaining some amount of familiarity with the project's code, their general code of conduct, their methodology when it comes to the product's lifecycle and the rhythm they like to work with.

Hang around the repository and look at other people's pull requests, talk about issues that are raised, and keep learning. Sooner or later, you'll be ready enough to tackle that issue you've been eyeing for the past couple weeks. And if you're not quite sure where to start, I can think of no better way than to simply submit an issue on the problem or enhancement you're looking to create, lay out what you do know, what you don't, and more often than not someone else will happily help you figure out where to start.

If you hit a major roadblock, don't be afraid to submit a WIP pull request and ping a few of the active maintainers or contributors asking for a bit of assistance. Most of them are only too happy to help, overjoyed that someone else is also willing to help out, even if they're having difficulties.

Contribute

With that all said, I really only can urge everyone to contribute wherever they can. It's all too easy to say "Well, someone else will make it happen eventually." And, sure, maybe they will. But maybe they won't, because they think just like you.

Change happens when a community makes it happen, not when one lone person decides it's time. Time and time again we see in this world that some corporate "leader" or CEO tries to make something change, but doesn't really tell anyone else how or why it should change. Inevitably, it backfires — the community is a rubber band that pulls things back to how they were in some form or another.

If you want your favourite tool, framework, or language to get better, it's very much up to you. Many will grow without you, if they must, but in just about every case, another pair of helping hands and a fresh set of eyes with just a slightly different perspective can really amplify the growth and accelerate progress immensely.

What deserves your time?

What deserves your perspective?

How can you make an impact?

All About PSKoans

2018-12-27T00:00:00-05:00

It's been a handful of months since I started working on the PSKoans module, so I thought I'd take the time to set down the story behind the PSKoans module. It borrows from some koans modules from other languages, and the concept of koans as a programming tool seems to originate from a Ruby project — although at least a handful of other languages now have koan libraries available for learners.

The concept of kōans was first introduced to me when I first decided to learn how to code in F#. Initially, I struggled quite a bit. For those of you who mightn't be familiar with it, it's a bit of an unusual language that aims to bring functional language features into a .NET family language. This places it in a bit of a strange in-between, where many of the libraries are heavily object-oriented, and yet the language itself is quite clearly built for more functional constructs.

Whilst searching for learning resources, I stumbled across Chris Marinos' F# Koans, which (like other koans projects) aim to teach you F# by presenting you with a series of failing assertions, and guiding you towards fixing them, teaching you the logic behind how it works. These in turn are inspired by EdgeCase's Ruby Koans, and the readme there has further links and information as the origins of the concept itself.

First, Some Definitions

You could say the simplest definition of a kōan is "a logical contradiction." The term is old, dating back to Zen Buddhist monks, who would be instructed to meditate upon logical contradictions until they could arrive at a purely intuitive understanding of a concept being studied, rather than simply reasoning logically. It is often written simply as koan, mainly because the accented O character can be a bit tricky to type on English keyboards.

For programming purposes, a koan usually is framed as an assertion which is initially written in a 'failing' state. A contradiction. Something simple like 1 + 1 | Should -Be 3 (at least, if you write it in Pester syntax) should give you the general idea. In actuality, many koans projects opt to define a custom blank function to represent a space to be filled in; in PSKoans I have a function named Get-Blank which uses __ as its alias, and a similar idea is used in Chris Marinos' F# Koans.

This is used to create an assertion that will always fail the first time around, and often some interaction framework is used as an in-between to help guide the pupil to where to look in the error message and how to find the issue and resolve it For example:

Describe 'Addition' {
    It 'is basic arithmetic' {
        __ | Should -Be ( 5 + 1 )
    }
}

Executing this in PowerShell (with PSKoans installed, or a simple return $null function aliasing the __ command call) will result in the following output:

+ Describing Addition
-   [-] is basic arithmetic 3.9s
-     Expected 6, but got $null.
-     3:         __ | Should -Be ( 5 + 1 )

Pretty straightforward; tells you what's wrong, and where it needs to be fixed.

Closing the Gap

Up until recently, a majority of PowerShell learning materials fit into the following categories:

Books
Talks
Blog Posts
Documentation

These are all wonderful ways to learn and expand on what you know. However, they lack a certain something that I would have loved to have when I was learning the ropes myself — that extra bit of interactivity. For me, learning to code from a book has always been a doomed prospect. There's something incredibly disconnecting about the code being on a printed (or even digital) book page; if you ever want to try something out, you have to write everything out all at once, and if you're not familiar with the language, chances are you'll mess it up and not even begin to know where you went wrong. Books are great for teaching concepts, but for me they miss by a large margin when trying to convey or really impress the practicality of the language, or even the depth of its usefulness.

Talks have a similar issue, I fear; they're fantastic for opening up your breadth of knowledge and showing you some really cool stuff you mightn't see or hear about otherwise, but they fall quite flat when you don't have the basics down and can't really follow what's going on quite yet.

And blog posts and documentation can often be far too dry to really spark those learning moments. Not that they're bad, of course. They have their place, as everything else I listed does — and I'm sure I missed a few methods, as well — but there's a gap where it really truly pays to have some form of interactive experience, and I hope that that's where PSKoans can come in and really shine. My wife likens it to a game — we set the challenges, and you need to solve them one by one or in bulk if you prefer, and we can give you feedback of how you're doing, where you're going wrong, and (down the development road, anyway) potentially even eventually some suggestions of how to proceed in areas you might get stuck.

Taking the Plunge

I'm sure many of my readers are probably tending towards the more advanced, but it's been my own experience that we tend to have little holes in our knowledge even of the more basic things. In fact, as I create the koans for a particular topic I often learn new things about that very topic while I'm writing them! So, if you've not yet found the time to knock out a few koans and see why I keep suggesting new folks try it, I can only encourage you to give it a shot — you may well learn something from the experience.

If nothing else, perhaps you'll gain enough familiarity to simply be able to provide a gentle recommendation for those learning PowerShell. I've been chatting to a few people about their possible application in a more controlled group classroom or training-type setting as well, and Thomas Rayner has already begun teaching some of his coworkers using PSKoans as one of the primary tools. His initial results are encouraging, but as with any stress test I tend to miss the occasional oddity until someone stumbles across it. It's very hard to properly review your own writings, after all, even when they are in PowerShell — but I try my best!

Installing the Module

PSKoans is available from the PowerShell gallery, or the Github page, so you can install it anywhere you have a PowerShell 5.1 or PowerShell Core engine available. It's designed to be fairly easy to work with, but it does require a recent version of Pester to be installed, so make sure you have that down and then it's simply an Install-Module PSKoans away! If you have any issues or concerns, feel free to leave an issue on the Github and I'll get back to you shortly — or if it's urgent, you can reach me on the PowerShell Discord server.

Clear-Script

Verify Your Module's Help with Pester v5

What Worked in Pester v4

Testing Phases in Pester v5

Limitations of -TestCases

Prepare for Pester v5

ScriptBlocks and GetNewClosure()

Capturing State

GetNewClosure()

Handling PowerShell's Versatile Subexpressions

Subexpressions and Anonymous Functions

Types of Subexpressions

Regular Subexpressions

Array Subexpressions

Inline Scriptblock Invocations

{}.GetNewClosure()

Putting it All Together

Examples

Generating a List of Arguments for a Native Command

Creating Match Patterns

Collating Output from Multiple Sources

Familiarity

Building Arrays and Collections in PowerShell

Background Information

Background - Collections in .NET

ICollection

Collections in PowerShell

Common Patterns

Use of += to Build Arrays

What's Wrong with this Picture

Using an Expandable List

Why Use a List

Using the Pipeline

Working With Collections

New-Object v.s. ::new() v.s. Casting

Adding Many Items to an Existing Collection

General Recommendations

Working with PowerShell's -replace Operator

Working with -replace

Using Match Groups

Named Match Groups

Summary

Error Handling in PowerShell - Best Practices

Different Types of Errors

Terminating Errors

Throw

ThrowTerminatingError()

Non-Terminating Errors

Write-Error

WriteError()

Emit Errors Responsibly

Playing Around with System.Drawing in PowerShell

Disclaimers

Some Fun Possibilities

Checking Collisions and Overlaps

PowerShell's Functional Language Features

Functional Concepts

Concept 1: Functions are Pure

Concept 2: Functions are First-Class Citizens

Concept 3: Variables are Immutable

Concept 4: Functions Have Referential Transparency

Concept 5: Basis in Lambda Calculus

Nameless Functions

Currying Functions

Implicit Return

PowerShell is Weird

Contributing to Open Source Projects - PowerShell Core

Selfish Selflessness: Perspective is Important

Reason 1: It helps the overall development of the project

Reason 2: It helps you develop your own skills, or learn new ones

Reason 3: It prepares you to take on the tougher stuff

Contribute

All About PSKoans

First, Some Definitions

Closing the Gap

Taking the Plunge

Installing the Module

Limitations of `-TestCases`

`GetNewClosure()`

`{}.GetNewClosure()`

Use of `+=` to Build Arrays

`New-Object` v.s. `::new()` v.s. Casting

Working with `-replace`

`Throw`

`ThrowTerminatingError()`

`Write-Error`