By the end, you’ll be familiar with several .NET classes and understand how to discover and use others that might better meet your specific needs.
Listing Available Methods and Properties of .NET Classes
If you’re wondering what other static methods LINQ has, you may have tried to pipe the class to Get-Member
. You may have also tried to do the same for a List<T>
object and found unexpected results.
What you’ll find with collections like List<T>
is Get-Member
returns data of the type of the first item inside the collection.
C:\> = ::new
C:\>
: You must specify an object the cmdlet
C:\> Add1
C:\>
TypeName: SystemInt32
Name MemberType Definition
CompareTo Method int CompareToSystemObject value int CompareToint value int IComparableCompareToSystemObject obj int IComparableCompareToint other
Equals Method bool EqualsSystemObject obj bool Equalsint obj bool IEquatableEqualsint other
GetByteCount Method int IBinaryIntegerGetByteCount
GetHashCode Method int GetHashCode
GetShortestBitLength Method int IBinaryIntegerGetShortestBitLength
We can use GetMembers()
to get the list of methods and properties. When calling it on an already instantiated object we first need to bubble up the class using GetType()
, and then call GetMembers()
.
C:\> C:\> GetTypeGetMembers ? MemberType ExpandProperty Name Unique
get_Capacity
set_Capacity
get_Count
get_Item
set_Item
Add
AddRange
AsReadOnly
BinarySearch
Clear
Contains
ConvertAll
CopyTo
EnsureCapacity
Exists
Find
If calling it on a type, we can omit the GetType()
call. Here are some of the LINQ methods available:
C:\> GetMembers ? MemberType ? IsStatic ExpandProperty Name Unique
Empty
Aggregate
Any
All
Append
Prepend
Average
OfType
Cast
Chunk
Concat
Contains
Count
TryGetNonEnumeratedCount
LongCount
DefaultIfEmpty
Distinct
DistinctBy
ElementAt
ElementAtOrDefault
AsEnumerable
Except
ExceptBy
First
$cyberarkAccounts = Get-ADUser -identity users01 -Properties samaccountname,enabled,memberof
$cyberarkAccountTable = @{}
foreach ($account in $cyberarkAccounts) {
$cyberarkAccountTable[$account.samaccountname] = $account.MemberOf | %{(Get-ADGroup $_).sAMAccountName} | Where-Object { $_ -like 'apps Users*' -or $_ -like 'apps test Users*' }
}
Name Value
--- ----
Users01
Users02 apps users
Users03 apps users,apps test Users
My desired output:
Name Value
--- ----
Users01 NULL
Users02 apps users
Users03 apps users,apps test Users
$cyberarkAccounts = Get-ADUser -Identity users01 -Properties samaccountname, enabled, memberOf
$cyberarkAccountTable = @{}
foreach ($account in $cyberarkAccounts) {
$result = $account.MemberOf |
ForEach-Object { (Get-ADGroup $_).sAMAccountName } |
Where-Object { $_ -like 'apps Users*' -or $_ -like 'apps test Users*' }
$cyberarkAccountTable[$account.samaccountname] = if ($result) { $result -join ',' } else { 'NULL' }
}
$targetGroups = 'apps Users', 'apps test Users'
$groupMap = @{} # maps key: each item in `$targetGroups` value: hashset with each group member samAccountName
$targetGroups | ForEach-Object {
$groupDn = (Get-ADGroup $_).DistinguishedName
$members = (Get-ADUser -LDAPFilter "(memberOf=$groupDn)").SamAccountName
$groupMap[$_] = [System.Collections.Generic.HashSet[string]]::new(
[string[]] $members)
}
$cyberarkAccounts = 'user1', 'user2', 'useretc' # these should be samAccountNames!
foreach ($account in $cyberarkAccounts) {
$membership = foreach ($group in $targetGroups) {
if ($groupMap[$group].Contains($account)) {
$group
}
}
[pscustomobject]@{
User = $account
Membership = if ($membership) { $membership -join ',' } else { 'NULL' }
}
}
PowerShell is the best option for this kind of file operation. Let’s check each method with a complete example.
Method 1: Using Get-Unique Cmdlet
The PowerShell Get-Unique cmdlet is designed to compare each item in a sorted list to the next item, eliminate duplicates, and return only one instance of each item. This method requires the input to be sorted first.
Here is how it will work:
- Read the file and sort the content: $lines = Get-Content “C:\MyFolder\MyExample.txt” | Sort-Object
- Get unique lines: $uniqueLines = $lines | Get-Unique
- Output the unique lines to a new file: $uniqueLines | Set-Content “C:\MyFolder\MyExample_unique.txt”
This method ensures you get unique lines, below is the complete script.
$lines = Get-Content "C:\MyFolder\MyExample.txt" | Sort-Object
$uniqueLines = $lines | Get-Unique
$uniqueLines | Set-Content "C:\MyFolder\MyExample_unique.txt"
$uniqueLines
I executed the above script, and it showed me the output in the console and written in the text file. Check out the screenshot below.

Method 2: Using Select-Object with -Unique Parameter
Let me show you another relatively simple method. In PowerShell, you can use the Select-Object with the -Unique parameter. When we use the -Unique parameter, we do not need to sort the input.
This is how it will work:
- Read the file and get unique lines: $uniqueLines = Get-Content “C:\MyFolder\MyExample.txt” | Select-Object -Unique
- Output the unique lines to a new file: $uniqueLines | Set-Content “C:\MyFolder\MyExample_unique.txt”
Here is the complete PowerShell script:
$uniqueLines = Get-Content "C:\MyFolder\MyExample.txt" | Select-Object -Unique
$uniqueLines | Set-Content "C:\MyFolder\MyExample_unique.txt"
$uniqueLines
Check out the screenshot below, it is giving the output like the above after I executed the PowerShell script using VS code.

Method 3: Using HashSet
If you are working with a large text file, you can use the HashSet to get unique lines in PowerShell.
Here is how it works.
- Initialize the HashSet and read the file:
$hashSet = [System.Collections.Generic.HashSet[string]]::new() $lines = Get-Content "C:\MyFolder\MyExample.txt"
- Add each line to the HashSet:
foreach ($line in $lines) { $hashSet.Add($line) | Out-Null }
- Output the unique lines to a new file:
$hashSet | Set-Content "C:\MyFolder\MyExample_unique.txt"
Here is the complete script.
# Step 1: Initialize the HashSet and read the file
$hashSet = [System.Collections.Generic.HashSet[string]]::new()
$lines = Get-Content "C:\MyFolder\MyExample.txt"
# Step 2: Add each line to the HashSet
foreach ($line in $lines) {
$hashSet.Add($line) | Out-Null
}
# Step 3: Output the unique lines to a new file
$hashSet | Set-Content "C:\MyFolder\MyExample_unique.txt"
Method 4: Using Group-Object Cmdlet
Let me show you the final method.
You can also use the Group-Object cmdlet in PowerShell to group and then select unique lines.
Here is how it works:
- Read the file and group by line content:
$groupedLines = Get-Content "C:\MyFolder\MyExample.txt" | Group-Object
- Select unique lines:
$uniqueLines = $groupedLines | ForEach-Object { $_.Name }
- Output the unique lines to a new file:
$uniqueLines | Set-Content "C:\MyFolder\MyExample_unique.txt"
Below is the complete script.
# Step 1: Read the file and group by line content
$groupedLines = Get-Content "C:\MyFolder\MyExample.txt" | Group-Object
# Step 2: Select unique lines
$uniqueLines = $groupedLines | ForEach-Object { $_.Name }
# Step 3: Output the unique lines to a new file
$uniqueLines | Set-Content "C:\MyFolder\MyExample_unique.txt"
$uniqueLines
After I executed the above script, you can see the output in the screenshot below.

Conclusion
I hope now you have a complete idea of how to get unique lines from a file in PowerShell using various methods, such as the Get-Unique Cmdlet, the Select-Object with -Unique Parameter, and the Group-Object Cmdlet.
PowerShell is an interactive shell and scripting language from Microsoft. It’s object-oriented — and that’s not just a buzzword, that’s a big difference to how the standard Unix shells work. And it is actually usable as an interactive shell.
Getting Started
PowerShell is so nice, Microsoft made it twice.
Specifically, there concurrently exist two products named PowerShell:
- Windows PowerShell (5.1) is a built-in component of Windows. It is proprietary, Windows-only, and is based on the equally proprietary and equally Windows-only .NET Framework 4.x. It has a blue icon.
- PowerShell (7.x), formerly known as PowerShell Core, is a stand-alone application. It is MIT-licensed (developed on GitHub), available for Windows, Linux, and macOS, and is based on the equally MIT-licensed and equally multi-platform .NET (formerly .NET Core). It has a black icon.
Windows PowerShell development stopped when PowerShell (Core) came out. There are some niceties and commands missing in it, but it is still a fine option for trying it out or for when one can’t install PowerShell on a Windows system but need to solve something with code.
All examples in this post should work in either version of PowerShell on any OS (unless explicitly noted otherwise).
Install the modern PowerShell: Windows, Linux, macOS.
Objects? In my shell?
Let’s try getting a directory listing. This is Microsoft land, so let’s try the DOS command for a directory listing — that would be dir
:
This looks like a typical (if slightly verbose) file listing.
Now, let’s try to do something useful with this. Let’s get the total size of all .txt
files.
In a Unix shell, one option is du -bc *.txt
. The arguments: -b
(--bytes
) gives the real byte size, and -c
(--summarize
) produces a total. The result is this:
Let’s try something in PowerShell. If we do $x = dir
, we’ll have the output of the dir
command in $x
. Let’s try to analyse it further, is the first character a newline?
What if we try getting the Length
property out of that?
It turns out that dir
returns an array of objects, and PowerShell knows how to format this array (and a single item from the array) into a nice table. What can we do with it? This:
We can iterate over all file objects, get their length (using ForEach-Object
and a lambda), and then use Measure-Object
to compute the sum (Measure-Object
returns an object, we need to get its Sum
property). We can replace the ForEach-Object
call with the -Property
argument in Measure-Object
. And if we want to look into subdirectories, we can easily add -Recurse
to Get-ChildItem
. We get actual integers we can do math on.
You might have noticed I used Get-ChildItem
instead of dir
in the previous example. Get-ChildItem
is the full name of the command (cmdlet). dir
is one of its aliases, alongside gci
and ls
(Windows-only to avoid shadowing /bin/ls
). Many common commands have aliases defined for easier typing and ease of use — Copy-Item
can be written as cp
(for compatibility with Unix), copy
(for compatibility with MS-DOS), and ci
. In our examples, we could also use measure
for Measure-Object
and foreach
or %
for ForEach-Object
. Those aliases are a nice thing to have for interactive use, but for scripts, it’s best to use the full names for readability, and to avoid depending on the environment for those aliases.
More filesystem operations
Files per folder
There’s a photo collection in a Photos
folder, grouped into folders. The objective is to see how many .jpg
files are in each folder. Here’s the PowerShell solution:
- Group by
$_.Directory.Name
(same as before) to gettwo
- Group by
Split-Path -Parent ([System.IO.Path]::GetRelativePath("$PWD/Photos", $_.FullName))
to getone/two
- Group by
([System.IO.Path]::GetRelativePath("$PWD/Photos", $_.FullName)).Split([System.IO.Path]::DirectorySeparatorChar)[0]
to getone
(All of the above examples work for a single folder as well. The latter two examples don’t work on Windows PowerShell.)
Duplicate finder
Let’s build a simple tool to detect byte-for-byte duplicated files. Get-FileHash
is a shell built-in. We can use Group-Object
again, and Where-Object
to filter only matching objects. Computing the hash of every file is quite inefficient, so we’ll group by the file length first, and then ensure the hashes match. This gives us a nice pipeline of 6 commands:
# Fully spelled out # Using aliases # Using less readable aliases
Serious Scripting: Software Bill of Materials
Software Bills of Materials (SBOMs) and supply chain security are all the rage these days. The boss wants to have something like that, i.e. a CSV file with a list of packages and versions, and only the direct production dependencies. Sure, there exist standards like SPDX, but the boss does not like those pesky “standards”. The backend is written in C#, and the frontend is written in Node.js. Since we care only about the production dependencies, we can look at the .csproj
and package.json
files. For Node packages, we’ll also try to fetch the license name from the npm API (the API is a bit more complicated for NuGet, so we’ll keep it as a TODO
in this example).
# stop execution on any error
Just like every well-written shell script starts with set -euo pipefail
, every PowerShell script should start with $ErrorActionPreference = "Stop"
so that execution is stopped as soon as something goes wrong. Note that this does not affect native commands, you still need to check $LASTEXITCODE
. Another useful early command is Set-StrictMode -Version 3.0
to catch undefined variables.
For .csproj
files, which are XML, we look for PackageReference
elements using XPath, and then build a PSCustomObject out of a hashmap — extracting the appropriate attributes from the PackageReference
nodes.
For package.json
, we read the file, parse the JSON, and extract the properties of the dependencies
object (it’s a map of package names to versions). To get the license, we use Invoke-RestMethod
, which takes care of parsing JSON for us.
In the main body of the script, we look for the appropriate files (skipping things under node_modules
) and call our parser functions. After retrieving all data, we concatenate the two arrays, convert to CSV, and use Tee-Object
to output to a file and to standard output. We get this:
Could it be done in a different language? Certainly, but PowerShell is really easy to integrate with CI, e.g. GitHub Actions or Azure Pipelines. On Linux, you might be tempted to use Python — and you could get something done equally simply, as long as you don’t mind using the ugly urllib.request
library, or alternatively ensuring requests
is installed (and then you get into the hell that is Python package management).
Using .NET classes
PowerShell is built on top of .NET. This isn’t just the implementation technology — PowerShell gives access to everything the .NET standard library offers. For example, the alternate ways to group photos in multiple subdirectories we’ve explored above involve a call to a static method of the .NET System.IO.Path
class.
Other .NET types are also available. Need a HashSet? Here goes:
It is also possible to load any .NET DLL into PowerShell (as long as it’s compatible with the .NET version PowerShell is built against) and use it as usual from C# (although possibly with slightly ugly syntax).
Sick Windows Tricks
Microsoft supposedly killed off Internet Explorer last year. Attempting to launch iexplore.exe
will bring up Microsoft Edge. But you see, Internet Explorer is a crucial part of Windows, and has been so for over two decades. Software vendors have built software that depends on IE being there and being able to show web content. Some of them are using web views, but some of them prefer something else: COM.
COM, or Component Object Model, is Microsoft’s thing for interoperability between different applications and/or components. COM is basically a way for classes offered by different vendors and potentially written in different languages to talk to one another. Under the hood, COM is C++ vtable
s plus standard reference counting and class loading/discovery mechanisms. The .NET Framework, and its successor .NET, have always included COM interoperability. The modern WinRT platform is COM on steroids.
Coming back to Internet Explorer, it exposes some COM classes. They were not removed with iexplore.exe
. This means you can bring up a regular Internet Explorer window in just two lines of PowerShell:
We have already explored the possibility of using classes from .NET. .NET comes with a GUI framework named Windows Forms, which can be loaded from PowerShell and used to build a GUI. There is no form designer, so it requires manually defining and positioning controls, but it actually works.
Getting out of PowerShell land
As a shell, PowerShell can obviously launch subprocesses. Unlike something like Python, running a subprocess is as simple as running anything else. If you need to git pull
, you just type that. Or you can make PowerShell interact with non-PowerShell commands, reading output and passing arguments:
"Not a git repository" "Getting changes from git failed" "No changes found" # Alternate spelling for regex fans: untracked files in VS Code" "No untracked files"
I chose to compute untracked files with the help of standard .NET string manipulation methods, but there’s also a regex option. On a related note, there are three content check operators: -match
uses regex, -like
uses wildcards, and -contains
checks collection membership.
Profile script
I use a fairly small profile script that adds some behaviours I’m used to from Unix, and to make Tab completion show a menu. Here are the most basic bits:
# Commands starting with space are not remembered.
Apart from that, I use a few aliases and a pretty prompt with the help of oh-my-posh.
The unusual and sometimes confusing parts
PowerShell can be verbose. Some of its syntax is a little quirky, compared to other languages, e.g. the equality and logic operators (for example, -eq
, -le
, -and
). The aliases usually help with remembering commands, but they can’t always be depended on — ls
is defined as an alias only on Windows, and Windows PowerShell aliases wget
and curl
to Invoke-WebRequest
, even though all three have completely different command line arguments and outputs (this was removed in PowerShell).
Moreover, the Unix/DOS aliases do not change the argument handling. rm -rf foo
is invalid. rm -r foo
is, since argument names can be abbreviated as long as the abbreviation is unambiguous. rm -r -f foo
is not valid, because -f
can be an abbreviation of -Filter
or -Force
(so rm -r -fo foo
) will do. rm foo bar
does not work, an array is needed: rm foo,bar
.
C:\Windows\regedit.exe
launches the Registry editor. "C:\Program Files\Mozilla Firefox\firefox.exe"
is a string. Launching something with spaces in its name requires the call operator: & "C:\Program Files\Mozilla Firefox\firefox.exe"
. PowerShell’s tab completion will add the &
if necessary.
There are two function call syntaxes. Calling a function/cmdlet uses the shell-style syntax with argument names: Some-Function -Arg1 value1 -Arg2 value2
, and argument names can be abbreviated, and can sometimes be omitted. Calling a method requires a more traditional syntax: $obj.SomeMethod(value1, value2)
. Names are case-insensitive in either case.
The escape character is the backtick. The backslash is the path separator in Windows, so making it an escape character would make everything painful on Windows. At least it makes it easy to write regex.
The ugliest part
Conclusion
PowerShell is a fine interactive shell and scripting language. While it does have some warts, it is more powerful than your usual Unix shell, and its strongly-typed, object-oriented code beats stringly-typed sh
spaghetti any day.
.NET HashSet in PowerShell
A HashSet<T>
is another generic .NET collection class similar to a List<T>
with a few notable differences.
- It is unordered (though there is also a
SortedSet<T>
) - All items in the HashSet must be unique (no duplicates)
- It cannot be indexed into using
$HashSet[<int>]
As we’re now familiar with most of the syntax, let’s jump straight into the examples:
Creating the HashSet<T>
object:
namespace SystemCollectionsGeneric
= ::New
Adding, searching for, and removing items:
# Adding an item
C:\> Add1
True
# Checking if the set contains an item
C:\> Contains1
True
# Duplicates are not permitted
C:\> Add1
False
# Removing an item
C:\> Remove1
True
As HashSet<T>
implements IEnumerable<T>
we can use LINQ without any casting. Let’s look at slightly more complex example than before. We’re going to create two HashSets, the first will be a set of IP addresses we “trust”. The second will be a list of IP addresses perhaps pulled from some access logs. What we want to do is return a collection of unique IPs in the second set, but exclude the “trusted” IPs from the first set.
namespace SystemCollectionsGeneric
= ::New \trustedtxt
= ::New \accesslogtxt
Here are the contents of those two files:
102501100
102501101
192168150
102501120
102501100
215714595
102501100
102501100
102501100
125286035
125286035
We can see our accesslog file has several duplicates, but once we add it to our set the duplicate values are no longer present.
C:\> ExpandProperty IPAddressToString
192168150
102501120
102501100
215714595
125286035
Now let’s use LINQ to extract the logged IPs but exclude our trusted addresses.
C:\> = ::Except
C:\> Expand IPAddressToString
192168150
102501120
215714595
125286035
That wraps it up. Hopefully you’ve not only discovered some useful .NET classes but also have a bit of insight into how to find and use other types that haven’t been covered here.
Using LINQ with other .NET collection types
LINQ works with any collection class that implements IEnumerable<T>
, not just lists. So what is IEnumerable<T>
and how do we find out what “implements” it?
.NET Interfaces
Interfaces are a contractual blueprint for classes that define what methods and properties implementing classes must provide. The classes that implement that interface then define how those members are implemented.
For example, we could have an IAnimal
interface (the convention for interface naming is to begin with a capital I
) which states that any classes implementing it must have a MakeNoise
method that returns a string
(e.g., “Woof”).
When we then create our various animal classes we make them implement the IAnimal
interface, and in each animal class we will need to write the MakeNoise
method (or we will get IDE / compilation errors) and have it return the noise of that specific animal.
Using interfaces means we can have confidence that regardless of the animal, there will be a MakeNoise
method that returns a string
. There are other advantages to interfaces such as decoupling which helps facilitate changes but none of that is important for our needs here.
Back to LINQ – it works with classes that implement the IEnumerable<T>
interface. How do we find what those are?
There are two ways in PowerShell.
GetInterfaces
We can use the GetInterfaces
method:
C:\> GetInterfaces
IsPublic IsSerial Name BaseType
True False IList`1
True False ICollection`1
True False IEnumerable`1True False IEnumerable
True False IList
True False ICollection
True False IReadOnlyList`1
True False IReadOnlyCollection`1
We can see List<T>
list implements both IEnumerable
and IEnumerable'1
, so what’s the difference? The the backtick and digit indicates this is a generic type (List<T>
instead of just List
) and refers to the number of type parameters it accepts.
For example, List<T>
only accepts one, but a Dictionary<T key,T value>
accepts two (one for the key and one for the value), as seen below:
C:\> GetInterfaces
IsPublic IsSerial Name BaseType
True False IDictionary`2True False ICollection`1
True False IEnumerable`1
True False IEnumerable
True False IDictionary
True False ICollection
True False IReadOnlyDictionary`2
True False IReadOnlyCollection`1
True False ISerializable
True False IDeserializationCallback
PowerShell native arrays only implement IEnumerable
, not IEnumerable<T>
, so they don’t work with LINQ without casting (more on this below).
C:\> GetInterfaces
IsPublic IsSerial Name BaseType
True False ICloneable
True False IList
True False ICollection
True False IEnumerableTrue False IStructuralComparable
True False IStructuralEquatable
Let’s try it out:
C:\> = @123456789
C:\> ::Sum
MethodException: Cannot find an overload and the argument count:
The second method to determine whether something implements an interface is to use IsAssignableFrom
.
IsAssignableFrom
First we need to define our type, then the interface, and then call the IsAssignableFrom
method.
C:\> =
C:\> =
C:\> IsAssignableFrom
True
Let’s check PowerShell arrays.
C:\> =
C:\> =
C:\> IsAssignableFrom
False
We can also check the class documentation.
Documentation
Here is a screenshot showing what interfaces List<T>
implements.
‘Using’ statements
‘Using’ statements allow us to use types without having to reference the entire namespace each time we want to interact with the class. For example, if we wanted to create a List<T>
(a .NET class we will cover below) we can do so in two ways.
The first is to specify the full namespace path:
= ::New
The second is to place a using
statement at the top of the script, and then reference the type only by its name.
namespace SystemCollectionsGeneric
= ::New
We already have a similar concept to this in PowerShell. We can run Get-Process
by itself, or we can reference the module-qualified cmdlet with Microsoft.PowerShell.Management\Get-Process
. If we had two commands that mapped to Get-Process
we could remove ambiguity by writing out the full path.
Back to using List<T>
.
List<T> in PowerShell
However, when we attempt to perform some task against each item in the collection (as is common) we’re likely to encounter errors.
C:\> @1020 2
5
10
InvalidArgument: Cannot convert value to Error: "The input string 'thirty' was not in a correct format."
With generics we specify what type will be contained inside the collection. Let’s begin with creating a List<T>
to hold integers.
namespace SystemCollectionsGeneric
= ::New
Add10
Add20
Now when we add “thirty” we will receive an error.
C:\> Add
MethodException: Cannot convert argument with value: to : "Cannot convert value "thirty" to type "SystemInt32". Error: "The input string was not in a correct format
List Caveats
= ::New
Add
Add
Add30
Here the integer 30
was automatically cast to the string "30"
.
C:\>
ten
twenty
30
C:\> 2GetType
IsPublic IsSerial Name BaseType
True True String SystemObject
This appears to be a PowerShell specific behaviour as the equivalent C# code will throw an exception.
Numbers
Numbers
Numbers
Numbers
Error CS1503 Argument cannot convert from '' to ''
Another caveat is that List<T>
accepts null
values regardless of the type specified.
C:\> = ::New
C:\> Add
C:\>
0
C:\> = ::New
C:\> Add
C:\> Count
1
Setting aside tangents and caveats, List<T>
vastly outperforms PowerShell arrays, offering significant performance improvements.
Using LINQ in PowerShell
LINQ (Language Integrated Query) is a .NET component that allows for data querying and filtering. Think of a combination PowerShell’s Where
, Measure-Object
, and Select
cmdlets, but usually a lot faster than native PowerShell cmdlets, especially when dealing with large collections.
Let’s look at a couple of simple examples.
List comparison with LINQ in PowerShell
We’re going to create a few lists and compare their contents with LINQ, including the order of the items in the lists.
=
=
= # Same items, different order
::SequenceEqual
::SequenceEqual
# False due to ordering being different
There is a Sort
method we can use if we have lists where the items may have different ordering:
::SequenceEqual
::SequenceEqual
Getting minimum, maximum, average values from a collection
= ::new
1100 Add Minimum 1 Maximum 1000
C:\> ::Min
32
C:\> ::Max
983
C:\> ::Average
51395
C:\> ::Sum
51395
As a very quick performance comparison let’s see how LINQ compares with Measure-Object
for calculating the sum of one million numbers.
= ::new
11000000 Add Minimum 1 Maximum 100
C:\> 110 Sum ExpandProperty TotalMilliseconds Average
Average : 85756657
C:\> 110 ::Sum ExpandProperty TotalMilliseconds Average
Average : 045215
In this example LINQ is ~1897 times faster than piping to Measure-Object
There are too many LINQ methods to cover here, and I’ve only scratched the surface to show the most basic syntax. Michael Sorens has written a fantastic article on using LINQ with PowerShell titled High Performance PowerShell with LINQ that I recommend you check out.
Determining what version of .NET CLR PowerShell is using
Different .NET/Core versions support different classes and behaviors. Knowing the .NET CLR (Common Language Runtime) version PowerShell is using helps you refer to the correct documentation and troubleshoot unexpected behaviors.
PowerShell 5.1 uses .NET Framework 4.0.30319.42000.
C:\> PSVersionToString
51190413996
C:\> ::VersionToString
403031942000
PowerShell 7.4.1 uses .NET Core 8.0.1
C:\> PSVersionToString
741
C:\> ::VersionToString
801
We can already see a big difference here, v5.1 uses the older .NET Framework while 7.x uses the more modern .NET Core. This is why PowerShell 7.x can be cross platform, and why it can’t ship with Windows as Microsoft does not want to bundle the .NET Core runtime due to support lifecycles of the products being different (at least that is my understanding).
There will be an example below where the .NET CLR version is important.
Let’s begin with something simple, validating whether a string is null, empty, or whitespace.
IP Address validation using [IPAddress]
C:\>
Address : 16843018
AddressFamily : InterNetwork
ScopeId :
IsIPv6Multicast : False
IsIPv6LinkLocal : False
IsIPv6SiteLocal : False
IsIPv6Teredo : False
IsIPv4MappedToIPv6 : False
IPAddressToString : 10111
However, it has limitations; for example, 10.1
is considered a valid value:
C:\>
Address : 16777226
AddressFamily : InterNetwork
ScopeId :
IsIPv6Multicast : False
IsIPv6LinkLocal : False
IsIPv6SiteLocal : False
IsIPv6Teredo : False
IsIPv4MappedToIPv6 : False
IPAddressToString : 10001
However, we can get around this by comparing the input against the IPAddressToString
output:
IPAddressToString
Here are some examples:
C:\> @ IPAddressToString
True
False
False
False
InvalidArgument: Cannot convert value to Error: "An invalid IP address was specified."
InvalidArgument: Cannot convert value to Error: "An invalid IP address was specified."
InvalidArgument: Cannot convert value to Error: "An invalid IP address was specified."
The second value of 10.1
while accepted as valid input for the class failed the equality check, and the same goes for the third hex value and the fourth decimal value. Both are ‘valid’ input for the class as they’re just different ways a valid IP can be represented, but if we’re expecting the usual dotted-decimal IP representation in our input they fail the string equality check.
The last three are invalid IP addresses and return errors.
While we’re on the topic of networking, let’s look at MAC Addresses.
Why use .NET classes in PowerShell?
There are two main reasons:.
- Extended Functionality: Access features in .NET not available natively in PowerShell.
- Enhanced Performance: Many .NET classes perform faster than their PowerShell equivalents.
Exploring .NET also helps deepen your understanding of the foundation upon which PowerShell is built.
Casting PowerShell arrays to an IEnumerable object
We may already have an existing array that we would like to use with LINQ, and in these cases LINQ provides a casting method that returns a usable objects which implements IEnumerable<T>
.
= @1 2 3 4 5
= ::Cast
# We can now use LINQ
C:\> ::Average
3
C:\> GetType
IsPublic IsSerial Name BaseType
False False <CastIterator>d__68`1 SystemObject
Let’s move onto the final .NET class for this post: HashSets
.NET Syntax in PowerShell
We have explored several .NET classes, which might raise questions about their syntax in PowerShell.
Understanding [ClassName]::MethodName() Syntax
Here it is shown in the Microsoft documentation for the PhysicalAddress
class.
When we use the usual .
notation to invoke a method (as we did with $List.Add()
) this the same as invoking a method on a PowerShell object, for example $SomeString.ToUpper()
.
Default imports
C:\> ::CurrentDomainGetAssemblies ? Location
GAC Version Location
False v4030319 C:\Program Files\PowerShell\7\SystemCollectionsdll
False v4030319 C:\Program Files\PowerShell\7\SystemCollectionsConcurrentdll
False v4030319 C:\Program Files\PowerShell\7\SystemCollectionsSpecializeddll
False v4030319 C:\Program Files\PowerShell\7\SystemCollectionsNonGenericdll
False v4030319 C:\Program Files\PowerShell\7\SystemCollectionsImmutabledll
C:\> ::New
InvalidOperation: Unable to find
In short, some namespaces will be imported by default, others you’ll need to specify even if the assembly is loaded.
Microsoft’s documentation is an excellent resource for mapping classes to namespaces and assemblies.
And here is a list of all classes in the System.Collections.Generic
namespace. I will briefly cover HashSets, but I’d recommend exploring others as a learning exercise.
Loading custom or third party assemblies
Sometimes the classes we want to work with are in third party assemblies. We can load these using Add-Type
. As this is slightly out of scope for this post and I’ve shown examples of it before, you can check it out here.
Fast TCP port testing with [System.Net.Sockets.TcpClient]
This is one of my favourites simply because it’s so much faster than Test-NetConnection
thanks to the configurable timeout.
Here is an example:
C:\> =
C:\> = 443
C:\> = 100
C:\> = ::New
C:\> ConnectAsyncWait
True
C:\> Dispose
It returns $true
if the connection was successfully established, and $false
if not.
String validation
.NET has very two very convenient string validation methods built into the String class, IsNullOrEmpty
, and IsNullOrWhiteSpace
. While not difficult, performing this kind of validation in PowerShell 5.1 requires a bit more work than just calling a method.
Here are a few examples:
=
=
=
=
::IsNullOrEmpty
::IsNullOrEmpty
::IsNullOrEmpty
::IsNullOrEmpty
::IsNullOrWhiteSpace
::IsNullOrWhiteSpace
::IsNullOrWhiteSpace
::IsNullOrWhiteSpace
IsNullOrEmpty
True
True
False
False
IsNullOrWhiteSpace
True
True
True
False
String validation natively in PowerShell
PowerShell 5.1 has ValidateNotNullOrEmpty
built in, while PowerShell 7 contains both validators in the form of attributes. Here is an example:
= # Works in PowerShell 5.1+
= # Works in PowerShell 7+
Using the .NET method however works across all versions.
Moving on, let’s tackle IP address validation
Generating passwords with [System.Web.Security]
I’m sure at some stage we’ve all needed to programmatically generate passwords. The System.Web.Security
namespace has a nice method for doing this. The downside is that it did not make it across to .NET Core, so it only works on PowerShell 5.1.
The syntax is GeneratePassword (int length, int minimumNumberOfNonAlphanumericCharacters)
Here are a couple of examples:
C:\> ::GeneratePassword201
7!=vvIOh45ib3p+dU&AGSvHy42+
C:\> ::GeneratePassword2010
g2Ht@AM-pv&iz+dNU
Let’s move onto some .NET classes that have a wider use cases.
MAC Address Validation and Normalisation using [PhysicalAddress]
As with IP addresses we can use regex and some string manipulation here, but .NET again has a convenient method to make our code a little cleaner and easier to read.
Valid MAC address example:
C:\> ::Parse
34ED1BAABBCC
Invalid MAC address example:
C:\> ::Parse
Exception calling with arguments: "An invalid physical address was specified."
At line:1 char:1
::Parse
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
CategoryInfo : NotSpecified: : MethodInvocationException
FullyQualifiedErrorId : FormatException
Compatibility Across .NET Versions
If you recall earlier the underlying version of .NET CLR can make a difference, this the is one class where Microsoft has made improvements between .NET Framework (used by PowerShell 5.1) and .NET Core.
Unfortunately the above format (all uppercase, -
delimited) is the only format valid under .NET Framework. All of these valid and commonly formatted MAC address values fail:
C:\> ::Parse
Exception calling with arguments: "An invalid physical address was specified."
C:\> ::Parse
Exception calling with arguments: "An invalid physical address was specified."
C:\> ::Parse
Exception calling with arguments: "An invalid physical address was specified."
However, they all work under PowerShell 7.x which is built on top of .NET Core.
C:\> ::Parse
34ED1BAABBCC
C:\> ::Parse
34ED1BAABBCC
C:\> ::Parse
34ED1BAABBCC
These changes are documented in the Microsoft documentation.
Sticking with the networking theme, let’s test TCP ports