Processing CSV data in PowerShell


In this post, I explain how to create a PowerShell function to process CSV data. This allows you to reuse your code whenever you are working with CSV files in PowerShell.

CSV (Comma-Separated Values) is used by almost every technology platform that we encounter. Manipulating this data can be cumbersome if you’re NOT an Excel wizard, but PowerShell can simplify this job. For example,

Read More


Design Before Coding: Kanban

The first post in this three-part series focused on gathering requirements before spending time coding.  While it is not necessary, I recommend that everyone read part 1 (Gathering Requirements).  Understanding and having clear “User Stories” outlining the intended purpose of a tool is crucial, especially as we move towards building our new tool.

Once our “User Stories” have been defined and agreed upon by all stakeholders, we then need to begin breaking them down into tasks.  There are many different methodologies that allow us to track these different tasks. Part 2 of this three-part series will focus on one of them; Kanban.

I use Kanban daily, both personally and professionally.  If you want to understand the principles of Kanban, then I recommend that you start with this Wikipedia page. No matter what methodology or process you use, having a consistent and repeatable utility is a must.

This post will go over using Kanban both digitally and manually, but first you must understand the general purpose of Kanban.  Tracking our work can be difficult, especially with the use of E-mail, Slack, Skype, Teams, Yammer, HipChat, etc. etc.  The constant communication that is available to us is overwhelming, but having a central location that you can organize your “To-Do’s” or tasks is necessary.  Not for our management or organizations tracking purposes, it is so that we can track exactly what we need to do next.


The theory of Kanban is to visually identify items that need to be worked on. Kanban allows you to understand what tasks need to be worked on, which ones are in-progress, and which ones are complete.  Using this information, we begin to identify any blockers or obstacles that may need to be fixed so that an item/task can move from “To Do” to “Done”.  This principle helps us identify our WIP (Work in Progress) so that the flow between “gates” or “hand-off points” is as smooth and controlled as possible.  Think of this like an assembly line:

Let’s say we have two workers in an assembly line.  The first person’s job is to paint a car door handle, the second person’s job is to install that door handle on a new car.  If the first person can paint 5 handles in 10 minutes, but the second person takes 10 minutes to install the handle; we have a problem.  This means that we either need to increase staff that install car door handles or maybe the same person should do both.

Understanding these blockers will help us identify gaps in our processes, or the lack of labor to accomplish our goals.  Visually displaying this information on a Kanban board is much easier to understand than a bunch of tasks lists or “tickets”.

There are many tools available for a digital Kanban board.  I have listed a few below that are both free and paid.  In my professional life, I use Jira’s Kanban feature.  In my personal life, I use both a whiteboard with post-it notes and built-in features of GitHub Projects (example):

If you just do a quick google search for “free Kanban tools” or go to and search for “Kanban” you will find many open source projects that you can host yourself.  If you don’t want to host it yourself, then check out  If you think a digital tool would work for you, you can even use Outlook (here or here) or Gmail to manage your tasks in the Kanban style.

I’m not sure about you, but I personally carry a leather-bound notebook (with DoanePaper) with me wherever I go.  Whether at work, out with friends, going to bed, etc., I always have my notebook.  In my opinion a digital tool is great, but I’m faster at writing things down on paper than I am opening an app.  Plus, I always have it with me.


Another option or addition is by using a whiteboard in your home or work office.  The first thing you do is draw some columns.  Typically, you have three different columns: To-Do, In-Progress, and Done.  Starting out you will have everything in your To-Do column.  As you begin a new task you move that sticky-note to the In-Progress column.  If that item gets blocked or cannot be completed, you should move it back to the To-Do column.  You should only have a few items In-Progress (ideally, you would only have one).  Once you have completed that task, you move it to the Done column.

That’s it! Kanban allows you to visually see tasks and progress you are making on your projects.  The manual process is great for a single person or a team that is in close quarters.  If you’re spread out or in a cubicle this may not work as well.  This is where having a platform like Jira or Trello really shines.  Especially for teams that are global or in different locations.

Kanban is powerful, yet extremely simple. Using a tool to manage your tasks so that you understand exactly what you, your team, and others are working on reduces potential re-work while also making sure that all tasks for a project are complete (and on-time).  The third post in this series I will take the concepts of both post one and two, and we will begin to breakdown our User Stories onto our Kanban board.  Remember, we have not started coding yet.  Next, we will design how our stakeholders may use our new tool.  Understanding this, will speed up our development time and reduce re-work.

Design Before Coding: Gathering Requirements

If you have worked in IT for any duration, I’m sure you have overheard or been asked to build a tool to complete X or Y. Creating tools with PowerShell is fun, but it can become daunting when you create a tool that does not meet its intended purpose.  Without understanding the full requirements, you may waste time and energy on developing a tool that no one will use.

Creating tools with PowerShell to automate a manual process or to help an internal stakeholder accomplish a desired result, typically does not need to turn into a large initiative with a Project Manager or Project Management Office (PMO). Being tasked with creating these tools usually comes in the form a short conversation or through an email.  Out of habit, we usually dive right into writing a script or function to solve the problem.  This approach can cause a lot of re-work or redesign of our tool once complete.  Even though we believe we understand all the requirements, it is better to have the stakeholder create a “Goal Statement” that defines the intended purpose of the tool.  The “Goal Statement” helps everyone involved understand when the initiative is done.

A “Goal Statement” does not need to be a large body of work, it can simply be a couple of sentences or paragraphs.  Personally, I take the Scrum/Kanban approach and use User Stories.

User Stories are typically designed in the following format (there are different styles, but in my experience this the simplest form):

As a <type of user>, I want <goal> so that I <receive benefit>.

Having a defined User Story reduces re-work and ensures that all stakeholders involved agree on the intended uses of this new tool.  Agreement on the intended results of this new initiative may not solve the problem completely, but it’s a great start!

At this point, we have not written any scripts, functions, modules, classes, etc.  You may want to dive right into writing a POC (Proof of Concept), but I recommend that you hold-off.  Once all stakeholders have agreed on our User Story, we should move to the design phase (Part 3 coming soon).

Let’s say that we work on a “Automation Team” in our organization that focuses on building tools to streamline processes for both IT and business teams.  We have been tasked with helping the organizations IT managers identify and verify that all Active Directory groups in our Forest have the correct owners associated with them.  As we start our requirements gathering phase, we ask all stakeholders to provide our team with agreed upon User Stories.  Our team receives the following:

As a manager, I want to know all Active Directory groups owned by myself so I can ensure that they are correct. 

As a manager, I want to know all Active Directory groups owned by my employees so that I can ensure that they should have access. 

As a manager, I want to know all Active Directory groups that do not have an owner but reside in my Active Directory OU so that I can assign the correct owner.

Now that we have our agreed upon User Stories, we can begin the next phase; designing the “look and feel” of our new tool based on the requirements we have been given.  Having a general idea of what our stakeholders are needing reduces our work effort, as well as setting clear expectations that are agreed upon.

Understanding why we need to gather requirements is the first step.  The next post in this series I will discuss how you can use Kanban, digitally and manually, to organize our tasks so that you or your team can keep track of the status/progress along the way.

The final post in this series, we will begin designing our code layout.  This will help us and our stakeholders understand what parameters need to be present, what objects should be accepted in the pipeline, what return objects should look like, and how the new tool will be used.

PhishReporter: PowerShell Module

If you work in the Info Sec world you know that phishing is a pain in the neck, especially when you’ve been targeted by a large phishing campaign.  I’ve been through these massive phishing attacks, and it is not fun!

One such attack, about a year ago, was especially difficult because the attackers were using very unique tactics and spoofing lots of internal e-mail communications.

Our procedure was to go and find the company that was hosting these phishing URLs and e-mail them to tell them to shut the site down immediately.  Hosting companies like,,, etc. were pretty good at shutting these sites down as soon as they were put online, but this process would take about 5 to 10 minutes and sucked!

Since this last attack, I decided to automate this process.  This new script/tool is a PowerShell Module called PhishReporter.

The PowerShell module can be downloaded on GitHub:

This PowerShell Module is designed to send notifications to hosting companies that host phishing URLs by utilizing the major WHOIS/RDAP Abuse Point of Contact (POC) information.

  1. This function takes in a .msg file and strips links from a phishing URL.
  2. After getting the phishig email, it is then converted to it’s IP Address.
  3. Once the IP Address of the hosting website is identified, then we check which WHOIS/RDAP to search.
  4. Each major WHOIS/RDAP is represented: ARIN, APNIC, AFRNIC, LACNIC, & RIPE.
  5. We call the specific WHOIS/RDAP’s API to determine the Abuse POC.
  6. Once we have the POC, we send them an email telling them to shut the website down. This email contains the original email as an attachment, the original phishing link, and e-mail body telling them to remove the website.

This Module came out of necessity. I was sick of trying to contact these individual sites, so I have began automating our response time to these events.

The next steps for this project are to fully integrate into Outlook and automate this even further by enabling a simple text search or based on a selected ‘folder’ event. Please share with the Security community and contribute/improve as you deem fit. I only ask that you share your edits back with this project.