Can ChatGPT Write PowerShell Code?

We put ChatGPT’s PowerShell scripting abilities to the test. See how the AI chatbot performed.

Brien Posey

April 18, 2023

4 Min Read
a robot and a man face one another
Alamy

As amazing as ChatGPT is, I recently heard several developers express concern that the platform might eventually put them out of a job because of its ability to write code. Conversely, other developers have said that ChatGPT is of no concern because ChatGPT writes code poorly.

The simple fact that developers are talking about ChatGPT’s code-writing capability got me thinking about PowerShell. Specifically, could ChatGPT write PowerShell code? If so, are its scripts well written?

Test #1: A Basic PowerShell Script

I began my tests. I asked ChatGPT to help me to write a simple “Hello World” script using PowerShell. As you can see in Figure 1, ChatGPT provided instructions for creating a Hello World script, right down to explaining how to open PowerShell.

ChatGPT provides instructions for how to write Hello World script in PowerShell

Chat GPT for IT 2_0

Figure 1. ChatGPT gave me instructions for writing a Hello World script.

The experiment shown in Figure 1 convinced me that ChatGPT can indeed write PowerShell scripts. Even so, explaining how to use PowerShell for performing a simple task isn’t the same as generating a usable script. As such, I decided to do another experiment.

Test #2: More Complicated PowerShell Code

This experiment involved writing a PowerShell script myself and then asking ChatGPT to write the same script. I would compare the scripts. I didn’t worry about trying to see who could write the script fastest. Even though I work with PowerShell every day, ChatGPT would almost certainly write the code more quickly than I could.

Related:What Is AI-Assisted Coding, and Should You Use It?

For my test, I wanted to create a more complicated script than the Hello World script yet not super-complex. After all, my goal was to compare AI-generated code against my code, and that can be accomplished without writing a long, arduous script.

I decided to ask ChatGPT to create a simple script to count from 1 to 100, wait 250 milliseconds between each number, and display a progress bar.

Here is the PowerShell code that I wrote:

For ($Count = 1; $Count -le 100; $Count++ ){    Write-Progress -Activity “Counting” -Status “$Count% Complete:” -PercentComplete $Count    Start-Sleep -Milliseconds 250}

Example PowerShell code used in PowerShell

Chat GPT PowerShell 2

Figure 2. Here is what it looks like when I run my script.

Having seen my code, let’s look at ChatGPT's code. You can see my query and the ChatGPT results in Figure 3. (In case you are curious, I didn’t time myself against ChatGPT, but it took me three or four minutes to write my code, and ChatGPT less than a minute to generate its code.)

$progressPreference = 'SilentlyContinue'for ($i = 1; $i -le 100; $i++) {    Write-Progress -Activity "Counting from 1 to 100" -Status "Counting..." -PercentComplete ($i) -SecondsRemaining ((100 - $i) * 0.25)    Start-Sleep -Milliseconds 250}

ChatGPT produces PowerShell code

Chat GPT PowerShell 344

Figure 3. Here is the code that ChatGPT created.

Although the pair of PowerShell codes are nearly identical, the code that ChatGPT generated did not work when I tried it. The code did indeed count to 100 and wait 250 milliseconds between each number. However, the code did not display the progress bar.

I have worked with PowerShell for long enough to know that the first line of ChatGPT’s code caused the problem. If you removed that line, the code would work fine. Even so, the purpose of this experiment was to see if ChatGPT could write reliable code, not if I can debug the code.

I decided to click the Regenerate Response button to see if ChatGPT would produce better results. Interestingly, ChatGPT regenerated the code using a While loop, as shown in Figure 4.

ChatGPT regenerates the PowerShell code, this time using a While loop

Chat GPT PowerShell 465

Figure 4. ChatGPT has rewritten the code using a While loop.

Despite these revisions, ChatGPT’s code still does not work. I wanted to see if I could help ChatGPT to write better code. I told ChatGPT what the problem was, as shown in Figure 5. ChatGPT acknowledged its mistake and then recreated the script. This time the script worked properly.

ChatGPT fixes broken PowerShell script

Chat GPT PowerShell 5

Figure 5. ChatGPT fixed the broken script it had written.

Assessment of ChatGPT’s PowerShell Coding Abilities

My assessment: ChatGPT is indeed capable of writing PowerShell code but isn’t perfect (yet).

What surprised me about these experiments was what followed when I clicked the Regenerate Response button: ChatGPT rewrote the script to use a different type of loop. This suggested that ChatGPT is probably actually writing the PowerShell code itself rather than just reusing a script it found online.

About the Author

Brien Posey

Brien Posey is a bestselling technology author, a speaker, and a 20X Microsoft MVP. In addition to his ongoing work in IT, Posey has spent the last several years training as a commercial astronaut candidate in preparation to fly on a mission to study polar mesospheric clouds from space.

http://brienposey.com/

Sign up for the ITPro Today newsletter
Stay on top of the IT universe with commentary, news analysis, how-to's, and tips delivered to your inbox daily.

You May Also Like