Chad is a one-man IT guy in a small business. Chad wears many hats and is the proverbial jack of all trades master of none. He has so many responsibilities he doesn’t have time to dive specifically into any one technology even though he really wants to. One day, the dying Exchange 2003 server that hosts all the company’s mail on seems to be on the fritz. Chad’s fielding helpdesk calls like a mad man with one hand with another hand on a keyboard frantically trying to log into the server to see what happened.
After he’s managed to notify everyone in the company that email is down, he can finally get to work troubleshooting what’s going on. Chad thinks, “It looks like inbound email is the problem. If I send a test email from my personal account, I never receive it in my mailbox.” Knowing that inbound email can be tracked on the wire, Chad installs Wireshark on the Exchange server, starts a packet capture, tries to send himself another email and stops the packet capture. Chad believes he’s successfully captured the problem in his newly generated PCAP file and now all he has to do is quickly scan the contents and get to the root of the problem.
Three hours and more angry phone calls later Chad is still trying to decipher his packet capture. “There’s so much activity in this thing! I know that email goes over TCP port 25 and I see some activity but I don’t see any obvious signs of a problem!”, he shouts to himself. He’s getting increasingly frustrated by the minute until he gives up and decides to check the event log. He sees an Exchange-related error message, searches for the error online and immediately sees a hotfix that was recently released by Microsoft just for the sort of problem he’s having. So he decides to give it a shot.
Chad installs the hotfix, reboots the server and all the mail that was queued up begins being released to mailboxes. It turns out that mail was actually getting to the Exchange server without issue. This was the activity Chad was seeing in the packet capture. The problem was with the Exchange server software itself. Chad is now finally able to get back to his normal daily fire-fighting.
This may be an extreme example but have you or have you seen co-workers do something similar to this? You might have a passing familiarity with Wireshark and know that the problem involves network traffic somehow and immediately be drawn to that as one of your first approaches. Resist that temptation. Wireshark (or other packet capture software) should be your last resort unless really know a protocol’s behavior. You may not think so, but packet sleuthing is a very complicated activity. There’s even a Wireshark University! There are much simpler ways to track down problems instead of spending hours trying to decipher something that might turn out to not be a problem at all.
One of the simplest starts with asking and answering these three questions.
Step #1: What Changed?
Troubleshooting is an art. It is a skill that’s mastered over time. Diving straight into a packet sniffer to troubleshoot a problem is about as advanced as you can get. Let’s start simpler, shall we? Every problem comes from a change. Since Chad is the only IT guy, he should have first thought back to himself, “Have I changed anything that might have affected this?” If so, the fix might be a simple revert of the change.
Step #2: Have you used the Event Log?
Chad’s problem was with an on-prem Exchange server. The server was running a version of Windows Server. Where’s the first place you should go when troubleshooting any problem on a Windows server? The event log. Nearly every Windows application writes something to the event log or uses software that writes to the event log.
The event log contains lots of information and can sometimes be overwhelming, so set up filters to filter out everything but errors and warnings. The information will be cut down drastically to allow you to more easily parse through and pinpoint the problem.
Step #3: Does the application generate text logs?
Many applications produce their own text logs. These logs typically contain more detailed information about the activity of the software and can sometimes be used to track down problems. If troubleshooting an application for the first time you probably don’t know where the log files are located. In this case, fire up your search engine and perform a search on the application and the words “log files”. This will typically turn up the log file locations somewhere.
These are just three basic steps to follow when troubleshooting. As you get more experienced, troubleshooting will become easier. Just remember: put down Wireshark! Wireshark is just a single tool and is not meant to be the first goto tool. Follow the steps I’ve outlined here first and I assure you you’ll track down 90% of the issues without reverting to a packet sniffer.
