What Makes a Good Analyst

Technical Depth Understanding Log Sources Knowing how a tool works Having experience being on the other side (As the attacker)

All of the above mentioned aren't wrong completely. If you ask 10 cybersecurity analysts what makes someone "good at investigations", and you'll get 10 different answers. Chris Sanders in his doctoral dissertation (The Analyst Mindset), asked and answered this very same question. And the result he got wasn't one of the above mentioned.

Let's be clear: Having technical depth, Understanding the log sources and the environment architecture, Having experience and knowledge on the offensive tradecraft, are all very good skills to have for an analyst. But the main one is far simpler, yet it takes practice and time to accumulate.

First take a close look at the following scenario, and what questions did the analysts try to ask first?

An event got generated where there is a PowerShell encoded base64 command.

Analyst A: Is this malicious?

Analyst B: What does this base64 command do?

Notice the two questions asked. One is too broad, the other one is specific and spot on. One is going to move the investigation further, and have better follow-up questions, and the other will probably fail at knowing which route to take next.

Chris Sanders researched 10 experts analysts, and observed how do they navigate through investigations, By asking the right questions.

It's all about asking the right question!

Asking the right question is what separates good and bad analysis. I know it seems so simple, but believe me it is not!. It's something that takes experience to get to. However, Chris Sanders gives us the anatomy of the right question. Which will help us to understand and apply these characteristics into our own investigations. Have a look at the following three questions, and try to pin point the three commonalities in them.

Question 1: "Did this user login anywhere else in the environment in the last 24h?" Question 2: "What is the parent process of RunTime.exe with PID number 2263 running on this host?" Question 3: "When was this service account created?"

Good questions have three main characteristics: Relevant, Specific, Answerable.

Relevant: the question is grounded in evidence you already have, not a guess out of nowhere.

Specific: the answer comes back in a narrow shape. a name, a yes/no, an IP.

Answerable: the evidence needed to answer it actually exists in your environment.

Take a look again at the two questions the analysts asked in the first scenario. Analyst A questions don't pass the three characteristics. While Analyst B questions does.

The question Analyst B asked, is relevant, specific, and answerable.

The question Analyst A asked, is relevant, but not specific, and not answerable yet.

One analyst is moving towards the right direction with steadiness. The other is still trying to figure out what to ask!.

So Analyst B is moving forward. But what's the next question they ask?

Sanders observed three types of questions expert analysts ask:

Event-Relative Questions

These questions are directed at the event itself. Whether asking about Succeeding events i.e. "Was there an outbound connection from this program after it got executed?".

Preceding events i.e. "Who was the parent of this powershell.exe process?".

Context of the event. i.e. "This process was initiated under which user account?".

Proximate i.e. "Was there any other successful RDP connection in the last 7 hours?".

Capability-Matching Questions

This type is a bit different. The previous type Event-relative questions start from evidence. You see an event, and you ask what happened before it, after it, or around it. Capability-matching questions start from your own knowledge. You know how a technique works, so you ask the environment: "If this technique was used here, where would it show up?"

To make it clear, Let's go back to Analyst B. He decoded the base64 command, and found it downloads a file from an external IP then executes it. Now he understands the technique. So the next question becomes: "Did this same download-and-execute pattern happen anywhere else in the environment?". Notice that this question didn't come from an alert or a log. It came from his head. He took a capability he now understands, and matched it against the environment.

Utility Questions

Utility questions are a bit strange, they don't tell you exactly what is happening. But they offer in my opinion a tremendous value to the analyst. Let me give you a couple of examples of utility questions then I will explain it. "Who are the Domain Admins in this environment?", "Is RDP permitted to be used on all machines?", "In what department does this user reside?", "Do we have Sysmon on this host, and is it configured to log DNS queries?". As you can see these questions fits the three criteria's of the right question. Yet, They aren't directed at the event itself. The value of these questions might be immediate or delayed. Think about how different you would approach the investigation if you have known that a beacon is running from a host that resides in the financial department, as opposed to a beacon running from a host in the marketing department. One is in a very sensitive place, and probably has to be eradicated right now. The other is still dangerous, but the beacon running there, still buys us some time knowing they aren't one step away from the crown jewels. (Also as a side-note, It's great if you can gather this information before you touch the keyboard. As it can help you plan strategically in your investigation).

Now, Let's watch Analyst B finish the job

It's time to put everything together, and watch the full chain of questions Analyst B asked from the beginning:

"What does this base64 command do?" He decodes it.

Answer: It downloads a file from an external IP and executes it. (Event-relative: context)

"Was there an outbound connection from this host after the command ran?" Answer: Yes, To the same IP. (Event-relative: succeeding)

"What process launched this PowerShell command in the first place?" Answer: "winword.exe". Now he knows the entry point: a malicious document. (Event-relative: preceding)

"Which user account opened this document, and in what department do they reside?" Answer: "An accountant in the finance department." Now he knows the stakes. (Utility)

"Did this same behavioral pattern happen anywhere else in the environment?" Answer: "Two more hosts". The incident just changed size. (Capability-matching)

Notice, Analyst B never asked "Is this malicious?". He never needed to. The answer just revealed itself by asking specific, relevant, answerable questions. Meanwhile Analyst A is still staring at his first question, because it wasn't the right question. It was a conclusion, masquerading as a question.

The LOOP

If we zoom out from the story above, we will see a pattern being repeated. Analyst B reads a piece of evidence/event. Sees something interesting. He forms a question. He gets an answer. And the answer becomes the next piece of evidence. Then the loop runs again.

Evidence → something interesting → question → answer → new evidence. Then again. And again. Until the picture is complete, or until you run out of questions. And if this happens revisit your questions, and you might find another route.

Sanders calls this the model of diagnostic inquiry. And here is the part that I keep thinking about: T_his loop is small. Drawn on paper, it looks too simple to explain years of expertise. But that's exactly the point._

The most important thing is: This skill is trainable. The next investigation you run, slow down before you touch the keyboard, and say your question out loud. Is it relevant? Is it specific? Is it answerable? If yes, run the loop. If no, you are not ready to query yet.

Back to those four skills

Remember the four skills I mentioned in the beginning? Technical depth, understanding log sources, knowing how your tools work, and having been on the other side as the attacker. I said none of them were the answer. And that's true. But don't read that as "they don't matter." They matter a lot.

Here is the shift though. Every one of those skills, when you sit down to learn it, is really teaching you how to ask a better question. The deeper your technical knowledge, the more precise your questions get. Understanding your log sources tells you which questions are even answerable in your environment. Knowing how a tool works lets you ask the tool the right thing instead of fighting it. And having attacked systems yourself is what powers those capability-matching questions. It would be really hard to ask "where else would this technique show up?" about a technique you've never used.

So learn them. All of them. Go deep. Just learn them with the right intention. You're not collecting knowledge for its own sake. You're sharpening the questions you'll ask when it counts.

Conclusion

Thank you Chris Sanders for the research behind this piece. What I covered here is a small slice. The full dissertation goes much deeper, from evidence interpretation to decision-making under pressure. Go read The Analyst Mindset. It's worth it.

Sanders, C. (2021). The Analyst Mindset: A Cognitive Skills Assessment of Digital Forensic Analysts. [Doctoral dissertation, Baylor University].