Post cover image

July 29, 2026

Sysmon Parser project using Claude-2

Improved it by adding:

By The Commoness

25 min read

Support for more Sysmon Event IDs such as network connections, file creation, registry changes, and DNS queries.
Detection rules that flag suspicious commands like PowerShell download commands or encoded scripts.
Output options for JSON, JSONL, and CSV files.
Filters for user, process name, Event ID, computer, or time range.
A summary showing total events, top processes, users, and suspicious activity.
Automated tests using different valid, missing-field, and broken XML samples.
Command-line options such as:

Create the advanced project files

Paste this into Claude:

Keep parser.py unchanged because it is the basic Sysmon XML parser.
Create these new empty files and folders for the advanced version:
1. advanced_parser.py
2. ADVANCED_README.md
3. tests/test_advanced_parser.py
4. .github/workflows/test.yml
5. advanced_events.xml
Do not add code yet and do not modify any existing files.

Create advanced sample logs

Paste this into Claude:

Add five valid Sysmon XML events inside advanced_events.xml:
- Event ID 1: Process creation
- Event ID 3: Network connection
- Event ID 11: File creation
- Event ID 13: Registry value change
- Event ID 22: DNS query
Wrap all five events inside one <Events> root.
Use realistic Windows sample values, but do not modify any Python files yet.
After creating the file, show the five Event IDs found.
None

Build support for the five Event IDs

Do not update parser.py; keep the basic version unchanged. Replace the text currently typed in Claude with:

Write advanced_parser.py to read advanced_events.xml and extract useful fields based on the Sysmon Event ID:
Event ID 1 — Process creation:
Image, CommandLine, ParentImage, User, IntegrityLevel
Event ID 3 — Network connection:
Image, User, Protocol, SourceIp, SourcePort, DestinationIp, DestinationPort, DestinationHostname
Event ID 11 — File creation:
Image, TargetFilename, CreationUtcTime, User
Event ID 13 — Registry value change:
Image, TargetObject, Details, User
Event ID 22 — DNS query:
Image, QueryName, QueryStatus, QueryResults, User
Every event must also include EventID, TimeCreated, and Computer.
Output all parsed events as formatted JSON.
Missing fields should return null.
Use only built-in Python libraries.
Do not modify parser.py or any other file.
Test it using advanced_events.xml.

Add suspicious command detection

Paste this into Claude:

Update only advanced_parser.py to add rule-based suspicious activity detection.
For Event ID 1, inspect Image and CommandLine and detect:
- PowerShell encoded commands using -enc or -encodedcommand
- PowerShell downloads using Invoke-WebRequest, wget, curl, or DownloadString
- Execution policy bypass
- Commands using certutil to download files
- Commands using bitsadmin
- Suspicious rundll32 or regsvr32 execution
Add these fields to every event:
"Suspicious": true or false
"DetectionReasons": a list of matched reasons
Normal events should return false and an empty list.
Do not modify parser.py or other files.
Test it using advanced_events.xml and show the detected results.

Create the timeline correlation engine

Paste this into Claude:

Keep parser.py and advanced_parser.py unchanged.
Create a new file named timeline_analyzer.py.
It must import and use the parsed events from advanced_parser.py, then:
1. Sort events from start time to end time.
2. Correlate related events using:
   - Computer
   - User
   - Image/process
   - ProcessGuid when available
   - Events occurring close together in time
3. Produce a timeline where every step contains:
   - StepNumber
   - Timestamp
   - EventID
   - Activity
   - Computer
   - User
   - Process
   - RiskScore
   - DetectionReason
   - Evidence
   - MITRETactic
   - MITRETechnique
4. Use deterministic rule-based scoring only:
   - Normal activity: 0–20
   - Slightly unusual: 21–40
   - Suspicious: 41–60
   - High risk: 61–80
   - Critical correlated behavior: 81–100
5. Every score must include a reason in brackets, for example:
   RiskScore: 70
   DetectionReason: "PowerShell contacted an external host shortly after execution"
   Evidence: "[Event ID 1 PowerShell process followed by Event ID 3 network connection within 30 seconds]"
6. Add:
   - TimelineStart
   - TimelineEnd
   - OverallRiskScore
   - OverallSeverity
   - CorrelationReasons
   - MITREAttackPath
7. Never guess a MITRE mapping.
   Use a fixed mapping dictionary.
   When evidence is insufficient, return:
   MITRETactic: "Unmapped"
   MITRETechnique: "Unmapped"
8. Output formatted JSON.
Use only Python built-in libraries.
Test it using advanced_events.xml.
Do not create the flowchart yet.
None

55 is reasonable based on the rules you defined.

It received:

  • 10 points: Registry modification observed
  • 35 points: Exact CurrentVersion\Run persistence location
  • 10 points: Executable located in AppData, which attackers commonly abuse

Total: 55 — Suspicious, not confirmed malicious.

The MITRE mapping T1547.001 — Registry Run Keys / Startup Folder is also correct because the log directly shows a Run-key modification.

The explanation shown is too long for the final user report. Keep the detailed ScoreBreakdown inside JSON, but display something simpler:

Risk Score: 55/100 — Suspicious
Reason:
Registry Run key persistence was created for an executable stored in AppData.
Evidence:
[Event ID 13: TargetObject contains CurrentVersion\Run]
[Details reference an executable under AppData]
Missing Evidence:
[No proof that the payload executed]
[No confirmed malicious hash or destination]

Create the visual MITRE timeline flowchart

Paste this into Claude:

Keep parser.py, advanced_parser.py, and timeline_analyzer.py unchanged.
Create a new file named flowchart_generator.py.
It must:
1. Import the final analysis from timeline_analyzer.py.
2. Generate a Mermaid flowchart named attack_timeline.md.
3. Arrange events chronologically from TimelineStart to TimelineEnd.
4. Show one node for every timeline step containing:
   - Step number
   - Timestamp
   - Event ID
   - Activity
   - Risk score and verdict
   - MITRE tactic and technique
   - Short evidence in brackets
5. Connect the nodes using arrows to show the start-to-end activity flow.
Example node:
Step 4
Registry Run Key Modified
Risk: 55 — Suspicious
MITRE: Persistence / T1547.001
[CurrentVersion\Run points to AppData executable]
6. Use different Mermaid classes based on verdict:
   - Observed
   - Unusual
   - Suspicious
   - High Risk
   - Critical
7. At the top of the report include:
   - Timeline start
   - Timeline end
   - Overall risk score
   - Overall severity
   - Correlation reasons
   - Missing evidence
8. Do not invent connections or MITRE mappings.
9. When an event is unmapped, clearly display "MITRE: Unmapped."
10. Keep node explanations short; detailed evidence must remain in the JSON timeline report.
Run it using advanced_events.xml and show the generated Mermaid flowchart text.
Use only built-in Python libraries.
Do not modify any other files.

Fix the flowchart accuracy

Paste this into Claude:

Update only flowchart_generator.py.
Make these corrections:
1. Add the event Timestamp inside every Mermaid node.
2. Do not use a normal solid arrow between events when correlation is not proven.
Use:
step1 -. "next in time" .-> step2
for chronological order only.
3. Use a solid arrow labeled "correlated" only when timeline_analyzer.py provides evidence that the two events are related through the same computer, user, process, ProcessGuid, and time window.
4. Add a small legend explaining:
- Dotted arrow = chronological order only
- Solid arrow = evidence-supported correlation
5. Do not claim that Step 1 caused Step 2 unless supporting evidence exists.
Regenerate attack_timeline.md using advanced_events.xml and show the corrected Mermaid section.
Do not modify any other files.
None

Separate process linkage from attack correlation

Paste this into Claude:

Update only flowchart_generator.py.
Correct the arrow meanings:
1. Dotted arrow:
   Label: "next in time"
   Meaning: events are only chronologically ordered.
2. Normal solid arrow:
   Label: "same process context"
   Use only when events share the same Computer, User, Process/Image, and occur within the configured time window.
   This must not imply malicious behavior.
3. Thick arrow:
   Label: "security-correlated"
   Use only when timeline_analyzer.py confirms that the events contributed to a correlation rule, appear in SupportingEventIDs, and affected the final risk score.
4. In the current advanced_events.xml result:
   - Step 1 to Step 2 should remain "next in time."
   - Steps 2 to 5 may use "same process context."
   - No arrow should say "security-correlated" because CorrelationReasons is empty.
5. Update the legend:
   - Dotted = chronological order only
   - Solid = shared process context, not proof of attack
   - Thick = evidence-supported suspicious correlation
Regenerate attack_timeline.md and show the corrected Mermaid arrows.
Do not modify any other files.

Exclude private/internal indicators

Replace the text currently typed in Claude with:

Update only indicator_agent.py.
Before building the Human Approval Gate, correct indicator privacy handling:
1. Do not include these IPs in ExternalLookupCandidates:
   - Private IPs
   - Loopback IPs
   - Link-local IPs
   - Multicast IPs
   - Reserved IPs
   - Unspecified IPs
2. Keep those IPs in a separate LocalContextIndicators section so they remain available for timeline correlation.
3. Add these fields to every indicator:
   - ExternalLookupAllowed
   - ExclusionReason
Example for 10.20.30.15:
"ExternalLookupAllowed": false
"ExclusionReason": "Private/internal IP address"
4. Only public indicators may appear in the list proposed for VirusTotal, AbuseIPDB, GreyNoise, or urlscan.
5. Do not make any API calls yet.
6. Do not modify any other files.
Test using advanced_events.xml and confirm that:
- 10.20.30.15 remains in LocalContextIndicators
- 10.20.30.15 is excluded from ExternalLookupCandidates
- 140.82.121.6 remains eligible for external lookup

Build the Human Approval Gate

Paste this into Claude:

Create a new file named human_approval_gate.py.
Keep all existing files unchanged.
The Human Approval Gate must:
1. Import ExternalLookupCandidates from indicator_agent.py.
2. Create this service plan:
   Public IP:
   - VirusTotal
   - AbuseIPDB
   - GreyNoise
   Domain:
   - VirusTotal
   - urlscan.io search
   URL:
   - VirusTotal
   - urlscan.io search
   MD5, SHA1, SHA256:
   - VirusTotal
3. Before approval, clearly display:
   - Indicator value
   - Indicator type
   - Source timeline steps
   - Risk score and verdict
   - Services that would receive it
4. Display this warning:
   "Approved indicators will be sent to third-party OSINT services.
   Do not approve confidential, internal, customer, or private information."
5. Ask:
   Proceed with OSINT enrichment? [y/N]
6. Default to No for:
   - Empty input
   - N
   - No
   - Any unrecognized response
7. When approved, create approval_record.json containing:
   - Approved
   - ApprovalTimestamp in UTC
   - ApprovedIndicators
   - ApprovedServices
   - RejectedIndicators
   - LocalContextIndicators
   - ApprovalWarning
8. When rejected:
   - Do not make any external requests
   - Record Approved as false
   - Keep ApprovedIndicators empty
   - Clearly print "OSINT enrichment cancelled"
9. Never display, request, store, or hardcode API keys.
10. Do not make any actual OSINT API calls yet.
11. Test both:
   - User enters y
   - User presses Enter without typing anything
Use only built-in Python libraries.
Do not modify any other file.

Build the OSINT agent in dry-run mode

Replace the text currently typed in Claude with:

Create a new file named osint_enrichment_agent.py.
Keep all existing files unchanged.
For this step, build the enrichment framework in DRY-RUN MODE ONLY.
Do not make any real network or API requests yet.
The agent must:
1. Read approval_record.json.
2. Stop immediately when:
   - Approved is false
   - ApprovedIndicators is empty
   - approval_record.json is missing or invalid
3. Route approved indicators to these services:
   Public IP:
   - VirusTotal
   - AbuseIPDB
   - GreyNoise
   Domain:
   - VirusTotal
   - urlscan.io historical search
   URL:
   - VirusTotal
   - urlscan.io historical search
   MD5, SHA1 and SHA256:
   - VirusTotal
4. Never perform:
   - File uploads
   - Active URL submissions
   - URL scanning
   - Private IP lookups
   - Lookups for rejected indicators
5. Read future API keys only from environment variables:
   VT_API_KEY
   ABUSEIPDB_API_KEY
   GREYNOISE_API_KEY
   URLSCAN_API_KEY
6. Never print, save, log or hardcode API-key values.
7. Create enrichment_plan.json containing one record per
   indicator and service:
   - Indicator
   - IndicatorType
   - Service
   - SourceSteps
   - LocalRiskScore
   - LocalVerdict
   - ApprovalConfirmed
   - LookupMode: "historical-report-only"
   - Status: "dry_run"
   - RequestWouldBeSent
   - APIKeyEnvironmentVariable
   - APIKeyAvailable: true or false
   - Result: null
   - Error: null
8. APIKeyAvailable must only show true or false.
   It must never reveal the key.
9. Print a summary such as:
   Approved indicators: 3
   Planned lookups: 7
   Real external requests: 0
   Mode: DRY RUN
10. Add command-line usage:
   py osint_enrichment_agent.py approval_record.json --dry-run
11. Dry-run must be the default even when --dry-run is omitted.
12. Do not add final confidence scoring yet.
Use only built-in Python libraries.
Test approved and rejected approval records.
Do not modify any other files.
None

The agent works, but LocalEvidenceConfidence = 80 is too high because all four reasons come from the same Event ID 13; they are rule details, not four independent pieces of evidence.

Correct confidence inflation

Replace the typed instruction with:

Update only confidence_agent.py.
Fix LocalEvidenceConfidence so multiple ScoreBreakdown entries from the same event are not counted as independent corroborating evidence.
Rules:
1. Group evidence by:
   - Event ID
   - Source step
   - Process or ProcessGuid
   - Evidence type
2. Several rule matches from one event count as one local evidence group.
3. Use this confidence model:
   - Valid log and exact field evidence observed: 20 points
   - One suspicious event with a deterministic rule match: +25
   - Second independent suspicious Event ID supporting the same behavior: +15
   - Confirmed same-process correlation within the time window: +10
   - Payload execution confirmed by another event: +10
   - Reliable OSINT confirmation: +20
4. Maximum LocalEvidenceConfidence when evidence comes from only one Event ID: 55.
5. Do not increase confidence merely because one event matched several scoring reasons.
6. Keep LocalRiskScore unchanged at 55.
7. For the current advanced_events.xml sample, LocalEvidenceConfidence should be no higher than 55 because:
   - The Run-key modification is directly observed.
   - Only Event ID 13 provides suspicious evidence.
   - Payload execution is not confirmed.
   - No external OSINT result exists.
   - No independent suspicious Event ID corroborates it.
8. Update the confidence breakdown to show:
   - IndependentEvidenceGroups
   - SupportingEventIDs
   - CorroborationPresent
   - ConfidenceLimitReason
9. Test using the dry-run enrichment plan.
Do not modify any other files.
None

here is one small issue: it added +10 for same-process context even though the other events were not suspicious, while also saying CorroborationPresent: false; that is inconsistent.

Test with realistic fake OSINT results

Replace the typed instruction with:

Create tests/test_confidence_agent.py using Python unittest.
Do not make any real API or network requests.
Create temporary enrichment plans that simulate realistic OSINT responses and test these cases:
1. Positive malicious evidence:
   - VirusTotal reports malicious detections for an IP
   - AbuseIPDB reports multiple abuse reports for the same IP
   Expected:
   - OSINTConfidence is calculated
   - SupportingEvidence contains both sources
   - FinalConfidenceBasis becomes "Local and OSINT evidence"
2. Clean or unknown result:
   - No malicious detections
   - Indicator reputation is unknown
   Expected:
   - Adds zero confidence points
   - Must not claim the indicator is safe
3. Conflicting results:
   - One service reports malicious
   - Another reports clean or benign
   Expected:
   - ContradictingEvidence is populated
   - The conflict is clearly explained
4. Service failure:
   - Status is error
   - Error contains rate limit or timeout
   Expected:
   - SourceErrors is populated
   - Failed lookup adds no confidence points
5. Dry-run result:
   - Result is null
   Expected:
   - OSINTConfidence remains null
   - FinalConfidenceBasis remains "Local evidence only"
6. Prevent score inflation:
   - Several fields from the same service result must count as one source-evidence group
   - Do not blindly add points for every returned field
Use temporary files and delete them after every test.
Run:
py -m unittest tests/test_confidence_agent.py -v
Do not modify any production files unless a test exposes a genuine bug. If a test fails, report the failure before changing anything.
None

The tests passed, but Test 6 reveals score inflation: one VirusTotal result reached 100 because multiple fields were counted separately

Fix OSINT confidence inflation

Paste this into Claude:

Update only confidence_agent.py and tests/test_confidence_agent.py.
Fix OSINT scoring so multiple fields from the same service result for the same indicator count as ONE evidence group.
Rules:
1. Group OSINT evidence by:
   - Indicator
   - Service
2. A single VirusTotal result may contain:
   - MaliciousCount
   - SuspiciousCount
   - KnownMalicious
   - ThreatIntelMatch
   - Reputation
   These fields may strengthen the explanation, but they must not each add separate confidence points.
3. Maximum confidence contribution per indicator/service result:
   - VirusTotal: 25 points
   - AbuseIPDB: 20 points
   - GreyNoise: 15 points
   - urlscan.io: 15 points
4. Two independent services may provide separate evidence groups.
5. Prevent one service result from producing OSINTConfidence 100 by itself.
6. Update the score breakdown to include:
   - EvidenceGroupID
   - Indicator
   - Service
   - Points
   - MatchedFields
   - Reason
7. Update Test 6 so one VirusTotal result containing all supported malicious fields:
   - Produces only one evidence group
   - Receives no more than 25 OSINT confidence points
   - Does not create multiple score entries for the same service and indicator
8. Run:
py -m unittest tests/test_confidence_agent.py -v
Do not modify any other files.

Enable only VirusTotal live lookups

VirusTotal API v3 can retrieve existing reports for hashes, URLs, domains, and public IP addresses. This step must perform report lookups only — no file uploads or active URL submissions

Enable only VirusTotal live lookups

VirusTotal API v3 can retrieve existing reports for hashes, URLs, domains, and public IP addresses. This step must perform report lookups only — no file uploads or active URL submissions.

Paste this into Claude:

Update only osint_enrichment_agent.py.
Add optional LIVE VirusTotal historical-report lookups.
Safety requirements:
1. Dry-run must remain the default.
2. Real requests are allowed only when the user runs:
   py osint_enrichment_agent.py approval_record.json --live --service virustotal
3. Before any request, verify:
   - Approved is true
   - The indicator appears in ApprovedIndicators
   - ExternalLookupAllowed is true
   - VT_API_KEY exists
   - The selected service is virustotal
4. Ask for one final confirmation:
   "Send the approved indicators to VirusTotal? [y/N]"
   Only y or yes may continue.
5. Use VirusTotal API v3 historical-report/search functionality for:
   - Public IP
   - Domain
   - URL
   - MD5, SHA1 and SHA256 hash
6. Never:
   - Upload a file
   - Submit a URL for scanning
   - Request a new analysis
   - Send private/internal IPs
   - Send rejected indicators
   - Print or store the API key
7. Use only Python built-in libraries:
   - urllib.request
   - urllib.parse
   - json
   - os
   - time
8. Add:
   - 15-second request timeout
   - Clear handling for HTTP 401, 403, 404, 429 and 5xx
   - A short delay between requests
   - Continue processing when one lookup fails
9. Normalize each result into:
   {
     "Indicator": "...",
     "IndicatorType": "...",
     "Service": "VirusTotal",
     "Status": "completed | not_found | error",
     "LookupMode": "historical-report-only",
     "MaliciousCount": 0,
     "SuspiciousCount": 0,
     "HarmlessCount": 0,
     "UndetectedCount": 0,
     "Reputation": null,
     "LastAnalysisDate": null,
     "Permalink": null,
     "Result": {},
     "Error": null
   }
10. Save results as live_enrichment_results.json.
11. Do not save the complete raw API response when it may contain unnecessary information.
    Save only the normalized evidence fields needed by confidence_agent.py.
12. Add --service virustotal so the other OSINT services remain disabled.
13. Test using mocked urllib responses only.
    Do not make a real VirusTotal request during testing.
14. Add unit tests for:
    - Approved lookup
    - Approval rejected
    - Missing API key
    - Private IP blocked
    - 404 not found
    - 429 rate limit
    - Malformed response
    - Successful normalized result
Run the tests and show the result.
Do not modify any other file.

Connect live VirusTotal results to the Confidence Agent

Replace git status with:

Update only confidence_agent.py and tests/test_confidence_agent.py.
Make confidence_agent.py accept either:
1. enrichment_plan.json for dry-run analysis
2. live_enrichment_results.json for completed VirusTotal results
Requirements:
1. Detect the input type automatically from the JSON structure.
2. For live_enrichment_results.json:
   - Status "completed" with malicious or suspicious detections may provide OSINT evidence.
   - Status "not_found" adds no confidence and must not mean safe.
   - Status "error" must appear in SourceErrors.
   - Missing or null Result adds no confidence.
3. Use only normalized fields:
   - MaliciousCount
   - SuspiciousCount
   - Reputation
   - LastAnalysisDate
   - Indicator
   - IndicatorType
   - Service
   - Status
   - Error
4. Keep the existing evidence grouping:
   - One evidence group per Indicator + Service.
   - VirusTotal contributes a maximum of 25 points per indicator.
5. Do not count the same VirusTotal result twice if it appears in both Result and top-level fields.
6. Add these report fields:
   - EnrichmentMode: "dry_run" or "live"
   - LiveResultsUsed
   - CompletedLookups
   - NotFoundLookups
   - FailedLookups
7. When valid live evidence exists:
   - FinalConfidenceBasis must become "Local evidence and OSINT evidence"
   - SourcesChecked must include VirusTotal
   - SupportingEvidence must show the exact indicator and detection counts
8. When VirusTotal finds nothing:
   - Do not reduce LocalRiskScore
   - Do not claim the indicator is safe
   - Explain that no reputation evidence was found
9. Add tests for:
   - Successful malicious VirusTotal result
   - Successful result with zero detections
   - Not-found result
   - Rate-limit error
   - Duplicate fields not inflating confidence
   - Dry-run compatibility remains unchanged
Run:
py -m unittest tests/test_confidence_agent.py -v
Do not make network requests and do not modify any other files.

Create the mixed demonstration logs

Paste this into Claude:

Create a new file named demo_mixed_events.xml.
Do not modify any existing files.
Add multiple valid Sysmon XML events inside one <Events> root.
Include these scenarios:
SCENARIO 1 — Normal activity:
- Event ID 1
- whoami.exe launched by cmd.exe
- Normal user and medium integrity
SCENARIO 2 — Rule-based detection:
- Event ID 1
- powershell.exe using an encoded command
- Include -EncodedCommand in CommandLine
- Use a separate ProcessGuid
SCENARIO 3 — Behavioral attack chain:
Use the same Computer, User and ProcessGuid for all related events:
1. Event ID 1:
   PowerShell process with ExecutionPolicy Bypass
2. Event ID 3:
   The same PowerShell process makes a network connection
3. Event ID 11:
   The same PowerShell process creates updater.exe under the user's AppData directory
4. Event ID 13:
   A Registry CurrentVersion\Run key is created pointing to updater.exe
5. Event ID 22:
   The same PowerShell process performs a DNS query
Use timestamps in chronological order, with each related event occurring within 30 seconds.
Use only fictional or reserved IP addresses and domains so no real organization is involved.
After creating the file, show:
- Total number of events
- Event IDs included
- Which event demonstrates rule-based detection
- Which events form the behavioral chain
Do not run any OSINT lookup yet.
None

Create one combined investigation report

Paste this into Claude:

Create a new file named investigation_report.py.
Do not modify any existing files.
The script must accept any Sysmon XML file:
py investigation_report.py demo_mixed_events.xml
It must reuse:
- advanced_parser.py
- timeline_analyzer.py
Generate both:
1. investigation_report.json
2. investigation_report.md
The report must contain:
## Executive Summary
- Input filename
- Total events
- Timeline start and end
- Overall risk score
- Overall severity
- Number of normal events
- Number of rule-based detections
- Number of behavioral chains
- Number of suspicious events
## Rule-Based Findings
For every event matched by a detection rule, show:
- Timestamp
- Event ID
- Computer
- User
- Process
- CommandLine
- Risk score
- Rule ID
- Detection reason
- Exact supporting evidence
- MITRE tactic and technique
- Confidence
- Missing evidence
## Behavioral Findings
Show correlated events from beginning to end:
- Chain ID
- Start time
- End time
- Duration
- Computer
- User
- ProcessGuid
- Ordered timeline steps
- Relationship between the steps
- Chain risk score
- Confidence score
- Score breakdown
- Correlation reasons
- MITRE ATT&CK path
- Missing evidence
## Event Summary
- Count by Event ID
- Top processes
- Top users
- Top computers
- Suspicious versus observed event count
## Important Accuracy Rules
- Do not call chronological events correlated unless supporting evidence exists.
- Do not treat PowerShell alone as malicious.
- Every score must show its rule and evidence.
- Use "possible suspicious behavior" when malicious activity is not confirmed.
- Do not perform OSINT lookups.
- Do not invent MITRE mappings.
- Use null or "Unmapped" when evidence is insufficient.
Test it using demo_mixed_events.xml.
After testing, show:
- Executive Summary
- The rule-based finding
- The behavioral chain summary
- The generated report filenames
Use only Python built-in libraries.
None

The report is working and successfully compares normal activity, rule-based detection, and a behavioral chain, but it is not fully accurate yet:

  • ProcessGuid incorrectly shows null, even though the demo logs contain it.
  • Confidence 100 is too high because there is no OSINT confirmation and no proof that updater.exe executed. Keep the risk at 85, but local evidence confidence should be about 75.
  • Encoded PowerShell should map to T1059.001 PowerShell and T1027.010 Command Obfuscation; the Run-key mapping T1547.001 is correct.

Fix the report accuracy

Paste this into Claude:

Update only:
- advanced_parser.py
- timeline_analyzer.py
- investigation_report.py
Add or update tests if required.
Fix these report accuracy issues:
1. Extract these identifiers whenever present:
   - ProcessGuid
   - ParentProcessGuid
   - ProcessId
   - ParentProcessId
2. For Event IDs 1, 3, 11, 13 and 22, preserve ProcessGuid in the parsed output.
3. Behavioral-chain correlation must prefer:
   - Same ProcessGuid
   - Same Computer
   - Same User
   - Events inside the configured time window
4. Only fall back to Computer + User + Image when ProcessGuid is missing.
5. Add a field:
   "CorrelationMethod": "ProcessGuid" or "Fallback process context"
6. Fix the current demo report so ProcessGuid is not null for the behavioral chain.
7. Keep ChainRiskScore at 85 if the existing deterministic risk rules support it.
8. Recalculate local behavioral confidence using:
   - Exact valid log evidence: 20
   - Independent suspicious Event ID 1: +15
   - Independent suspicious Event ID 11: +15
   - Independent suspicious Event ID 13: +15
   - Confirmed same ProcessGuid and time-window correlation: +10
   Expected confidence: 75
9. Do not output confidence 100 without external confirmation or stronger independent evidence.
10. MITRE mappings:
   Encoded PowerShell:
   - Execution / T1059.001 PowerShell
   - Defense Evasion / T1027.010 Command Obfuscation
   Registry CurrentVersion\Run:
   - Persistence / T1547.001 Registry Run Keys / Startup Folder
11. Do not map ordinary network or DNS activity unless an exact deterministic rule supports it.
12. Update the behavioral assessment to:
   "Possible multi-stage suspicious behavior supported by correlated Sysmon events. The activity is not confirmed malicious because external threat-intelligence confirmation and execution of the dropped updater.exe are unavailable."
13. Regenerate:
   - investigation_report.json
   - investigation_report.md
14. Test using demo_mixed_events.xml and show:
   - ProcessGuid
   - CorrelationMethod
   - ChainRiskScore
   - ConfidenceScore
   - MITRE ATT&CK path
   - Final assessment
Do not perform OSINT requests.
Do not modify other files.

Verify filters and export formats

Replace the typed instruction with:

Audit advanced_parser.py without changing any files.
Using demo_mixed_events.xml, verify whether these command-line features already work:
1. Filter by Event ID:
   py advanced_parser.py demo_mixed_events.xml --event-id 1
2. Filter by user:
   py advanced_parser.py demo_mixed_events.xml --user asmith
3. Filter by process:
   py advanced_parser.py demo_mixed_events.xml --process powershell.exe
4. Filter by computer:
   py advanced_parser.py demo_mixed_events.xml --computer WORKSTATION02
5. Combined filters:
   py advanced_parser.py demo_mixed_events.xml --event-id 1 --process powershell.exe
6. Time-range filtering:
   Use --start-time and --end-time to return only the behavioral-chain events.
7. JSON file output:
   --format json --output test_results.json
8. JSONL file output:
   --format jsonl --output test_results.jsonl
9. CSV file output:
   --format csv --output test_results.csv
For every test, report:
- PASS or FAIL
- Number of events returned
- Output file created, when applicable
- Any error encountered
Use temporary output files and remove them after testing.
Do not modify advanced_parser.py or any other file yet.
If a feature fails or is missing, clearly identify it and stop before fixing it.
None

7 of 9 features passed. Only the shortened --user and --computer filters failed. Fix those two filters

Paste this into Claude:

Update only advanced_parser.py.
Fix the --user and --computer filters so they remain case-insensitive and support both full and shortened values.
User examples:
- --user asmith must match CORP\asmith
- --user CORP\asmith must also match exactly
Computer examples:
- --computer WORKSTATION02 must match WORKSTATION02.corp.local
- --computer WORKSTATION02.corp.local must also match exactly
Requirements:
1. For User:
   - Match the complete value
   - Also match the username after the final backslash
2. For Computer:
   - Match the complete hostname
   - Also match the short hostname before the first dot
3. Do not use loose substring matching.
   Example:
   - user "smith" must not accidentally match "asmith"
   - computer "WORK" must not match "WORKSTATION02"
4. Preserve all existing filters, exports and parser behavior.
5. Add automated tests for:
   - Full user
   - Short user
   - Wrong partial user
   - Full computer name
   - Short computer name
   - Wrong partial computer name
   - Combined filtering
6. Re-run all nine filter/export checks using demo_mixed_events.xml.
Expected result:
9 of 9 tests pass.
Do not modify any other production file.
None

all 59 automated tests pass, and all 9 filter/export checks pass.

Do not commit yet. Two requested items remain:

  • Automated parser tests for valid, missing-field, and broken XML
  • GitHub Actions to run tests automatically after every push

Add parser input/error tests

Replace commit this with:

Create a new file named tests/test_parser_inputs.py.
Do not modify any production files unless a test reveals a real bug.
Using Python unittest and temporary XML files, test:
1. Valid single Sysmon Event:
   - parser.py returns one JSON object
   - Important fields are extracted correctly
2. Valid multiple Sysmon Events:
   - parser.py returns a JSON list
   - Correct number of events is returned
3. Missing fields:
   - Remove Image, CommandLine, User and ParentImage
   - Parser must not crash
   - Missing values must return null
4. Broken XML:
   - Use mismatched or unclosed tags
   - Parser must show a clear invalid XML error
   - It must exit with a non-zero code
5. Missing input file:
   - Parser must show a clear file-not-found error
   - It must exit with a non-zero code
6. Advanced parser compatibility:
   - Confirm advanced_parser.py can still parse valid Event IDs 1, 3, 11, 13 and 22
Use tempfile so test files are removed automatically.
Run the complete test suite:
py -m unittest discover -s tests -p "test_*.py" -v
Show:
- Total tests run
- PASS or FAIL
- Confirmation that no real network requests occurred
- Confirmation that no production files were modified
None

let's preview the human-readable investigation report.

Do not modify any files.

Open investigation_report.md and display the complete report exactly as a user would read it.

After the report, briefly evaluate:

1. Is the Executive Summary easy for a non-technical person to understand?
2. Are rule-based and behavioral findings clearly separated?
3. Does every risk and confidence score include a reason?
4. Is the MITRE ATT&CK mapping understandable?
5. Are uncertain findings clearly described as unconfirmed?
6. Are any sections too technical, repetitive, or confusing?

Do not regenerate or edit the report yet.
None None None

simplify the report

Paste this into Claude:

Update only investigation_report.py.
Make the Markdown report easier to understand:
1. Remove the entire Evaluation section. It was only a review of the report and must not appear in the final investigation report.
2. Replace the current repetitive Score Breakdown with one de-duplicated Final Risk Breakdown per behavioral chain.
For the current demo chain, display:
- +20: PowerShell ExecutionPolicy Bypass
- +15: Executable created in AppData
- +35: Registry Run key persistence
- +15: Three suspicious Event IDs correlated through the same ProcessGuid and time window
- Total: 85/100
3. Add a separate Confidence Breakdown:
- +20: Valid Sysmon log evidence
- +15: Independent suspicious Event ID 1
- +15: Independent suspicious Event ID 11
- +15: Independent suspicious Event ID 13
- +10: Same ProcessGuid and time-window correlation
- Total: 75/100
4. Do not show the same correlation reason multiple times.
5. Clearly state that ordinary DNS and network events provided timeline context but did not add suspicious points.
6. Add a small explanation:
Risk Score = how dangerous the observed behavior may be.
Confidence Score = how strongly the available evidence supports the finding.
7. Keep detailed per-event evidence in investigation_report.json, but keep investigation_report.md short and readable.
8. Regenerate investigation_report.md using demo_mixed_events.xml and show only:
- Executive Summary
- Simplified Risk Breakdown
- Simplified Confidence Breakdown
- Final Assessment
Do not modify any other files.
None

The simplification is complete, but Claude kept the detailed findings in the report, which is useful.

None

Build the layered investigation report

Paste this into Claude:

Update investigation_report.py and its related report tests.
Do not change the detection or scoring values unless required to correct an inaccurate MITRE mapping.
Create an industry-style layered Markdown report that is readable at the top but keeps all technical evidence inside expandable GitHub Markdown sections using:
<details>
<summary>Section title</summary>
Content
</details>
The final investigation_report.md must have this structure:
# Sysmon Security Investigation Report
## 1. Executive Summary
Show:
- Input filename
- Total events
- Timeline start and end
- Overall risk score and severity
- Evidence confidence score
- Number of observed events
- Number of suspicious events
- Number of rule-based detections
- Number of behavioral chains
- One plain-English sentence explaining what happened
- Final assessment stating whether the activity is confirmed or unconfirmed
Add:
Risk Score = how dangerous the observed behavior may be.
Confidence Score = how strongly the available evidence supports the finding.
## 2. OSINT Enrichment Status
Always show one clear status:
- NOT RUN
- DRY RUN ONLY
- CANCELLED BY ANALYST
- LIVE LOOKUP COMPLETED
- LIVE LOOKUP PARTIALLY COMPLETED
- LIVE LOOKUP FAILED
Include:
- Human approval status
- Services planned
- Services actually checked
- Indicators checked
- Completed lookups
- Not-found lookups
- Failed lookups
- OSINT confidence
- Lookup timestamp
- Source errors
Accuracy requirements:
- When no OSINT lookup occurred, display:
  "OSINT confidence: Not evaluated"
- Do not say an indicator is clean or safe when OSINT was not run.
- A not-found result must mean:
  "No reputation record was found; this does not prove the indicator is safe."
- Never display an API key.
## 3. Attack Timeline
Include the existing Mermaid start-to-end flowchart.
Explain the arrows:
- Dotted arrow = next event in time only
- Solid arrow = same process context
- Thick arrow = evidence-supported suspicious correlation
## 4. Final Risk Breakdown
Keep the short de-duplicated risk calculation visible.
For the current demo:
- +20: PowerShell ExecutionPolicy Bypass
- +15: Executable created in AppData
- +35: Registry Run-key persistence
- +15: Three suspicious Event IDs correlated using the same ProcessGuid and time window
- Total: 85/100
State that ordinary network and DNS events supplied timeline context but added no suspicious points.
## 5. Confidence Breakdown
Keep the short confidence calculation visible:
- +20: Valid Sysmon evidence
- +15: Independent suspicious Event ID 1
- +15: Independent suspicious Event ID 11
- +15: Independent suspicious Event ID 13
- +10: Same ProcessGuid and time-window correlation
- Total: 75/100
## 6. Detailed Evidence
Create these collapsed sections:
<details>
<summary>Parsed Sysmon Events</summary>
For every event show:
- Step
- Timestamp
- Event ID and event name
- Computer
- User
- ProcessGuid
- ProcessId
- Image
- CommandLine
- ParentImage
- Event-specific extracted fields
- Suspicious true/false
- Rule IDs
- Exact evidence
This section must demonstrate how the XML was parsed into structured fields.
</details>
<details>
<summary>Rule-Based Detection Details</summary>
Show each independent rule match with:
- Rule ID
- Rule name
- Exact matched field
- Exact matched text
- Risk contribution
- Confidence
- Evidence
- Missing evidence
- Assessment
Do not count one event as multiple independent confirmations.
</details>
<details>
<summary>Behavioral Correlation Details</summary>
Show:
- Chain ID
- Start and end
- Duration
- ProcessGuid
- Correlation method
- Ordered steps
- Relationships between steps
- Supporting Event IDs
- Correlation reasons
- Risk score
- Confidence score
- Missing evidence
Clearly separate:
- Timeline context
- Shared process context
- Security-correlated evidence
</details>
<details>
<summary>MITRE ATT&CK Mapping</summary>
Show a table:
| Event or rule | Tactic | Technique | Why it maps | Evidence |
Correct these mappings:
1. Encoded PowerShell:
   - Execution / T1059.001 PowerShell
   - Defense Evasion / T1027.010 Command Obfuscation
2. Registry CurrentVersion\Run modification:
   - Persistence / T1547.001 Registry Run Keys / Startup Folder
3. ExecutionPolicy Bypass:
   - Do not map it to Registry Run Keys.
   - Map only when a fixed deterministic mapping exists.
4. Event ID 11 file creation:
   - Do not map it to T1547.001 by itself.
   - Use Unmapped unless another exact deterministic mapping is supported.
5. Ordinary network and DNS activity:
   - Keep Unmapped unless an exact detection rule supports a technique.
Do not invent mappings.
</details>
<details>
<summary>OSINT Evidence Details</summary>
Show:
- Approval decision
- Approved indicators
- Rejected indicators
- Private/internal indicators excluded
- Services planned
- Services checked
- Per-service lookup status
- Detection counts
- Reputation
- Errors
- Lookup timestamps
When OSINT was not run, clearly display:
"No external reputation evidence was used in this report."
</details>
<details>
<summary>Event Statistics</summary>
Show:
- Count by Event ID
- Top processes
- Top users
- Top computers
- Suspicious versus observed counts
</details>
<details>
<summary>Limitations and Missing Evidence</summary>
Include:
- XML batch analysis only
- Not continuous real-time monitoring
- No proof that updater.exe executed
- No transferred-byte visibility from Sysmon Event ID 3
- No lateral-movement confirmation without authentication, EDR, firewall or Windows Security logs
- OSINT is supporting evidence, not proof
- Missing or unavailable log sources
</details>
## 7. Analyst Conclusion
Keep this section visible and short.
Use careful language:
"Possible multi-stage suspicious behavior supported by correlated Sysmon events. The activity is not confirmed malicious because external threat-intelligence confirmation and execution of the dropped updater.exe are unavailable."
Additional requirements:
1. Do not include the old Evaluation section.
2. Do not duplicate scoring reasons.
3. Keep detailed evidence in investigation_report.json.
4. The Markdown report must be readable on GitHub.
5. Regenerate the report using demo_mixed_events.xml.
6. Add tests confirming:
   - All required sections exist
   - All details tags are correctly closed
   - OSINT status is always displayed
   - Incorrect T1547.001 mappings do not appear
   - No API keys appear
   - Scores remain 85 risk and 75 confidence
7. Run the complete test suite.
After completion, show only:
- Executive Summary
- OSINT Enrichment Status
- Attack Timeline section
- Names of all collapsed sections
- Analyst Conclusion
- Total number of tests passed

correct the final MITRE mapping

Paste this into Claude:

Audit and correct only the MITRE mapping logic and related tests.
Do not change risk scores, confidence scores, detections, timelines, or report structure.
Requirements:
1. PowerShell with ExecutionPolicy Bypass:
   - Map to Execution / T1059.001 PowerShell.
   - Do not automatically map to Disable or Modify Tools.
   - Do not add a Defense Evasion mapping unless another exact observed behavior supports it.
2. PowerShell with -EncodedCommand:
   - Execution / T1059.001 PowerShell
   - Defense Evasion / T1027.010 Command Obfuscation
3. Registry CurrentVersion\Run modification:
   - Persistence / T1547.001 Registry Run Keys / Startup Folder
4. File creation in AppData by itself:
   - Unmapped
   - It may contribute to risk scoring, but must not inherit the Registry Run-key MITRE mapping.
5. Ordinary DNS and network events:
   - Unmapped unless a specific deterministic rule supports a technique.
6. Update investigation_report.md and investigation_report.json using demo_mixed_events.xml.
7. Add tests confirming:
   - ExecutionPolicy Bypass shows T1059.001
   - It does not show T1562.001 or Disable or Modify Tools
   - EncodedCommand shows both T1059.001 and T1027.010
   - Only the Registry Run-key event shows T1547.001
   - File creation, normal network and DNS events remain Unmapped
8. Run the complete test suite and show the total tests passed.
Do not modify any unrelated files.

Add automatic GitHub testing

Replace the typed command with:

Create or update .github/workflows/test.yml.
Requirements:
1. Workflow name:
   Sysmon Security Pipeline Tests
2. Run on:
   - Push to main
   - Pull request to main
   - Manual workflow dispatch
3. Use:
   - ubuntu-latest
   - actions/checkout@v4
   - actions/setup-python@v5
   - Python 3.12
4. Set:
   OSINT_TEST_MODE: "true"
5. Run:
   python -m unittest discover -s tests -p "test_*.py" -v
6. Add a 10-minute timeout.
7. Do not configure API keys or secrets.
8. Do not perform live OSINT requests.
9. Do not modify Python files or tests.
After creating it:
- Show the complete YAML
- Validate its structure
- Confirm all 84 tests are included
- Do not commit yet

Final safety check before commit

Paste this into Claude: to check no real API keys or private runtime data are present.

Run git status and show me:
1. Every file that will be committed
2. Every generated file ignored by .gitignore
3. Confirm no API keys, .env files, approval records, live enrichment results, temporary files, or private data will be committed
Also scan the staged and untracked project files for strings such as:
API_KEY=
VT_API_KEY=
ABUSEIPDB_API_KEY=
GREYNOISE_API_KEY=
URLSCAN_API_KEY=
Do not add or commit anything yet.

protect future generated files

Paste this into Claude:

Update only .gitignore.
Add these runtime-generated files:
approval_record.json
enrichment_plan.json
live_enrichment_results.json
indicators.json
confidence_report.json
test_results.json
test_results.jsonl
test_results.csv
Keep these sample reports commit-ready because they demonstrate the project:
attack_timeline.md
investigation_report.md
investigation_report.json
Then run git status and confirm the new runtime-generated files would be ignored.
Do not add or commit anything yet.

The documentation placeholder your-own-key-here and fake test keys are safe to keep because they are not real credentials.

Stage the project files

Paste this into Claude:

Run:
git add .
Then run:
git status
Show me all staged files. Do not commit yet.

. Where Is It Used?

  • SOC (Security Operations Center) — triaging exported Sysmon batches to quickly separate ordinary activity from events that warrant escalation.
  • Incident response — building an ordered, scored timeline of what happened on a host during a suspected compromise.
  • Threat hunting — filtering large Sysmon exports by process, user, computer, or time window to test a hypothesis (e.g. "did any host run an encoded PowerShell command last week?").
  • SIEM log preparation — normalizing raw Sysmon XML into flat JSON/JSONL/CSV before it is ingested or forwarded into a SIEM.

4. Why Is It Useful?

  • No third-party dependencies. Runs anywhere Python 3 runs, with nothing to install and no supply-chain surface beyond the standard library.
  • Deterministic and evidence-based. Every risk point and every MITRE mapping is traceable to a specific rule and a specific piece of log evidence
  • Human-in-the-loop by design. No external lookup (e.g. VirusTotal) ever happens without an explicit, logged human approval step.
  • Auditable output. JSON output preserves full detail (scores, evidence, correlation reasoning) for machine/audit use, while the Markdown report stays readable for a human analyst.

Also you can connect me on Linkedin :https://www.linkedin.com/in/roshini-m-lakshmana-b51ab3243/