Key Terms
Log
A time-stamped record of an event on a system, application, or network device — the raw evidence from which security investigations are built.
Event ID
A unique identifier assigned to a specific event type in Windows or other OS — e.g., Windows Event ID 4625 indicates a failed logon attempt.
Log Aggregation
The process of collecting logs from multiple sources into a centralised system — essential for cross-system correlation and analysis at enterprise scale.
Baseline
The documented normal behaviour of a system or user — the reference point against which anomalies that may indicate an attack are identified.

What are logs and why do they matter?

A log is a time-stamped record of something that happened. Every login attempt, every file access, every network connection, every process execution — each one leaves a log entry somewhere in the infrastructure. Collectively, these records form a forensic trail that security analysts can follow to detect attacks, reconstruct timelines, and prove what happened during an incident.

Without logs, a security team is essentially blind. When an attacker moves laterally through a network, exfiltrates data, or deploys ransomware, logs are the evidence that records what they did, when they did it, and how far they got. Logging is so fundamental that most regulatory frameworks — GDPR, PCI DSS, ISO 27001, HIPAA — mandate that specific log types be retained for defined periods.

The challenge isn't generating logs — enterprise environments produce enormous volumes. The challenge is collecting them centrally, normalising the format, and finding the malicious signals in the noise. That is the role of the SIEM and the SOC analyst.

Types of logs security teams analyse

Security analysts work with logs from multiple source types, each revealing different aspects of activity:

  • System Logs: Windows Event Logs and Linux syslog capture OS-level activity — process creation, service starts/stops, privilege changes, account management. Windows Event IDs are particularly important: 4624 (successful logon), 4625 (failed logon), 4720 (account created), 4732 (added to privileged group).
  • Authentication Logs: Active Directory domain controller logs, VPN authentication logs, SSH auth logs, MFA logs — track all identity-based access events. Multiple failed authentications followed by a success is a classic brute-force indicator.
  • Network Logs: Firewall logs (allowed/blocked connections), IDS/IPS alerts, NetFlow/IPFIX (traffic metadata without payload), DNS query logs (attackers use DNS for C2 — DNS logs reveal unusual query patterns), proxy logs (web request history).
  • Application Logs: Web server access logs (Apache/nginx — reveal SQL injection attempts and path traversal), database audit logs (unusual query volumes or off-hours access), API gateway logs (authentication failures, rate limit violations).
  • Security Tool Logs: EDR telemetry (process trees, file operations, network connections from endpoints), CASB logs (cloud application access), DLP alerts (sensitive data movement).

The log analysis process

Raw logs from hundreds of sources are only useful once they are centralised, normalised, and correlated. The process follows a consistent pipeline:

  • Collection: Log agents (Winlogbeat, Filebeat, Syslog forwarders) ship logs from sources to a centralised aggregation platform.
  • Normalisation: Different log formats are parsed into a consistent schema — timestamps standardised to UTC, field names unified, IP addresses extracted as structured data.
  • Correlation: Rules and algorithms match patterns across multiple log sources — a failed authentication on a domain controller followed by a successful VPN login from the same account within 30 seconds may indicate credential stuffing.
  • Alerting: Correlated events that match threat detection rules trigger alerts for analyst investigation — the SIEM's primary output.
  • Investigation: Analysts examine alert context, pivot across related log sources, and build a timeline of events to determine whether the alert is a true positive or false positive.
Example: Spotting a Brute Force Attack in Logs

Windows Event ID 4625 — Failed Logon. Time: 03:14:22 UTC. Source IP: 185.220.101.42 (known Tor exit node). Target account: Administrator. Failure reason: Wrong password. When you see 847 of these in 60 seconds from the same IP, that's a brute-force attack in progress. One SIEM rule — "more than 5 failed logons to the same account within 60 seconds" — catches this automatically and generates an alert before the attacker succeeds.

Log analysis and SIEM

A SIEM (Security Information and Event Management) platform is the technology that makes enterprise-scale log analysis feasible. Without a SIEM, analysts would need to manually query dozens of separate log sources — an impossibly slow process during an active incident.

Modern SIEMs like Splunk, Microsoft Sentinel, and IBM QRadar provide: centralised storage and search across all log sources, pre-built detection rules for common attack patterns, dashboards showing the current security posture, timeline views that reconstruct attack sequences, and integration with threat intelligence feeds that automatically flag known malicious IPs and domains.

SOC analysts spend the majority of their working time in the SIEM — querying logs, investigating alerts, and building detection logic. Proficiency with Splunk SPL (Search Processing Language) or Microsoft KQL (Kusto Query Language) is one of the most in-demand practical skills in defensive security.

What SOC analysts hunt for in logs

Experienced analysts know that specific log patterns reliably indicate attack activity:

  • Brute force: High volume of Event ID 4625 (failed logons) against the same account, especially from external IPs at unusual hours.
  • Lateral movement: A service account that normally only accesses one server suddenly authenticating to 20 servers within a few minutes — a classic pass-the-hash or pass-the-ticket pattern.
  • Data exfiltration: Unusually large outbound data transfers to cloud storage services or external IPs — DNS logs showing large numbers of TXT record queries to suspicious domains may indicate DNS exfiltration.
  • Privilege escalation: Sudden group membership changes (Event ID 4732) adding a standard user account to Domain Admins.
  • Impossible travel: A user authenticates from London at 09:00 and from Singapore at 09:15 — impossible without compromised credentials. Active Directory and VPN logs reveal this instantly.
  • Living-off-the-land: Legitimate Windows tools (PowerShell, WMI, PsExec) used in patterns that indicate attacker activity — high-entropy command-line arguments, encoded payloads, spawning unexpected child processes.
80%
Of breaches would have been detectable in existing logs before significant damage occurred (Verizon DBIR)
500TB+
Daily log data generated by an average large enterprise — SIEM makes analysis feasible
10x
Faster incident investigation with centralised log aggregation vs siloed per-system analysis

Log analysis in a SOC career

Log analysis is the daily work of every SOC analyst, at every tier. L1 analysts investigate SIEM alerts by examining relevant log sources. L2 analysts correlate events across multiple systems to scope incidents and build attack timelines. L3 analysts write detection rules and hunt for attacker behaviour not yet caught by automated rules.

Key technical skills to develop: Splunk (SPL query language, free training via Splunk Education), Microsoft Sentinel and KQL, Linux command-line log parsing (grep, awk, cut), regular expressions, and Python for custom log processing scripts. CompTIA Security+ SY0-701 covers log analysis concepts and is the recommended baseline certification for SOC analyst roles.

Train with VAPTIC
CompTIA Security+ SY0-701
21 modules · Live classes · Covers SIEM, log analysis & SOC operations · CompTIA certified
Enrol Now