We live in a complex world with seemingly continual headlines of breaches, hacks, and other nefarious online activity. Security programs must be robust enough to address the continual threats bombarding organizations today.
Security practitioners have a lot on their plate — identification and authentication, access control, encryption of data in transit and at rest, data integrity, system and data availability, vendor management, incident response, personnel security, vulnerability management, malware defenses, application security, etc. — so having the ability to gather, analyze, and report information that supports these various areas is crucial.
While logging and monitoring is also a crucial part of an organizational security program like the areas previously mentioned, it has a special role in that logging and monitoring support and provide insight into all the other key areas of a security program. This blog will provide a brief overview of logging and monitoring, some of the associated challenges, as well as best practices.
What is Logging and Why Is It Important?
Simply put, a system log is a collection of individual records that represent specific activity, events, error conditions, faults, or general status on an information system or network. These log entries contain critical data that helps system and security administrators understand what is occurring on an information system.
As many logs contain security-relevant information (e.g. authentication events, changes in access, changes in system configuration, etc.), logs help administrators be aware of potential (or actual) malicious activity on the system or changes that occur that may weaken the security posture of the system.
If logs are not generated by the system, administrators, security personnel, and development teams will essentially be blind to the activities on the system. They will be unaware of potentially malicious activity on the system, and will certainly not be able to respond to it. They will not know how a compromise occurred and where the attacker has pivoted to since the initial breach. Attempted and successful attacks will go unnoticed indefinitely while data is exfiltrated and their malicious presence persists.
Maintaining audit logs is also important to support any required Federal legislation or regulatory requirements such as those outlined by the Federal Information Security Management Act (FISMA) or the Health Insurance Portability and Accountability Act (HIPAA).
What Should Be Logged? Logging & Monitoring Policy
The answer to this question is…it depends. There are a number of factors (e.g. risk appetite, log volume, security relevance, etc.) that contribute to deciding what should be included in an organization’s security logging configurations. There are also other logs that are used for performance, availability, error conditions, etc.
Just because a capability to generate log events is available doesn’t mean it should be turned on as it will just add to the potential log overload that is common in many environments. Essentially, an organization’s security logging and monitoring policy should drive what is logged, how logs are transmitted, log rotation, retention, storage, etc.
One of the primary reasons for enabling security logging is to support forensic investigations around potential or realized breaches. Therefore, it is important to log events that will support breach investigations such as the following:
- Successful and unsuccessful login events
- Account management activities (e.g. account creation, modification, deletion)
- Use of privileged commands on the operating system and for applications
- Changes in authorization
- Data access, modifications, and deletion of critical data sets
For each of the audit events above, the audit record should also define when the event occurred (via timestamp), where the event occurred (e.g. IP address of host), the source of the event, the results of the event (e.g. successful or failed), and any identity information (e.g. account name) for the individuals/processes acting on behalf of individuals that performed the action.
What is Monitoring and Why Is It Important?
Security logs provide little to no value if they are not monitored. In fact, attackers hedge their bet that their target does not monitor their logs.
Log monitoring is essentially reviewing the recorded log entries for anomalous, abnormal, or suspicious events. While log monitoring can be performed manually, it is not efficient and should be reserved for more detailed analysis spurred by automation.
Systems today generate incredible volumes of logs, so automation is essentially required in order to perform any reliable level of log monitoring and analysis. The primary tool used today for security log monitoring is a security information and event management (SIEM) platform.
There are numerous SIEMs on the market today which provide a host of different capabilities, but the primary premise of a SIEM is to collect or ingest logs from multiple sources, perform or enable efficient analysis, and perform a designated action such as alerting on events of interest.
The importance of monitoring security events via logs cannot be understated. Timeliness is key; without active log monitoring, the likelihood that an attacker maintains an undetected persistent presence increases significantly. While the prevention of breaches is highly preferred, detection of a breach is a must, and the primary detection mechanism for breaches is the identification of anomalous activity in security logs.
What Are the Challenges to Logging and Monitoring?
The primary challenges regarding security logging and monitoring are the sheer volume of logs that are generated by information systems and applications and the lack of trained security staff to identify abnormal events using a SIEM or other automated techniques.
Additional challenges include differing log formats based on the OS or application generating the log, differing log content which makes it difficult to follow a thread across multiple platforms, and non-standardized time stamps. Fortunately, today’s SIEM platforms are able to normalize log entries into a common, parsable format while also retaining the original log entry if required to support more in-depth analysis.
Network/Security Logging and Monitoring Best Practices — How to Get the Most From Your Efforts
The following are a few recommendations to make the most out of your organization’s security and network logging and monitoring efforts:
- Enable logging on all your operating systems, network devices, and applications. Every component in the system architecture should be configured to generate audit events so as to ensure complete coverage and not leave any blind spots which can be used for initial exploitation or pivot points.
- Tune what is logged on operating systems, network devices, and applications. Familiarize yourself with the auditing capabilities of each component within the architecture and make an overt decision on what events should be audited keeping in mind organizational logging and monitoring policies. Configure critical devices like firewalls and remote access points for verbose logging while tailoring the audit capabilities of other components to security-relevant events or other events of interest.
- Establish a baseline of “normal” activity. If the idea is to identify abnormal or malicious activity and alert appropriately, organizations must know what constitutes “normal” behavior, or non-malicious, standard activities that support business objectives.
- Tune your SIEM. Once you have a baseline of activities that represent “normal” behavior, it will be easier to tune your SIEM to identify activities that fall outside of the “normal” behavior patterns. These are the events that demand the most focus and attention from security staff. In addition, a tuned SIEM will produce fewer false alarms that end up taking a lot of time to investigate.
- Train security staff in event detection. Event analysis is a specialized skill and requires honed expertise to identify and understand attack patterns.
Summary
Today’s information systems are complex, and there are many avenues for attackers to exploit them. Insufficient logging and monitoring enable attackers to initially exploit and pivot within systems without detection. Therefore, it is paramount that organizations implement a security logging and monitoring program driven by their security logging and monitoring policy.
By leveraging technology for audit event reduction, correlation, analysis, and reporting, organizations are able to better understand the activities that are occurring on their systems, and through proper tuning of events, they will be able to better identify and respond to abnormal or potentially malicious activity.
Linford & Company has extensive experience in assisting organizations of all sizes to manage their compliance requirements. Please contact us if you are in need of compliance or advisory services for SOC 1, SOC 2, FedRAMP, HITRUST, and HIPAA audits and assessments.
Related blog posts:
Ray Dunham started his career as an Air Force Officer in 1996 in the field of Communications and Computer Systems. Following his time in the Air Force, Ray worked in the defense industry in areas of system architecture, system engineering, and primarily information security. Ray leads L&C’s FedRAMP practice but also supports SOC examinations. Ray enjoys working with clients to secure their environments and provide guidance on information security principles and practices.