How to Do Application Logging Right - Arctec Group

1 Building Security In Editors: John Steven, Gunnar Peterson, How to Do Application Logging Right A. s threats shift toward applications and as more logs are suitable for manual, semi- automated, and automated analy- companies struggle with compliance man- sis. Ideally, you can analyze them without having the Application dates, the need for useful, comprehensive that produced them and defi- nitely without having the appli- Application Logging can only increase. Here cation developer on call. From the log management viewpoint, we provide guidance on Application Logging to Application the logs can be centralized for analysis and retention. Finally, Anton developers and architects and to So, sometimes we must deal they won't slow the system down Chuvakin security professionals. with log messages like these: and can be proven reliable, if used SecurityWarrior as forensic evidence.

2 Consulting Application Aug 11 09:11:19 xx null Logging Today pif ? exit! 0 (which carries What to Log Gunnar Organizations have finally gotten absolutely no meaning), So, what types of events should Peterson network device Logging and to Message 202 User tran- you log? Arctec Group some extent server Logging un- sitioning priv level The first type is authentication, der control. However, after get- (which conveniently omits the authorization, and access events: ting used to neat Cisco Adaptive actual user identity the use Security Appliance or other fire- of secret numeric codes also successful and failed authentica- wall logs and Linux password causes trouble if no documenta- tion or authorization decisions;. accepted messages, security in- tion is available), or system access, data access, and cident investigators trying to re- userenv[error] 1040 XYZ- Application component access.

3 Spond to the next wave of attacks CORP\wsupx No descrip- and have been thrust into the horrific tion available (which scores remote access, including from one world of Application Logging . points for both honesty and use- Application component to another Problems with many applica- lessness). in a distributed environment. tion logs are truly staggering. Logs are often missing, they omit criti- In light of this, how do we The second type is changes: cal details, or they have no stan- guide Application developers and dard form or content. On top of architects toward creating secu- system or Application changes this, many security practitioners rity audit logs that are useful for (especially privilege changes), must deal with debugging logs forensics and monitoring and that data changes (including creation masquerading as security audit help satisfy compliance mandates and destruction), and logs.

4 Table 1 illustrates the key such as the Payment Card Indus- Application and component in- differences between the two. try Data Security Standard (PCI stallation and changes. Debugging logs appear in DSS)? We can start by establish- Application frameworks more ing criteria for good security au- The third type is availability issues: frequently than well-designed se- dit logs (which we just call logs . curity audit logs. However, using from now on). startups and shutdowns of sys- them for investigations is often tems, applications , and applica- an exercise in frustration because Logging Criteria tion modules or components;. they might not contain key details From a high level, the best logs faults and errors, especially er- needed for incident response and tell you exactly what happened, rors affecting the Application 's forensics. and when, where, and how.

5 Such availability; and 82 COPUBLISHED BY THE IEEE COMPUTER AND RELIABILITY SOCIETIES 1540-7993/10/$ 2010 IEEE JULY/AUGUST 2010. Building Security In Table 1. Comparing security audit logs and debugging logs. Security audit logs Debugging logs Intended consumers Security and audit personnel System operators and developers When the logger is on Always Sometimes Message content Attacks, activities, and faults Faults, failures, and errors Scope of what to log Known in advance Unknown Length of usefulness Years Hours or days backup successes and failures sixth W is actually an H.) This quire much digging for a hid- that affect availability. ancient wisdom applies perfectly den reason. Remember, the log's to logs and helps define a useful, customers are the security and The fourth type is resource issues: unambiguous log entry. audit personnel.

6 On the basis of the six Ws, the The action helps answer how . exhausted resources, exceeded following list provides a starting by providing the nature of the capacities, and so on; point for what to include: event. connectivity issues and prob- In addition, the priority helps in- lems; and The username helps answer dicate the event's importance. reached limits. who for those events relevant However, a uniform scale for to user or administrator activi- rating events by importance is The final type is badness or ties. In addition, it's helpful to impossible because different or- threats: include the name of the identity ganizations will have different provider or security realm that priorities. (For example, dif- invalid inputs and other likely vouched for the username, if ferent companies might have Application abuses and that information is available.)

7 Different policies regarding in- other security issues known to The object helps answer what formation availability versus affect the Application . by indicating the affected system confidentiality.). component or other object (such Creating a comprehensive what as a user account, data resource, So, a useful log message might to log list for every Application or file). look like this: and organization is impossible. The status also helps answer However, our list should provide a what by explaining whether 2010/12/31 10:00:01AM. useful starting point for your cus- the action aimed at the ob- GMT+7 priority=3, tom applications , especially those ject succeeded or failed. (Other system=mainserver, dealing with regulated data such types of status are possible, such module=authentication, as payment cards or sensitive per- as deferred. ) source= , sonal information.

8 The system, Application , or compo- user=anton(idp:solar), nent help answer where and action=login, What to Include must provide relevant applica- object=database, Next, what data should you log tion context, such as the initiator status=failed, for each event, and at what level of and target systems, applications , reason= password detail should you log it? The phi- or components. incorrect . losophy of relevant log details goes The source helps answer from back to ancient Greece (http:// where for messages related to This message has a field explain- ) network connectivity or distrib- ing the failure's reason. Also, it and focuses on the Six Ws : uted Application operation. isn't in XML; human readability is The time stamp and time zone useful in logs, and computers can Who was involved? help answer when. The time deal with name=value pairs just What happened?

9 Zone is essential for distribut- as well as with XML. Where did it happen? ed applications . In addition to When did it happen? the time stamp and time zone, What Not to Include Why did it happen? some high-volume systems use a Certain details should never be How did it happen? transaction ID. logged. Some examples are ob- The reason helps answer why, vious: logs should never contain (No, we can't explain why the so that log analysis doesn't re- Application or system passwords. 83. Building Security In Table 2. Defending against SQL injection attacks. Layer Injection defense What the logger will report Presentation layer Form input validation An invalid-input event that is, a failed white list or blacklist input validation event Business logic layer Business logic, rules, or A failed authentication, authorization, or access event that is, policies failed validation based on business logic, rules, or policies Data access layer Prepared statement or A resource issue event that is, an SQL syntax error event parameterized query (Sadly, this sometimes still hap- structure and behaviors from the each layer.))

10 To help correlate these pens with Web applications .) business logic, business rules, en- reports, the logger often adds a terprise policies, and Web front transaction or message exchange Centralization ends to the data structures and ID to facilitate log analysis when As we mentioned before, easy storage. This gives the Application reconstituting the events. centralization of logs is essential context, which is everything to for distributed log analysis across an incident responder. Sanitizing Audit Records either multiple systems or mul- Typically, Logging subsystems are tiple Application components of a Locating the placed to detect events around distributed Application . Although Logging Service sensitive assets, which means that syslog has been a flawed but de The Logging service's location re- they'll come into contact with facto standard of log centralization lates directly to the type of events sensitive data.