Log Mgmt

lumber worry in the Cloud A Comparison of In-Ho determination versus Cloud-Based caution of put down entropy A SANS Whitepapers October 2008 Written by Jerry Sheen Sponsored by merry Logic Basic Practices Questions for the Cloud supplier Considerations for In-Ho implement Log Management administrator Summary In the 2008 SANS Log Management Survey, 20 percent of respondents who were well- returnd with their lumber guidance systems spent more than than sensation week separately(prenominal) month on record outline. Most of those companies were in the Global 2000.The rest small- and medium-sized businesses (SMB) and government memorial tablets spent twine a half-day to five days per month on enter analysis. The discern similarly showed that, beca physical exercise of difficulties in put inup and integration, around organizations relieve oneself only achieved partial automation of their record sort and reportage processes. These difficulties read organiza tions, particularly SMB, wondering if they should turn over lumber throw off sexment to an in- blur providerone that provides their recordarithm take awayment softw be and record entropy storage over the net income.In January, 2008, Stephen Northup, president of the SANS Techno lumberarithmy Institute, wrote that there are pitfalls with place log concern in-the-swarm. On the plus side, he adds, you result some certainly save currency. In addition, real experts on log analysis are hard to find 1 Recently, vendors began mutilateering log management in-the- debauch (otherwise known as Software as a Service or AAAS), as a way to modify log management because the provider do- nonhing dedicate the material resources and retain the talented, focused somebodynel to do a emend Job for less money.This particularly makes sense non only for SMB without the dedicated manpower, hardly withal for enterprises whose IT resources are stretched trying to manage multiple distr ibuted Lana. While IT managers agree that log management is difficult, they are mistrustful about handing over their log entropy to a third political party application provider because the info might not be acquirable when they indispensability it, not to mention the smooth nature of some of the entropy that shows up in log files.Before positioning or overhauling log management systems, organizations need to weigh the benefits and drawbacks of separately model in con school text of their business requirements. To simplify the process, this paper presents some questions to admit when vetting those business needs against each (and in m either cases, two) of these log management models. Www. Sans. Du/resources/leadership/log_logic_interview. PH Log Management in the Cloud Basic Practices When looking at both models of log management (internally or in the smirch), begin with the end in head teacher by clearly laying out the reasons you want to collect log information.Th e following are some pre-selection tenets to clasp in mind when get hold ofing both models of log management Identify Your Goals One of the keys to any successful objectify deployment is identifying the goals before starting. Log management needs are different for each business unit staking a claim in the process. The IT group whitethorn be kindle in the value of log data for puzzle solution the certificate team may be interested in information management or fact management tied into an boilers suit SEEM and the audit and compliance group is roughly likely interested in look at in what spate are doing in regard to nociceptive data.Other possible uses for log data include marketing, forensics and HER accounting. As they identify goals, companies would do well to go through the broader advantages of log management and analysis, and look for systems or religious supporter that will allow a migration toward a more complete use of log data in the future. Of importance to all groups is the type of reporting supplied by the divine service or system. Log management systems often fill reporting that is geared toward compliance for PC, SOX, HAIFA and other similar standards.Apart from required reports, log management great deal generate reports that are helpful for system nutriment, warranter management and many other purposes. Whether log management is handled in-house or in the cloud, reporting and correlation features should be well to use and able to bear on current and future business goals. Locate Resources Critical to the success of any log management initiative is finding the staff needed to implement, manage and maintain the system. This is particularly difficult for SMB and government agencies that fag endt afford top horse for IT talent.Yet, according to a Gardner paper in May of 20082, compliance pokers are pushing organizations with smaller security staffs to acquire log management systems. In these cases, in-cloud services make sense. Larger organizations with dedicated security staffs and advanced log management processes, on the other hand, are more likely to musical accompaniment log management functions in-house. But even those organizations might use log management services for branches, or as a part of their hulky security or network management operations. 2 GO 56945, rank Nicole and Kelly Savannas.SANS Analyst Program Try Before You Buy The computer industry is fraught with solutions that dont work nearly as well as they purport. So, interrogatory and run use is critical to reconcile whether the system or service suits your needs. Put the search interface through its paces to rise for functionality and accuracy. Start tally with a few devices sending log data, but similarly set up as many devices as you are allowed to test during the trial period. Some log management systems work very well for a small beat of data but as the data provisions test larger, the accomplishment goes down str aightawaylyand the systems or services can miss events.A genuine way to test the system or service is to send some suspicious data to a device that is world monitored. Then go look for that particular data to make current its all there in the logs. One way to do this is to use the Kiwi Slog Message Generators to send messages to the tar live on, for example by using an option in the program to send a simple text message followed by a heel. This makes it simple to think if any of the test messages have been picked up by the log management system or service and reported upon as required.If there is a security function to the monitoring service (there usually is), try accessing your server and see how the provider responds. The specifics of how you would do this test will vary with your goals, but put down in as a drug user and intentionally mistyping the word of honor large time to lock the account should get a retort from the log service or system. I have actively use d this testing approach on some stratagems that collected security information and never got a response. If you choose to do this kind of testing, start slowly to get an idea of where the response threshold is.In addition to testing for nationality and security, pay watchfulness to the user interface. In close cases, this will be a weather vane-based face up end. Go through all the options and make current they work. Also, make sure that responses to the GUI are intuitive. If you have a report that you need regularly, you should be able to get that report reasonably easily, even have it e- mailed to a specified account. Custom reports and surplusized reports may be more mingled to receive as a test, but the prefatory hightail it of the system should make sense.Finally, make sure that the people who will use the service test the interface before decisions are finalized. Www. Sociology. Com/kiwi-slogged-overview Questions for the Cloud Provider Selecting a log management sof tware service provider is more like cementing a partnership than making a purchase. The service provider will have copies of critical log dataat times they may have the only copies of that data. The table below offers a quick snapshot of what to pout in a Service direct symmetricalness with a log management cloud service provider.Following that are questions to consider before taking the plunge. AAAS availability No more than 2 proceedings of downtime a day and no more than 5 transactions per week. seasonableness of log data showing up in system person logged events must be available to a search from the customer access within 90 befriends of the event. Timeliness of log data analysis restrictive compliance Alerts must be delivered to the client within 30 minutes of a critical event. The AAAS provider must maintain compliance to ever-changing regulations within 30 days of notification of change.New attack vectors should be confine to the bear upon system within 24 hour s of a parvenu attack being identified. The processing system must be upgraded to support changes and modifications to warn from support systems when systems are available for mineral release. Prompt upgrades to support new attack vectors Prompt upgrades to support upgrades to hardware and software 4 When considering cloud-based log management applications, organizations should ask the following questions (most of which can to a fault be use to in-house log management systems) Is It Safe?Many IT managers are concerned with the prophylactic of their log data, and rightly so Log data can be dangerous if it falls into the wrong hands. Attackers can get valuable information from reading the logs. For example, they can see if their attacks work, been known to show up in logs). Log data as crude as Web or e-mail handicraft often contains mystical information. Having control of logs can be useful to aggressors who, in some cases, will try to clean the log data to remove any traces of their activity. Therefore, its important to look at the gum elastic of log datawhether its stored on- or off-site.If the log data is stored local anestheticly, its often unploughed on each individual computer producing the data. Larger organizations will have log servers that will store the log data in a centralized attached storage device. Those systems are, in an ideal situation, secured and difficult to break through into. In the cloud model, this data storage would be handed off to the cloud provider, which relieves the organization of the hardware, security and HER burdens involved with keeping storage in-house. However, as they lose control of that data, organizations must rely on the cloud service to handle their data securely.The issue of whether a service organization is qualified is difficult to determine, and is ultimately based on reputation. Cloud providers must pretend a trust model as they manage collected log data securely and separately in a multi-tenant enviro nment. This creates the need for superfluous layers of security to operate multiple tenants from one another on a shared server, while also protecting the data stores from attackers. Firewalls, encryption and data loss prevention are all areas of security that apply to sensitive log data stored in the clouda cloud thats more and more brutalized.Fertilization, in itself, is not necessarily a negative, as long as proper security procedures are followed within the virtual cloud. The alike(p) characteristics of medical dressing that make it a concern as a hacking agent also provide a hiding technology that has the potential to make user accounts harder for attackers o access. Already security vendors are developing virtual technologies so that their anti-mallard products cant be detected and overruled by todays kernel boot- level rootlets. 5 How Is It Transported? charter the cloud provider for specifics about how the data gets transmitted from your systems to their operations cente r. Is the data encrypted in transit? What type and strength of encryption is used? Is the encryption proprietary? Be wary of providers that claim their encryption information is confidential or proprietary instead, look for providers that use proven technologies such as SSL or AES. There are numerous examples of companies that have invested vast amounts of money in creating their own encryption technologies only to find out afterwards release that they missed a critical component.How Are Keys Stored? It would be easier for a log management vendor to use the same encryption secrets client, that attacker can access the accounts of all clients. A different key for each customer account would not only offer better auspices against customers accessing one anothers accounts, but also against an attacker cracking a password and getting the keys to the entire kingdom. Logical separation of key storage is also important for the same reasons. How Often Is The Data Transmitted? Most log mana gement systems send data in batch mode.The collection appliance typically waits for either a specified time or amount of data before contagion. In general, a quicker frequency is better because the data is getting processed faster. More frequent transmission minimizes traffic bursts and gives an attacker less time to interrupt or block the transmission of qui vives, a technique attackers use in an attempt to avoid detection. What Is The Level Of Compression and Bandwidth Utilization? Bandwidth utilization is a question that youll want to keep an eye on as you test your log management service.It is common to get 90 percent compression or better on ASCII (plain text) logs, while binary log compression ratios may be less. If your Internet connection is currently heavily utilized, the log traffic may check other traffic, and youll want to plan for this issue ahead of time. One way to monitor the bandwidth is to capture traffic statistics using Net Flows. If you arent monitoring your o verall Internet traffic utilization, its best to get a handle on that prior to implementing a log management service and use this number as a baseline. What Backup and Redundancy Is Included ? If a cloud provider claims to be set up to handle this type of data correctly, verify that it is, in fact, doing a better Job than you would. The provider should have data stored at multiple locations and secure backups for redundancy. Check, too, with the company that is actually doing storage. In the cloud model, storage could be handed off to another vendor. aim questions about how stored data is encrypted, how it is transferred, where the tapes or other media are stored, and if there is a process for racking tapes.Find out how long backup data is bear, how its destroyed, and what happens to the data if the service is terminated. It will probably be impossible to verify most of this, but the cloud provider should be able to answer questions and provide benchmarks, customer references, se rvice agreements and other documentation. What Are The Responding Options? transcription. These built-in reports typically cover things like restrictive compliance and common performance and security metrics. see to it that the reports your organization needs are included as overbuilt reports, or that theyre easy enough to customize.Often, reporting is not as straightforward as people would like it to be. Sometimes, the logging application wont provide the required information directly, but it may be available indirectly. For example, a security manager may want to identify invalid session IDs on a Web site because a high frequency of invalid session IDs may point to an attacker trying to guess a session ID and knockoff or hijack the session. If the log manager doesnt report that information directly, it may be possible to get similar information by tracking the number of connections built from any given IP address. How Much Of The Data Is Actively Searchable?In some cases, the most recent data will be more quickly accessible for searching than data that has been take away from an active state. Moving data out of an active part of the database can make databases faster, so some data may be move into an area that provides slower access. Ask if there are any special requirements to access archived data or whether the only issue is a performance penaltyand request a demonstration. 7 How Much Of The Data Is Stored? If data is moved out of primary storage, is the full log data retained or will recovery of data be limited to searchable data fields and metadata?If some detail is eliminated, determine whether this will cause problems with regulatory compliance, forensics processes and other log management and reporting needs required crossways your organization. If there are processes that automatically eliminate some data, can those processes be suspended for special circumstances, such as litigation requiring the preservation of data? How long does it take to make such changes? What Log Data pull up stakes Be Accepted? What specific devices, operating systems and applications are supported?several(prenominal) operating systems and hundreds of widely used appliances and devices are critical o todays diverse organizational IT infrastructures. The number of applications a log manager may be called upon to understand is staggering. Prioritize on all your critical devices and applications. How are they supported by the service provider, and how thorough is that support? How Are Its Instructions For setting Up Devices? Log management can become more complicated as the number of log-producing for setting up devices, operating systems and applications that need to be monitored.Often, a company will need to deviate from the normal setup procedure, based on the peculiarities of its business that complicate the log data disembodied spirit cycle. As a result, setup instructions should be termed as guidelines, not hard and fast rules. Rules oft en must be massaged to work with the variable operating systems and applications (including their versions) that an organization needs coverage for. 8 How Are Alerts placed? If the cloud provider is offering to send alerts for events of interest, find out how they determine what is of interest and compare that to what is of interest to your organization.Are they looking solely at security events or do they include more routine support and maintenance events? If the events of interest include both types f events, how do they distinguish between the two? How much involvement does the log management client have in setting up what alerts are of interest to them? If a drive runs out of space, for example, that can often be Just as big a problem as an attacker compromising a system. Ask, also, if they can correlate related events to give the analysis situational awareness. For example, an administrator logging into a domain controller at 10 a. . And creating a user is quite an differen t from the DNS process starting a command shell and creating a user in the middle of the night. In both cases, a user is being created. In the first instance, the process seems normal but in the second instance this combination of events could be associated with the RPC DNS exploit as show in an April, 2007, SANS Webmaster. Cloud (and in- house systems) should, therefore, include situational awareness to understand when creating a user is a normal event and when, as in the second example, it is not normal.In addition to alter monitoring and alerts, it would be ideal if cloud providers could offer human review of logs as an add-on fee for service. kind-hearted review is required under some regulations, and is a good basic best practice for organizations to follow because automated systems dont catch everything. How Quickly Does touch on Occur? Timing is an important issue with log management that the cloud model is well- suited to address. One typical problem with in-house log man agement is that events are often found after a problem is noticed.It is, of course, best to detect log events leading up to a critical event in order to avoid the critical event. The question about processing speed encompasses a number of different issues Once an event has been logged at the local device, how long does it take for that event to show up in the yester? If that event should trigger an alert, how long will it be before the alert is relayed to the client IT department? Is there an option for the vendor to do more than April 24, 2007 Webmaster www. Sans. Org/websites/show. PH? Beastie=90861 9 How Often Are The Alerts Updated? Operating systems and network devices are constantly coming under new and different attacks requiring new responses. The errors from these devices also change with some upgrades, so it is important for the Log Management provider to conduct regular and timely updates to its system, and respond reasonably when errors occur. How Are System Upgrades Ha ndled? In the cloud, upgrades to the log management systems are handled by the provider, thereby relieving the organization from having to maintain these systems in-house.There is a risk, however, that the upgrades may cause outages in coverage by accidentally introducing new compatibility or protocol problems. It would be a good to ask the cloud provider about how upgrades are handled and how clients are saved during the upgrades. By the same token, how would updates to any internal system log-generating devices affect the cloud providers coverage? 10 Considerations for In-House Log Management Many of the same questions that apply to companies offering log management service in the cloud also apply to internally-managed log management systems.The 2008 SANS Annual Log Management survey indicates it is still incredibly difficult to automate log management systems to the pointedness organizations need. A recent article by Patrick Mueller in Information calendar week refers to log ma nagement as a monster. Just because its difficult doesnt dream up log management needs to be outsourced. When weighing in-house log management, consider the following factors Could A Personal Change Ruin Your Log Management Process? Log management is often the pet project of one person while the rest of the IT staff tries not to get involved.If that person leaves the company, it can be difficult for initiatives. Will Your Staff Monitor The Logs on a regular basis And Maintain Updates? Log management services have requirements built into their contracts for response time and full-time monitoring. Can your staff live up to those same expectations? One of the issues for log management companies is keeping up with updates to applications, operating systems and regulatory issues. Is your staff able to keep up with the changes? As an example, how did your staff do when Windows Server 2008 changed all its event Ids?At the time, most administrators used a collection of scripts however, al l those scripts, which were working, suddenly became broken. Floggers lashed out about it. For a log administrator who finally has everything working, that sort of a situation can be a demoralizing surprise. Maintaining updates and monitoring logs is complicated by the fact that most companies support a diversity of logging devices. To properly support local log management, an IT group will need to work with different vendors ho use different types of log data.At times, it may be necessary to bring in consultants to assist with tracking down specific issues. Organizations need to consider the associated costs and frustrations of working with multiple vendors and integrators along with the costs of the initial deployment and current internal staffing requirements. 56 www. Informational. Com/story/charities. Jhtml? Articled=208400730 www. ultimate windows security. Com/wick/WindowsServer2008VistaSecurityLog. stays 11 Roll Your Own Or Buy An Appliance? A big debate in the log managem ent arena is how to deploy log management tools.According to the SANS Log Management survey, the majority of organizations (38 percent) are create home grown solutions through the use of slog servers, custom scripts and applications. The stay respondents used a combination of commercial software and appliance-based tools or check up on other. In either case, organizations are not happy with their level of automated correlation or system coverage, according to the survey. Coverage, automation, correlation and access must all be addressed, maintained and improved upon as needs dictate, careless(predicate) of which option is chosen.