Information Security Effectiveness Metrics: What …

1 IBM Global ServicesSecurity & Privacy ServicesInformation Security EffectivenessInformation Security EffectivenessMetrics: what metrics ? what Role for metrics ? metrics : what metrics ? what Role for metrics ?MatundaNyanchama, PhD, CISSPN ational Leader, Security & Privacy Delivery ServicesIBM Global Services, CanadaE-mail: & PrivacyIBM Global ServicesNovember 21, 2004 November 21, 200422 Copyright IBM Global ServicesCopyright IBM Global ServicesAgendaAgenda Background Some Definitions Why metrics ? IS metrics -Background Value Information Security metrics metrics Development Process Scope of Measurement ISO 17799 Scoping out IS metrics Information Security Program Example Scope of Considerations for Measurement Examples of Measures metrics & Reporting Data Sources for IS metrics IS metrics Process & Reporting metrics Breadth, Depth & Purpose Incident Management Example Sample IS Dashboard State of IS metrics & Caveats & Some Suggestions SummarySecurity & PrivacyIBM Global ServicesNovember 21, 2004 November 21, 200433 Copyright IBM Global ServicesCopyright IBM Global ServicesSome DefinitionsSome Definitions Metric: relating to measurement; involving, or proceeding by, measurement (Webster s Revised Unabridged Dictionary) Information Security pertains to integrity, confidentiality& availability.

2 Auditabilityandaccountability Security Metric: A measurable attribute of the result of a Security engineering process that could [be] evidence its Effectiveness . (see references) Effectiveness :Having an intended/expected effect; operative; in effect; efficacy, force, punch, power, strength, success, validity, vigor, weight (The American Heritage Dictionary) Efficiency:Production of desired effect/results with minimum waste of time, effort, or skill ; a measure of Effectiveness ; specifically, theuseful output divided by input into a system; proficiency, capability, adeptness, adequacy, suitability (The American Heritage Dictionary) Benchmark:Reference, a standard by which something is measured; criterion, gauge, goal, measure, standard, touchstone, yardstick Return on Investment (ROI):A measure of profitability; it measures how effective a company uses its capital to generate profit; income that an investment provides in a specified time ( one year) Security & PrivacyIBM Global ServicesNovember 21, 2004 November 21, 200444 Copyright IBM Global ServicesCopyright IBM Global ServicesWhy metrics ?

3 Why metrics ? metrics are intended to: Focus on measurable attributes ..that could serve as evidence of Effectiveness /efficiency of a given program or process Facilitate decision making: what are the shortcomings? How closely are objectives met? Gaps/shortcomings if any? Need change of direction? Help improve performance and accountability: where are gaps? How can things be done better? Who is responsible? Can be objectiveor subjective, and quantitativeor qualitative. To be relevant, metrics should be SMART, Specific, Measurable, Attainable, Repeatable and Time-independentRemember: If you cannot measure it, you cannot manage it. -anonQuestion? Where are we with Information Security metrics ? Security & PrivacyIBM Global ServicesNovember 21, 2004 November 21, 200455 Copyright IBM Global ServicesCopyright IBM Global ServicesValue Information Security metrics Value Information Security metrics --II IS performance against defined IS goals, Efficacy of Information Security Accountability to stakeholders Assess IS plans, programs, processes, etc.

4 For Efficiency how well Information Security resources are utilized Effectiveness of Information Security program + existing Security controls Identify IS risks what assets need protection? what is their value? what threats and vulnerabilities exist to the assets? what chances for exploitation exist? IS Risk Management Risk assessment -extent of exposure to threats + potential business impacts should attacks happen Controls - what countermeasures/controls to identified risks Controls assessment -How effective are those controls Assess IS posture Security & PrivacyIBM Global ServicesNovember 21, 2004 November 21, 200466 Copyright IBM Global ServicesCopyright IBM Global ServicesValue Information Security metrics Value Information Security metrics --IIII Security posture trends is the state of Security improving, staying the same or getting worse? Help identify priorities for resource deployment in based on risk levels to assets Facilitate corrective action where controls are weak, whereincident response times are unacceptable Demonstrate the value of Information Security to executives Benchmark against industry, where possible -how do we compare with our peers in industry?

5 Can be used for compliance-related assessments SOX for internal controls assessmentSecurity & PrivacyIBM Global ServicesNovember 21, 2004 November 21, 200477 Copyright IBM Global ServicesCopyright IBM Global ServicesInformation Security metrics Benefits SummaryInformation Security metrics Benefits Summary Productivity indicators: Effectiveness & efficiency of a Security program Security return on investment (ROI) (where possible to measure) Information Security program maturity Information Security posture: Collected data can be used as baseline for measurements & trending Risks are identified and a business case made to address the risks Help define a baseline and hence deviations: Apply risk management methodology for deviations from baseline Quantify risk and hence plan for better risk management strategyUsed appropriately: metrics can engender process improvement. Demonstrate value of Information Security investment, ROI Facilitate risk management Allow benchmarking with industry peersSecurity & PrivacyIBM Global ServicesNovember 21, 2004 November 21, 200488 Copyright IBM Global ServicesCopyright IBM Global ServicesMetrics Development ProcessMetrics Development ProcessFollow ISO17799 s plan-do-check-act cycle Plan Establish key objectives for the metrics required Identify the required metrics and hence required data Design & implement strategy for data collection & metrics generation Establish targets/benchmarks.

6 Where possible compare with industry Determine the process for collecting and analyzing data, and reporting Establish metrics review program, and the refinement process/cycle DO Communicate with stakeholders and ensure buy-in Implement the metrics program people, process and technology CHECK/Monitor Continuously review metrics report against objectives and benchmarks Monitor program performance against objectives and benchmarks Identify gaps, if any, in the program ACT Address gaps in program Refine specific metrics , where necessary Refine metrics program, where necessarySecurity & PrivacyIBM Global ServicesNovember 21, 2004 November 21, 200499 Copyright IBM Global ServicesCopyright IBM Global ServicesScope of Measurement Scope of Measurement ISO 17799 ISO 17799 Sample MeasurementsISO AreaSample MeasurementsGaps in policies; Potential impacts of policy gaps; # Security violations per period of Policy% staff with certification; formal roles and responsibilities; staff turnover; Security spending/employee; IS spending as % IT budgetSecurity Organization% assets in inventory; % assets with classification; % assets with valuation; % assets with protection planAsset Classification & Control# Security training sessions; level of Security awareness; # of personnel Security -related incidentsPersonnel SecurityFrequency of review of physical access; # access anomalies or violationsPhysical & Environmental Security # incidents; incident impacts; frequency of assessment; % systems with exposures; incident response metrics ; how quickly threats are communicated;frequency of awareness activities; change control issuesCommunications & Operations ManagementAccess activation/termination turnaround; % of expired accounts;% accounts with expired pwds; % of accounts with weak passwordsAccess Control% projects that use IS; # policy exceptions/risk acceptances.

7 %projects that perform code reviews; freq. of VAs; % systems with vulnerabilitiesSystems Development & Maintenance% systems with BCP/DRP; frequency of BCP/DRP testing; % systems that pass BCP/DRP testing; System availabilityBusiness Continuity Management# & trend of exemptions; ComplianceSecurity & PrivacyIBM Global ServicesNovember 21, 2004 November 21, 20041010 Copyright IBM Global ServicesCopyright IBM Global ServicesElements of an IS Program Elements of an IS Program The IS Management Life CycleThe IS Management Life CycleMaintain & Improve Security Management ProgramMonitor & Continuously Review Program PerformanceEstablish Information Security Management ProgramImplement Information Security Management ProgramDevelopment, Maintenance & Improvement of the ISM ProgramACTCHECKDOPLANKey Security Program ElementsStrategic Governance, Policies & Business Strategy Strategy, Policy, Procedures, Standards, Awareness PlanTactical Risk Assessment, Design Reviews, Due Care, New Technology Insertion, Risk Acceptance, Policy ExceptionsOperational Active Security : Intrusion Detection & Alerts, Incident Management, Vulnerability Assessments, Data Aggregation & Analysis, Trending, Root Cause Analysis.

8 what takes place daily captures the robustness or weakness of controls, incidents, external eventsSecurity & PrivacyIBM Global ServicesNovember 21, 2004 November 21, 20041111 Copyright IBM Global ServicesCopyright IBM Global ServicesInformation Security Information Security --Another ViewAnother (6-12) monthsOperational - Active Security Posture & Analysis Vulnerability Assessments Intrusion Detection & Alerts Incident Response Anti-virus Management Data Analysis & Trending Reporting AwarenessStrategicGovernance & Policies Business Strategy Policies Standards Procedures Guidelines Awareness Strategy ResearchTacticalApplications & Systems Development Risk Assessment Design & Code Reviews IS Solutions Due Care Risk Acceptance Policy Exceptions Technology Insertion AwarenessACTCHECKDOPLAND evelopment, Development, Maintenance & Maintenance & ImprovementImprovement-(1-3) yearsReference to Industry Standards: ISF, ISO17799, ITIL, COBITS ecurity & PrivacyIBM Global ServicesNovember 21, 2004 November 21, 20041212 Copyright IBM Global ServicesCopyright IBM Global ServicesScope of Considerations for MeasurementScope of Considerations for MeasurementPossible MeasuresOrganizational LevelSecurity OperationsIncidents, Vulnerability Assessment, Patch Management, threat advisoriesApplications & System DevelopmentProject AssessmentsRisk Acceptances, Code ReviewsStrategy & GovernanceInfo Sec Program + Framework; Information Security Budget Spending/employee% of IT budget in Info Sec; Policy Gaps in existenceBench marking against industry.

9 Industry standards adoptedAwareness plan% projects going assessment process# Outstanding policy exceptions & Risk acceptances% projects performing code reviewsFrequency of vulnerability assessments# outstanding vulnerabilitiesRate of fixing vulnerabilitiesRate of response to incidents & $ImpactsTrend of incident response losses# & frequency of awareness sessionsSecurity & PrivacyIBM Global ServicesNovember 21, 2004 November 21, 20041313 Copyright IBM Global ServicesCopyright IBM Global ServicesExamples of MeasuresExamples of MeasuresBusiness StrategyPoliciesStandardsProceduresGuide linesAwareness StrategyResearchVulnerability AssessmentsIntrusion Detection & AlertsIncident ResponseAnti-virus ManagementData Analysis & Trending ReportingAwarenessRisk AssessmentDesign & Code ReviewsIS SolutionsDue Care Risk AcceptancePolicy ExceptionsTechnology InsertionAwarenessSpan of Measurement Across the ISLCS trategic: Governance & PoliciesOperations Active SecurityApplications & Systems DevelopmentSecurity Spending/employeeStrength of the Security organizationSoundness of a Security framework and Security program% of IT budget given to Info SecBench marking against industry; Industry standards adoptedExistence or otherwise of an Examples of MeasuresExamples of MeasuresExamples of measures % projects going through Security assessment process Outstanding policy exceptions & risk acceptances % projects performing code reviewsFrequency of vulnerability assessments; # outstanding vulnerabilities; Rate of fixing vulnerabilities; Rate of response to incidents & $Impacts; Trend of incident response losses; # & frequency of awareness sessions; Existing policy gaps; IS program fit with other processes.

10 Feedback integration to Security life cycle Reference to Industry Standards ISO17799awareness planmanagementSecurity & PrivacyIBM Global ServicesNovember 21, 2004 November 21, 20041414 Copyright IBM Global ServicesCopyright IBM Global ServicesSources of Data for MetricsSources of Data for MetricsInformation SecurityVulnerability AssessmentsIncident dataIntrusion detection statisticsAntivirus statisticsProject assessment reportsPolicy exceptions & risk acceptancesEducation & Awareness dataRisk control self-assessmentAccess management reportsRisk Management GroupsAudit external & internalConfiguration managementOrganization UnitsLog analysis exceptionsCorporate Security reportsRisk control self-assessmentsIS ReportingRisk Assessment ReportsIS PostureSecurity & PrivacyIBM Global ServicesNovember 21, 2004 November 21, 20041515 Copyright IBM Global ServicesCopyright IBM Global ServicesIS metrics Process & ReportingIS metrics Process & ReportingAudienceManagementOperations TeamPlanningDivisionsAnalysis outputsSecurity PostureIS Posture ReportBenchmarksValue @ RiskOtherOtherOtherProcessRisk Management Methodology/ProcessInformation Sources Assessments projects, systems, infrastructure Policy reviews Vulnerability Assessments Intrusion detection statistics Incident Response Data Anti-virus statistics Access Management Systems (physical & logical) Logs Audit reports ext/Int.