Background

It's not simple to know what happens in a bigger network. There's a multitude of applications, services and appliances working together. Many of them provide some kind of events or state information. The network administrator needs to get hands on all of them. But they come in many different flavors and multiple canals. Therefore, it's hard to get the big picture. Furthermore, we have learned that it's impossible to protect a system against all malicious attacks and to keep all the possible faulty handling away. A monitoring of the systems to guarantee a pro-active handling is therefore needed..

Therefore, more and more organizations collect and analyze all logfiles in a centralized system, called a SIEM (security information and event management). The technology provides two major functions for security events from networks, systems and applications: log management and compliance reporting (SIM - security information management) and real-time monitoring and incident management (SEM - security event management).

About Apache ALOIS

Apache ALOIS is a log collection and correlation software with reporting and alarming functionalities. ALOIS stands for "Advanced Log Data Insight System" and is meant to be a fully implemented open source SIEM security information and event management system.

While almost all other SIEM software, be it closed or open source, concentrate on the technological part of security monitoring, Apache ALOIS is aimed to monitor the security of the content. It intends to be pro-active in the detection of potential loss, theft, mistaken modification or unauthorized access. Apache ALOIS works on log messages and thus contains all the basic functionality of a conventional SIEM, as centralized collecting, normalizing, aggregation, analyzing and correlating of all log messages, as well as reporting all security related events. Therefore it can be used as any other SIEM.

Why another security information and event management system? It's true, there's already plenty of them. While the proprietary software is way too expensive for smaller to mid-sized companies, we find that the open source solutions are either too simple or not completely open. For example, behind each of the well known systems "OSSIM" and "Prelude", there is a company that either closes central functionality for its own business or has dual licensing and therefore asks the full copyright for all contributed code.

Apache ALOIS is aimed to be totally free and open for all contributions. The openness provided for other programming languages is certainly proof of this. The plug-ability - yet to be further developed - is meant to guarantee that individual needs can be realized without stressing the whole system too much. In our opinion, the Linux kernel is a good example that this can work very well.

Security Information and Event Management (SIEM)

Security Information and Event Management (SIEM) solutions are a combination of the formerly disparate product categories of SIM (security information management) and SEM (security event management). SIEM technology provides real-time analysis of security alerts generated by network hardware and applications. SIEM solutions come as software, appliances or managed services, and are also used to log security data and generate reports for compliance purposes.

The acronyms SEM, SIM and SIEM have been used interchangeably, though there are differences in meaning and product capabilities. The segment of security management that deals with real-time monitoring, correlation of events, notifications and console views is commonly known as Security Event Management (SEM). The second area provides long-term storage, analysis and reporting of log data and is known as Security Information Management (SIM).

The term Security Information Event Management (SIEM), coined by Mark Nicolett and Amrit Williams of Gartner in 2005, describes the product capabilities of gathering, analyzing and presenting information from network and security devices; identity and access management applications; vulnerability management and policy compliance tools; operating system, database and application logs; and external threat data. A key focus is to monitor and help manage user and service privileges, directory services and other system configuration changes; as well as providing log auditing and review and incident response.

[This is an excerpt from Wikipedia. Read full text here: http://en.wikipedia.org/wiki/SIEM.]

Architecture of Apache ALOIS

Apache ALOIS consists of five modules interacting to ensure a scaleable functionality of a SIEM:

  • Insink is the message sink, which is the receiving entry point for all the different log messages into Apache ALOIS. It is partly based on the syslog-ng software. Insink listens for messages (UDP), waits for messages (TCP), receives message collections (files, emails) and pre-filters them to prevent from message flow overload.
  • Pumpy is the incoming FIFO buffer, implemented as a relational database tables. which contain the incoming original messages (in raw format). In a complex system setup, there may be several insink instances, e.g. for a group of hosts, for specific types of messages, or for high-avaliablity.
  • Prisma contains logic to split up the text of log messages into separate fields, based on regular expressions. Actually, "prisma" is a set of "prismi", each one prisma for one type of log message (apache, cisco etc. Several prismi can be applied to the same message. This allows for stacked messages, i.e. forwarded log messages contained in compressed files contained in e-mail messages. The data retrieved form the log messages is stored in a database called Dobby. Due to prisma being written in Ruby, prismi can be applied interactively (when having system access).
  • Dobby is the central log database. It should be separated from the Pumpy database for availability and performance reasons. The current implementation is based on MySQL.
  • The Analyzer contains the two sub-systems Lizard and Reptor. Lizard is the analysis engine and user interface of Apache ALOIS, implemented in Ruby on Rails using AJAX. It allows for interactive browsing through the collected data, exclusion/inclusion/selection of data, data sorting, data filtering, creation of views, ad-hoc textual and graphical reporting. Reptor allows for automatic activation of views and comparison of these views' results to a predefined result (pattern matching). In case of mismatch, Reptor sends the result to predefined e-mail addresses.

Since an image expalins more than a thousand words, here is an overview of the data flow through the different modules:

Its modular design guarantees Apache ALOIS to scale from little to large organizations. Since there exists a Debian package, it's easy to build a test system or even a productive system for small environments.

Apache ALOIS and data retention

Apache ALOIS is a great tool to collect and analyze logs. As a scalable open source tool it may be used in a personal environment to create a log of personal activities in much the same way as a state data retention would collect logs. It could therefore be used for personal awareness regarding what could and would be retained. However, not all of what state data retention would collect, is currently available to Apache ALOIS. Most prominently, the mobile phone roaming data is only available on the mobile phones themselves, and although these devices are in the user's hand, getting that data to be transferred to Apache ALOIS is as of yet not possible. There is some work to be done on the mobile phone side to allow that.

On the other hand, Apache ALOIS is a dual-use tool and could be used in state data retention environments. As it was not intended to be used this way, as it contains functionality way beyond the collection for retention purposes and as it probably does not (yet) scale well enough for dumping massive data amounts, its use as a state data retention tool is unprobable and no such cases are known. However, it is open source software and may be put to any use.

Outlook

Although the software has been in productive use for a few years, there is still a lot of desired functionality missing. The plugability of new connected systems is given, but needs some revision. It is a given goal of the project to allow modules in other programming language. Furthermore, it has been discussed if parts of the existing implementation may be replaced with other proven open source software, e.g. the correlation engine or the web frontend. The other way round, it has been discussed that the filter creation engine would make a good tool for any kind of structured data, and thus could be separated from ALOIS and standardized as a stand-alone tool.

alois的更多相关文章

  1. [转] Ramda 函数库参考教程

    学习函数式编程的过程中,我接触到了 Ramda.js. 我发现,这是一个很重要的库,提供了许多有用的方法,每个 JavaScript 程序员都应该掌握这个工具. 你可能会问,Underscore 和  ...

  2. ARC 066D Xor Sum AtCoder - 2272 (打表找规律)

    Problem Statement You are given a positive integer N. Find the number of the pairs of integers u and ...

  3. 诊断Java代码中常见的数据库性能热点问题应该这么做!

    “你的Java应用程序的性能是怎样诊断和优化的?不妨看看这两位西医的方子.如果你有更好疗效的药方,也欢迎在评论区告诉我们. 当我在帮助一些开发者或架构师分析及优化Java应用程序的性能时,关键往往不在 ...

  4. CNCF 宣布成立应用交付领域小组,正式开启云原生应用时代

    作者|赵钰莹 作为云原生领域的顶级开源社区, Cloud Native Computing Foundation (云原生基金会,以下简称 CNCF)近日宣布成立 Application Delive ...

随机推荐

  1. Spring注释(转)

    转自:http://www.ibm.com/developerworks/cn/java/j-lo-spring25-ioc/#ibm-pcon 概述 注释配置相对于 XML 配置具有很多的优势: 它 ...

  2. Android-Ant自动编译打包android项目 -- 2 ----签名与渠道包

    上篇介绍了怎么使用ant自动编译打包现有的android项目,这篇将继续介绍如果如何在ant打包应用的时候加入签名信息以及自动打包渠道包. 1. 加入签名信息: 在项目的根目录下建一个ant.prop ...

  3. poj 1065 简单的贪心算法

    题意大概是:有一组木头需要处理,每块木头有长度,重量两个属性,处理的前一块木头长len,重wei,当下一块木头的len1,与wei1满足:len1>len&&wei1>we ...

  4. 优步北京B组(8月10日-8月16日奖励规则)

    奖励政策: [优步北京B组] 定义为2015年7月20日前激活的部分司机(以优步后台数据显示为准) 滴滴快车单单2.5倍,注册地址:http://www.udache.com/ 如何注册Uber司机( ...

  5. 【COM学习】之二、HRESULT,GUID

    HRESULT 来向用户报告各种情况.   他的值位于  WINERROR.H中 QueryInterface返回一个HRESULT值. HRESULT不是一个句柄,他是一个可分成三个域的32位值. ...

  6. [LeetCode] Rotate Image [26]

    题目 You are given an n x n 2D matrix representing an image. Rotate the image by 90 degrees (clockwise ...

  7. 如何在Github Pages搭建自己写的页面?

    教程一大堆,却没有几个能看懂的,问题一:90%的都在讲解如何搭建博客,和我想要将自己的网页部署到上面还是有点区别的.问题二:所有的教程都用到了Git,而我只知道Git是一个开源的分布式版本控制系统.完 ...

  8. html系列教程--article audio

    <article> 标签 <article> 标签规定独立的自包含内容.一篇文章应有其自身的意义,应该有可能独立于站点的其余部分对其进行分发. <article> ...

  9. UCML快速开发平台学习1-UCML环境安装

           最近公司项目时间紧张,经过各位大神的PK,决定用多年前话10W采购过来,一直被雪藏的UCML来开发.为啥花了钱买回来不用我就不吐槽了. UCML安装         翻看安装手册,貌似不 ...

  10. TCP应用编程--套接字C#实现

     套接字之间的连接过程可以分为三个步骤: 1.服务器监听 2.客户端请求 3.连接确认 Ø服务器监听:是指服务器套接字并不定位具体的客户端套接字,而 是处于等待连接的状态,实时监控网络状态. Ø客户端 ...