loader

Using Elasticsearch to Trigger Alerts in TheHive

Security Incident Response Platforms (SIRP) (Also known as SOAR – Security Orchestration, Automation and Response) are providing integrated and real-time countermeasures against security breaches.  As SIRP usually generates incident reports after a breach, security analysts can use reports for further triage and investigation. 

Key features of  SIRP include:

  • Cyber Data collected from SIEM, endpoints and other sources
  • Prebuilt knowledge base of threats and vulnerabilities
  • Attack behavior analysis, including major observables in the breach 
  • Integrated work process in security case analysis and handling 
  • Forensic data retention for post-incident reporting and analysis
  • Tasks assignment and tracking

The following article introduces a typical SIRP, named TheHive, and how it receives security alerts generated by Elasticsearch for SOC analysts to investigate. 

What is Elasticsearch for TheHive?

TheHive

TheHive, as stated in its website, is “a scalable, open source and free Security Incident Response Platform, tightly integrated with MISP (Malware Information Sharing Platform), designed to make life easier for SOCs, CSIRTs, CERTs and any information security practitioner dealing with security incidents that need to be investigated and acted upon swiftly.”  In other words, it collects alerts from different sources, with which security analysts will manage the security cases from creation to closure. 

Elastic SIEM

Elasticsearch used to act as a log repository. With the introduction of Elastic SIEM, it is also powerful and versatile for threat hunting and anomaly detection. SIEM enables analysis of host-related and network-related security events as part of alert investigations or interactive threat hunting. It uses pre-built rules and machine learning jobs to periodically search for documents and detect anomalies that meet the criteria. All the detected signals will be stored in certain indices for management and retrieval.  

Elasticsearch for TheHive

While Elasticsearch can ingest and store all types of data, it can be used to pre-process the security logs and send detected signals to TheHive for further analysis. Think it as a connector that allows data to flow bi-directionaly so the Elasticseatch engine capabilities can be leveraged.

Use Elasticsearch for TheHive to Generate Alerts

The flow chart

The above chart describes the workflow of using Elasticseatch to send alerts to TheHive.

Components to be included:

Beats are open source data shippers which are installed as agents on users’ systems. Beats send security events and other data to Elasticsearch. In the 7.9 version, a single and unified solution called Elastic Agent is introduced. It means that users don’t need to install multiple beats on hosts and can centrally manage fleet of agents in Ingest Manager. 

Elasticsearch  is a real-time, distributed storage, search, and analytics engine. Elasticsearch can index streams of semi-structured data, such as logs or metrics.

Kibana is an open source analytics and visualization platform designed to work with Elasticsearch. It is used to search, view, and interact with data stored in Elasticsearch indices. Users can perform advanced data analysis and visualize your data in a variety of charts, tables, and maps. The watcher UI in Kibana is used to configure alerts sent to TheHive. Watcher is an Elasticsearch feature that you can use to create actions based on conditions, which are periodically evaluated using queries on your data. Specifically, the webhook service can be used to send a request to any web service, thus it enables the communication between Elasticsearch and TheHive. 

TheHive is an alert management platform that analysts can use to triage and investigate security breaches. 

What is included in the alert

When alerts are generated and sent to TheHive, the contents need to follow some particular templates in order to be accepted. 

After the alerts are imported into TheHive, analysts can go to the UI for further investigation. Usually they can get access to information including title, source, reference, type and description. Some alerts also include the severity level and observables so analysts can merge similar cases and speed up the investigation process. 

Benefits of using the Open Source SIRP with Elasticsearch 

Gain more security insights from data 

Elasticsearch takes all kinds of data collected from users’s systems and stores them in structured JSON documents, with a higher performance result. Its scalability, let alone the capability to enrich security data with, for example, geoip information, allows an integrated view of data and from which deep insights. 

Leverage SIEM to do threat detection

Elsatic SIEM is built with pre-defiend rules and machine learning jobs, which comply with frameworks, such as The MITRE ATT&CK, or other best practices. With known malicious executables or anomalies, Elastic SIEM is able to find signals of possible breach and largely reduce false positives according to historical patterns.  

Based on signals detected by the Elastic SIEM, watchers are able to use customized criteria, such as the severity level or the number of events to trigger alerts in TheHive. It streamlines the threat hunting and alert management process. 

Real-time response 

Watchers that query on the data and send alerts to TheHive can be customized to evaluate the condition on real-time basis. This allows security analysts to collaborate on investigation on the live stream and respond quickly on detected threats. Not surprisingly, the sources, references, observables and other enriched contents sent in an ongoing fashion enable analysts to fastly assign and trace alerts when they are created in TheHive.

Written by: Astrid Liu

References:

  1. Arnaudloos.com. 2020. Open Source SIRP with Elasticsearch and TheHive – Overview. [online] Available at: <https://arnaudloos.com/2019/open-source-sirp-overview/> [Accessed 29 August 2020].
  2. Elastic.co. 2020. Get Up And Running | SIEM Guide [7.8] | Elastic. [online] Available at: <https://www.elastic.co/guide/en/siem/guide/current/install-siem.html> [Accessed 29 August 2020].
  3. TrustRadius. 2020. List Of Top Incident Response Platforms 2020. [online] Available at: <https://www.trustradius.com/incident-response#:~:text=Incident%20response%20(IR)%20platforms%20guide,deploy%20preplanned%2C%20automated%20threat%20responses.&text=Through%20automated%20orchestration%2C%20incident%20response,resources%20required%20to%20manage%20incidents.> [Accessed 29 August 2020].
  4. Thehive-project.org. 2020. Thehive Project. [online] Available at: <https://thehive-project.org/> [Accessed 29 August 2020].
  5. GitHub. 2020. Thehive-Project/Thehivedocs. [online] Available at: <https://github.com/TheHive-Project/TheHiveDocs> [Accessed 29 August 2020].