Elite: Automatic Orchestration of Elastic Detection Services to Secure Cloud Hosting

Abstract

Intrusion detection on today’s cloud is challenging: a user’s application is automatically deployed through new cloud orchestration tools (e.g., OpenStack Heat, Amazon CloudFormation, etc.), and its computing resources (i.e., virtual machine instances) come and go dynamically during its runtime, depending on its workloads and configurations. Under such a dynamic environment, a centralized detection service needs to keep track of the state of the whole deployment (a cloud stack), size up and down its own computing power and dynamically allocate its existing resources and configure new resources to catch up with what happens in the application. Particularly in the case of anomaly detection, new application instances created at runtime are expected to be protected instantly, without going through conventional profile learning, which disrupts the operations of the application.

To address those challenges, we developed Elite, a new elastic computing framework, to support high-performance detection services on the cloud. Our techniques are designed to be fully integrated into today’s cloud orchestration mechanisms, allowing an ordinary cloud user to request a detection service and specify its parameters conveniently, through the cloud-formation file she submits for deploying her application. Such a detection service is supported by a high-performance stream-processing engine, and optimized for concurrent analysis of a large amount of data streamed from application instances and automatic adaptation to different computing scales. It is linked to the cloud orchestration engine through a communication mechanism, which provides the runtime information of the application (e.g., the types of new instances created) necessary for the service to dynamically configure its resources. To avoid profile learning, we further studied a set of techniques that enable reuse of normal behavior profiles across different instances within one user’s cloud stack, and across different users (in a privacy-preserving way). We evaluated our implementation of Elite on popular web applications deployed over 60 instances. Our study shows that Elite efficiently shares profiles without losing their accuracy and effectively handles dynamic, intensive workloads incurred by these applications.