Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/beloglazov/openstack-neat

OpenStack Neat: A Framework for Dynamic Consolidation of Virtual Machines in OpenStack Clouds
https://github.com/beloglazov/openstack-neat

Last synced: 4 days ago
JSON representation

OpenStack Neat: A Framework for Dynamic Consolidation of Virtual Machines in OpenStack Clouds

Awesome Lists containing this project

README 

        
# OpenStack Neat: A Framework for Dynamic Consolidation of Virtual Machines in Openstack Clouds



[![Build Status](https://travis-ci.org/beloglazov/openstack-neat.svg?branch=master)](https://travis-ci.org/beloglazov/openstack-neat)



OpenStack Neat is an extension to OpenStack implementing dynamic consolidation

of Virtual Machines (VMs) using live migration. The major objective of dynamic

VM consolidation is to improve the utilization of physical resources and reduce

energy consumption by re-allocating VMs using live migration according to their

real-time resource demand and switching idle hosts to the sleep mode.



For example, assume that two VMs are placed on two different hosts, but the

combined resource capacity required by the VMs to serve the current load can be

provided by just one of the hosts. Then, one of the VMs can be migrated to the

host serving the other VM, and the idle host can be switched to a low power mode

to save energy. When the resource demand of either of the VMs increases, they

get deconsolidated to avoid performance degradation. This process is dynamically

managed by OpenStack Neat.



In general, the problem of dynamic VM consolidation can be split into 4

sub-problems:



- Deciding when a host is considered to be underloaded, so that all the VMs

  should be migrated from it, and the host should be switched to a low power

  mode, such as the sleep mode.

- Deciding when a host is considered to be overloaded, so that some VMs should

  be migrated from the host to other hosts to avoid performance degradation.

- Selecting VMs to migrate from an overloaded host out of the full set of the

  VMs currently served by the host.

- Placing VMs selected for migration to other active or re-activated hosts.



The aim of the OpenStack Neat project is to provide an extensible framework for

dynamic consolidation of VMs based on the OpenStack platform. The framework

provides an infrastructure enabling the interaction of components implementing

the 4 decision-making algorithms listed above. The framework allows

configuration-driven switching of different implementations of the

decision-making algorithms. The implementation of the framework includes the

algorithms proposed in our publications [1], [2].



## More details



For more information please refer to the

[paper](http://beloglazov.info/papers/2014-ccpe-openstack-neat.pdf) describing

the architecture and implementation of OpenStack Neat and Chapter 6 of Anton

Beloglazov's PhD thesis: http://beloglazov.info/thesis.pdf



## Installation



Unfortunately, there is no clear installation and usage guide yet. However, the

basic installation steps are the following:



- Clone the repository on every compute and controller node.

- Adjust the configuration by modifying neat.conf file in the repo directory on

   every node. In particular, you have to specify the names of your compute nodes 

   in the `compute_hosts` field. Then, update the following three fields:

   `admin_tenant_name`, `admin_user` and `admin_password` according to your OpenStack 

   configuration. If you have the default values, the correct settings are:

``` 

admin_tenant_name: service

admin_user: nova

admin_password: NOVA_PASS

```

- [UBUNTU] Edit all the `openstack-*` files present in `/etc/init.d/`

   replacing `/etc/rc.d/init.d/functions` with `/lib/lsb/init-functions` and 

   `exec="/usr/bin/neat-$suffix"` with `exec="/usr/local/bin/neat-$suffix"` 

   on every neat node.

- [UBUNTU] Install all the dependencies with (while doing this, DO NOT remove 

   any python related package previously installed by OpenStack, it will break 

   your OpenStack install!):

```

apt-get install python-pip numpy scipy libvirt-python

sudo pip install --upgrade pyqcy PyContracts SQLAlchemy bottle requests Sphinx python-novaclient

sudo pip install mocktest

```

- Install the package by running the following command from the repo directory

   on every node: 

```

sudo python setup.py install

```

- Start the Neat services by running the following commands on every compute node:

```

python2 start-data-collector.py

python2 start-local-manager.py

```

- Start the global manager service by running the following command on the controller: 

```

python2 start-global-manager.py

```



You can monitor the current VM placement using the `./vm-placement.py` script.



Some information about running experiments on the system can be found in the

following thread:

https://groups.google.com/forum/#!topic/openstack-neat/PKz2vpKPMcA



## Who we are



This work is conducted within the Cloud Computing and Distributed Systems

(CLOUDS) Laboratory at the University of Melbourne: http://www.cloudbus.org/



The problem of dynamic VM consolidation considering Quality of Service (QoS)

constraints has been studied from the theoretical perspective and algorithms

addressing the sub-problems listed above have been proposed [1], [2]. The

algorithms have been evaluated using CloudSim and real-world workload traces

collected from more than a thousand PlanetLab VMs hosted on servers located in

more than 500 places around the world.



## Discussion group / mailing list



Please feel free to post any questions or suggestions in the project's

discussion group: http://groups.google.com/group/openstack-neat



## Issues / bugs



Please submit any bugs you encounter or suggestions for improvements to our

issue tracker: http://github.com/beloglazov/openstack-neat/issues



## Publications



[1] Anton Beloglazov and Rajkumar Buyya, "OpenStack Neat: A Framework for

Dynamic and Energy-Efficient Consolidation of Virtual Machines in OpenStack

Clouds", Concurrency and Computation: Practice and Experience (CCPE), John Wiley

& Sons, Ltd, USA, 2014 (in press, accepted on 19/05/2014).



Download: http://beloglazov.info/papers/2014-ccpe-openstack-neat.pdf



[2] Anton Beloglazov, "Energy-Efficient Management of Virtual Machines in

Data Centers for Cloud Computing", PhD Thesis, The University of Melbourne,

2013.



Download: http://beloglazov.info/thesis.pdf



[3] Anton Beloglazov and Rajkumar Buyya, "Managing Overloaded Hosts for

Dynamic Consolidation of Virtual Machines in Cloud Data Centers Under Quality of

Service Constraints", IEEE Transactions on Parallel and Distributed Systems

(TPDS), Volume 24, Issue 7, Pages 1366-1379, IEEE CS Press, USA, 2013.



Download: http://beloglazov.info/papers/2013-tpds-managing-overloaded-hosts.pdf



[4] Anton Beloglazov and Rajkumar Buyya, "Optimal Online Deterministic

Algorithms and Adaptive Heuristics for Energy and Performance Efficient Dynamic

Consolidation of Virtual Machines in Cloud Data Centers", Concurrency and

Computation: Practice and Experience (CCPE), Volume 24, Issue 13, Pages:

1397-1420, John Wiley & Sons, Ltd, New York, USA, 2012.



Download: http://beloglazov.info/papers/2012-ccpe-vm-consolidation-algorithms.pdf



## License



The source code is distributed under the Apache 2.0 license.



Copyright (C) 2012-2014 Anton Beloglazov, Google Inc.