Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/manuparra/clustering-openstack
Make a dynamic and customizable cluster with OpenStack
https://github.com/manuparra/clustering-openstack
cluster deployment hadoop openstack openstack-command script slave-nodes spark
Last synced: about 1 month ago
JSON representation
Make a dynamic and customizable cluster with OpenStack
- Host: GitHub
- URL: https://github.com/manuparra/clustering-openstack
- Owner: manuparra
- Created: 2018-02-16T16:36:23.000Z (almost 7 years ago)
- Default Branch: master
- Last Pushed: 2018-02-18T23:47:22.000Z (almost 7 years ago)
- Last Synced: 2024-11-07T09:31:36.115Z (3 months ago)
- Topics: cluster, deployment, hadoop, openstack, openstack-command, script, slave-nodes, spark
- Homepage: http://dicits.ugr.es
- Size: 36.1 KB
- Stars: 1
- Watchers: 3
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# clustering-openstack
### Project Description
In this project we pretend to describe the mecanism, used to deploy a cluster on Spark. We gonna describe the
### Table of Content
1. Introduction
2. OpenStack commands used in this project
* [OpenStack command line](#OpenStack-command-line)
* [Create instances](#Create-instances)
* [List the flavors](#How-to-show-the-flavors)
* [List the availabel images](#List-the-availabel-images)
* [List the availabel networks](#List-the-availabel-networks)
* [List the created security groups](#List-the-created-security-groups)
3. Scripting
4. Spark configuration
#### Create instances
To launch an instance, we must at least specify the flavor, image name, network, security group, key, and instance name.
We will work with the ``Name`` or ``ID`` of the elements that we will show:
---------------------
#### List the flavors
```
openstack flavor list
```
A flavor specifies a virtual resource allocation profile which includes processor, memory, and storage.
------------------------------
#### List the availabel images
```
openstack image list
```
Screenshot: List of images
We will use CentOS7 or Fedora images
--------------------------------
#### List the availabel networks
```
openstack network list
```
#### List the created security groups
```
openstack security group listlaunch a cluster of master and slaves cirtual machines on openstack command line
```
Screenshot: Security groups
------------------------
#### Create the instance
Example:
```
openstack server create --flavor XXXXX --image XXXXXX --nic net-id=XXXXXXX --security-group XXXXXX --key-name XXXXXXX provider-instance
```
With our data:
```
openstack server create --flavor 3 --image CentOS7 --nic net-id=55c3bd97-fef8-47cf-bde7-a7f6c22f2d2c --security-group default --key-name rashadkey provider-instance
```
--------------------------
### IP Floating
* #### List of Floating IP
```
openstack floating ip list
```
For each floating IP address that is allocated to your project, the command outputs the ID of the floating IP address, the actual floating IP address, the private IP address of the instance the floating IP address is associated with, and the ID for the port that the floating IP address is connected to.
* #### Disassociate floating IP of an instance:
Firstly, let us see the availabel instances "servers":
Let's disassociate the floating IP of the instance with the name, CirrOS-cloud-init. We can see that its floating IP is 192.168.10.53. To disassociate it, we do the following:
```
openstack server remove floating ip CirrOS-cloud-init 192.168.10.53
```
Screenshot: Disassociate Floating IP:
We can see how this floating ip is no longer associated to the specified instance.
* #### Create floating IP
To associate a floating IP, we can use an exisiting one or create a new one and assign it to our project, before assign it to an instance.
```
openstack floating ip create
```
We can see that network in our example, is external.
* #### Associate floating IP to an instance
```
openstack server add floating ip CirrOS-cloud-init 192.168.10.68
```
Now we can see how the instance called, CirrOS-cloud-init, has an associated floating IP, and it's, 192.168.10.68; which we created in previous step.
---------------------------------
### Assign IP (internal)....
How to assing a specific Internal IP (10.....) to the instance.
## Script
The script will allow us to create an instance environment with specific configuration parameters.
Input Parameters:
- Options: {start, status and delete}
- Start: Create a new cluster with a name;
- Delete: Remove all instances assiciated to the cluster;
- Status: check the status of the cluster.
- Name of the cluster (identifier of the cluster)
- Number of master nodes
- Number of slave nodes
- IP master node (floating)
- IP slaves nodes (internal)
- Flavor of the set of instances
- Network Name for the instances
- Security group
- Key Name for the instances
- Image for the instances
Parameter Code:
```
import sys
import random,sys,os,math
import argparse
import json
def main():
parser = argparse.ArgumentParser(description='ClusterOpenStack')
parser.add_argument('-op','--operation', help='Operation of the cluster', required=True)
parser.add_argument('-name','--name', help='Name of the cluster', required=True)
parser.add_argument('-nm','--nummasters', help='Num Masters', required=True)
parser.add_argument('-ns','--numslaves', help='Num Slaves', required=True)
parser.add_argument('-ipm','--ipmasters', help='IPs of Masters', required=True)
parser.add_argument('-ips','--ipslaves', help='IPs of Slaves', required=True)
parser.add_argument('-fl','--flavor', help='Flavor of the instances', required=True)
parser.add_argument('-n','--network', help='Network name or ID', required=True)
parser.add_argument('-s','--security', help='Security name', required=True)
parser.add_argument('-i','--image', help='Image name', required=True)
args = vars(parser.parse_args())
print args.operation
print args.name
main()
```
#### Spark configuration.
#### Bibliograpgy
- Launch instance provider
* https://docs.openstack.org/mitaka/install-guide-ubuntu/launch-instance-provider.html
- Floating IP
* https://docs.openstack.org/python-openstackclient/pike/cli/command-objects/floating-ip.html
* https://help.dreamhost.com/hc/en-us/articles/215912768-Managing-floating-IP-addresses-using-the-OpenStack-CLI