Blog
Search…
Hacking
OpenStack
Docker and DevOps
Midonet integration with OpenStack Mitaka
MidoNet is an Open Source network virtualization software for IaaS infrastructure. | It decouples your IaaS cloud from your network hardware, creating an intelligent software abstraction layer between your end hosts and your physical network. | This network abstraction layer allows the cloud operator to move what has traditionally been hardware-based network appliances into a software-based multi-tenant virtual domain.
This definition from MidoNet documentation explains what MidoNet is and what MidoNet does.
At this I will post cover my experiences integrating MidoNet with OpenStack. | I used the following configurations:
All servers have CentOS 7.2 installed
OpenStack has been previously installed from RDO packages with multinode Packstack
- x3 NSDB nodes (Casandra and Zookeeper services)
- x2 Gateway Nodes (Midolman Agent)
- x1 OpenStack Controller (MidoNet Cluster)
- x1 OpenStack compute node (Midolman Agent)
NSDB NODES
Disable SElinux
1
setenforce 0
2
sed -i 's/SELINUX=enforcing/SELINUX=permissive/g' /etc/sysconfig/selinux
Copied!
Install OpenStack Mitaka release repository
1
sudo yum install -y centos-release-openstack-mitaka
Copied!
Add Cassandra repository
1
cat <<EOF>/etc/yum.repos.d/datastax.repo
2
[datastax]
3
name = DataStax Repo for Apache Cassandra
4
baseurl = http://rpm.datastax.com/community
5
enabled = 1
6
gpgcheck = 1
7
gpgkey = https://rpm.datastax.com/rpm/repo_key
8
EOF
Copied!
Add Midonet repository
1
cat <<EOF>/etc/yum.repos.d/midonet.repo
2
[midonet]
3
name=MidoNet
4
baseurl=http://builds.midonet.org/midonet-5.2/stable/el7/
5
enabled=1
6
gpgcheck=1
7
gpgkey=https://builds.midonet.org/midorepo.key
8
​
9
[midonet-openstack-integration]
10
name=MidoNet OpenStack Integration
11
baseurl=http://builds.midonet.org/openstack-mitaka/stable/el7/
12
enabled=1
13
gpgcheck=1
14
gpgkey=https://builds.midonet.org/midorepo.key
15
​
16
[midonet-misc]
17
name=MidoNet 3rd Party Tools and Libraries
18
baseurl=http://builds.midonet.org/misc/stable/el7/
19
enabled=1
20
gpgcheck=1
21
gpgkey=https://builds.midonet.org/midorepo.key
22
EOF
Copied!
Clean repo cache and update packages
1
yum clean all
2
yum update
Copied!
| Zookeeper Configuration | Install Zookeeper, java and dependencies
1
yum install -y java-1.7.0-openjdk-headless zookeeper zkdump nmap-ncat
Copied!
Edit zookeeper configuration file
1
vi /etc/zookeeper/zoo.cfg
Copied!
Add all NSDB nodes at the configuration file
1
server.1=nsdb1:2888:3888
2
server.2=nsdb2:2888:3888
3
server.3=nsdb3:2888:3888
4
autopurge.snapRetainCount=10
5
autopurge.purgeInterval =12
Copied!
Create zookeeper folder on which zookeeper will store data, change the owner to zookeeper user
1
mkdir /var/lib/zookeeper/data
2
chown zookeeper:zookeeper /var/lib/zookeeper/data
Copied!
Create myid file at zookeeper data folder, the ID should match with the NSDB node number, insert that number as follows:
1
#NSDB1
2
echo 1 > /var/lib/zookeeper/data/myid
3
#NSDB2
4
echo 2 > /var/lib/zookeeper/data/myid
5
#NSDB3
6
echo 3 > /var/lib/zookeeper/data/myid
Copied!
Create java folder and create a softlink to it
1
mkdir -p /usr/java/default/bin/
2
ln -s /usr/lib/jvm/jre-1.7.0-openjdk/bin/java /usr/java/default/bin/java
Copied!
Start and enable Zookeeper service
1
systemctl enable zookeeper.service
2
systemctl start zookeeper.service
Copied!
Test if zookeeper is working locally
1
echo ruok | nc 127.0.0.1 2181
2
imok
Copied!
Test if zookeeper is working at NSDB remote nodes
1
echo stat | nc nsdb3 2181
2
​
3
Zookeeper version: 3.4.5--1, built on 02/08/2013 12:25 GMT
4
Clients:
5
/192.168.100.172:35306[0](queued=0,recved=1,sent=0)
6
​
7
Latency min/avg/max: 0/0/0
8
Received: 1
9
Sent: 0
10
Connections: 1
11
Outstanding: 0
12
Zxid: 0x100000000
13
Mode: follower
14
Node count: 4
Copied!
| Cassandra configuration | Install Java and Cassandra dependencies
1
yum install -y java-1.8.0-openjdk-headless dsc22
Copied!
Edit cassandra yaml file
1
vi /etc/cassandra/conf/cassandra.yaml
Copied!
| Change cluster_name to midonet | Configure seed_provider seeds to match all NSDB nodes | Configure listen_address and rpc_address to match the hostname of the self node
1
cluster_name: 'midonet'
2
....
3
seed_provider:
4
- class_name: org.apache.cassandra.locator.SimpleSeedProvider
5
parameters:
6
- seeds: "nsdb1,nsdb2,nsdb3"
7
​
8
listen_address: nsdb1
9
rpc_address: nsdb1
Copied!
Edit cassandra\'s init script in order to fix a bug in the init script
1
vi /etc/init.d/cassandra
Copied!
Add the next two lines after #Casandra startup
1
case "$1" in
2
start)
3
# Cassandra startup
4
echo -n "Starting Cassandra: "
5
mkdir -p /var/run/cassandra #Add this line
6
chown cassandra:cassandra /var/run/cassandra #Add this line
7
su $CASSANDRA_OWNR -c "$CASSANDRA_PROG -p $pid_file" > $log_file 2>&1
8
retval=$?
9
[ $retval -eq 0 ] && touch $lock_file
10
echo "OK"
11
;;
Copied!
Start and enable Cassandra service
1
systemctl enable cassandra.service
2
systemctl start cassandra.service
Copied!
Check if all NSDB nodes join the cluster
1
nodetool --host 127.0.0.1 status
2
Datacenter: datacenter1
3
=======================
4
Status=Up/Down
5
|/ State=Normal/Leaving/Joining/Moving
6
-- Address Load Tokens Owns (effective) Host ID Rack
7
UN 192.168.100.172 89.1 KB 256 70.8% 3f1ecedd-8caf-4938-84ad-8614d2134557 rack1
8
UN 192.168.100.224 67.64 KB 256 60.7% cb36c999-a6e1-4d98-a4dd-d4230b41df08 rack1
9
UN 192.168.100.195 25.78 KB 256 68.6% 4758bae8-9300-4e57-9a61-5b1107082964 rack1
Copied!
Configure OpenStack Controller Nodes (On which Neutron Server is running)
Disable SElinux
1
setenforce 0
2
sed -i 's/SELINUX=enforcing/SELINUX=permissive/g' /etc/sysconfig/selinux
Copied!
Install OpenStack Mitaka release repository
1
sudo yum install -y centos-release-openstack-mitaka
Copied!
Add Midonet Repository
1
cat <<EOF>/etc/yum.repos.d/midonet.repo
2
[midonet]
3
name=MidoNet
4
baseurl=http://builds.midonet.org/midonet-5.2/stable/el7/
5
enabled=1
6
gpgcheck=1
7
gpgkey=https://builds.midonet.org/midorepo.key
8
​
9
[midonet-openstack-integration]
10
name=MidoNet OpenStack Integration
11
baseurl=http://builds.midonet.org/openstack-mitaka/stable/el7/
12
enabled=1
13
gpgcheck=1
14
gpgkey=https://builds.midonet.org/midorepo.key
15
​
16
[midonet-misc]
17
name=MidoNet 3rd Party Tools and Libraries
18
baseurl=http://builds.midonet.org/misc/stable/el7/
19
enabled=1
20
gpgcheck=1
21
gpgkey=https://builds.midonet.org/midorepo.key
22
EOF
Copied!
Clean repos cache and update the system
1
yum clean all
2
yum update
Copied!
Create an OpenStack user for MidoNet, change the password to match your own
1
# openstack user create --password temporal midonet
2
+----------+----------------------------------+
3
| Field | Value |
4
+----------+----------------------------------+
5
| email | None |
6
| enabled | True |
7
| id | ac25c5a77e7c4e4598ccadea89e09969 |
8
| name | midonet |
9
| username | midonet |
10
+----------+----------------------------------+
Copied!
Add admin role at tenant services to Midonet user
1
# openstack role add --project services --user midonet admin
2
+-----------+----------------------------------+
3
| Field | Value |
4
+-----------+----------------------------------+
5
| domain_id | None |
6
| id | bca2c6e1f3da42b0ba82aee401398a8a |
7
| name | admin |
8
+-----------+----------------------------------+
Copied!
Create MidoNet service at Keystone
1
# openstack service create --name midonet --description "MidoNet API Service" midonet
2
+-------------+----------------------------------+
3
| Field | Value |
4
+-------------+----------------------------------+
5
| description | MidoNet API Service |
6
| enabled | True |
7
| id | 499059c4a3a040cfb632411408a2be4c |
8
| name | midonet |
9
| type | midonet |
10
+-------------+----------------------------------+
Copied!
| Clean up neutron server | Stop neutron services
1
openstack-service stop neutron
Copied!
Remove neutron database and recreate it again
1
mysql -u root -p
2
DROP DATABASE neutron;
3
Query OK, 157 rows affected (11.50 sec)
4
​
5
MariaDB [(none)]> CREATE DATABASE neutron;
6
Query OK, 1 row affected (0.00 sec)
7
​
8
MariaDB [(none)]> GRANT ALL PRIVILEGES ON neutron.* TO 'neutron'@'localhost' IDENTIFIED BY 'ab4f81b1040a495e';
9
Query OK, 0 rows affected (0.00 sec)
10
​
11
MariaDB [(none)]> GRANT ALL PRIVILEGES ON neutron.* TO 'neutron'@'%' IDENTIFIED BY 'ab4f81b1040a495e';
12
Query OK, 0 rows affected (0.00 sec)
13
MariaDB [(none)]> exit
14
Bye
Copied!
Remove plugin.ini symbolic link to ml2_conf.ini
1
#rm /etc/neutron/plugin.ini
2
rm: remove symbolic link ‘/etc/neutron/plugin.ini’? y
Copied!
Remove br-tun tunnel used by neutron in all the nodes
1
ovs-vsctl del-br br-tun
Copied!
Install MidoNet packages and remove ml2 package
1
yum install -y openstack-neutron python-networking-midonet python-neutronclient
2
yum remove openstack-neutron-ml2
Copied!
Make a backup of neutron configuration file
1
cp /etc/neutron/neutron.conf neutron.conf.bak
Copied!
Edit neutron configuration file
1
vi /etc/neutron/neutron.conf
Copied!
Most of the options are already configured by our older neutron configuration, change the ones who apply to match this configuration
1
[DEFAULT]
2
core_plugin = midonet.neutron.plugin_v2.MidonetPluginV2
3
service_plugins = midonet.neutron.services.l3.l3_midonet.MidonetL3ServicePlugin
4
dhcp_agent_notification = False
5
allow_overlapping_ips = True
6
rpc_backend = rabbit
7
auth_strategy = keystone
8
notify_nova_on_port_status_changes = true
9
notify_nova_on_port_data_changes = true
10
nova_url = http://controller:8774/v2
11
​
12
[database]
13
connection = mysql+pymysql://neutron:[email protected]/neutron
14
​
15
[oslo_messaging_rabbit]
16
rabbit_host = controller
17
rabbit_userid = guest
18
rabbit_password = guest
19
​
20
[keystone_authtoken]
21
auth_uri = http://controller:5000/v2.0
22
admin_user=neutron
23
admin_tenant_name=services
24
identity_uri=http://controller:35357
25
admin_password=d88f0bd060d64c33
26
​
27
[nova]
28
region_name = RegionOne
29
auth_url = http://controller:35357
30
auth_type = password
31
password = 9ca36d15e4824d93
32
project_domain_id = default
33
project_name = services
34
tenant_name = services
35
user_domain_id = default
36
username = nova
37
​
38
[oslo_concurrency]
39
lock_path = /var/lib/neutron/tmp
Copied!
At my deployment these are the options I had to change to configure midonet
1
diff /etc/neutron/neutron.conf neutron.conf.bak
2
33c33
3
< core_plugin = midonet.neutron.plugin_v2.MidonetPluginV2
4
---
5
> core_plugin = neutron.plugins.ml2.plugin.Ml2Plugin
6
37c37
7
< service_plugins = midonet.neutron.services.l3.l3_midonet.MidonetL3ServicePlugin
8
---
9
> service_plugins =router
10
120c120
11
< dhcp_agent_notification = False
12
---
13
> #dhcp_agent_notification = true
14
1087c1087,1088
15
< lock_path = /var/lib/neutron/tmp
16
---
17
> lock_path = $state_path/lock
18
>
Copied!
Create midonet plugins folder
1
mkdir /etc/neutron/plugins/midonet
Copied!
Create a file called midonet.ini
1
vi /etc/neutron/plugins/midonet/midonet.ini
Copied!
Configure midonet.ini file to match your own configuration options
1
[MIDONET]
2
# MidoNet API URL
3
midonet_uri = http://controller:8181/midonet-api
4
# MidoNet administrative user in Keystone
5
username = midonet
6
password = temporal
7
# MidoNet administrative user's tenant
8
project_id = services
Copied!
Create a symbolic link from midonet.ini to plugin.ini
1
ln -s /etc/neutron/plugins/midonet/midonet.ini /etc/neutron/plugin.ini
Copied!
Sync and populate database tables with Midonet plugin
1
su -s /bin/sh -c "neutron-db-manage --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/midonet/midonet.ini upgrade head" neutron
2
su -s /bin/sh -c "neutron-db-manage --subproject networking-midonet upgrade head" neutron
Copied!
Restart nova api and neutron server services
1
systemctl restart openstack-nova-api.service
2
systemctl restart neutron-server
Copied!
Install midonet cluster package
1
yum install -y midonet-cluster
Copied!
Configure midonet.conf file
1
vi /etc/midonet/midonet.conf
Copied!
Add all NSDB nodes at zookeeper_hosts
1
[zookeeper]
2
zookeeper_hosts = nsdb1:2181,nsdb2:2181,nsdb3:2181
Copied!
Configure midonet to make use of NSDB nodes as Zookeeper and cassandra hosts
1
cat << EOF | mn-conf set -t default
2
zookeeper {
3
zookeeper_hosts = "nsdb1:2181,nsdb2:2181,nsdb3:2181"
4
}
5
​
6
cassandra {
7
servers = "nsdb1,nsdb2,nsdb3"
8
}
9
EOF
Copied!
Set cassandra replication factor to 3
1
echo "cassandra.replication_factor : 3" | mn-conf set -t default
Copied!
Grab your admin token
1
#egrep ^admin_token /etc/keystone/keystone.conf
2
admin_token = 7b84d89b32c34b71a697eb1a270807ab
Copied!
Configure Midonet to auth with keystone
1
cat << EOF | mn-conf set -t default
2
cluster.auth {
3
provider_class = "org.midonet.cluster.auth.keystone.KeystoneService"
4
admin_role = "admin"
5
keystone.tenant_name = "admin"
6
keystone.admin_token = "7b84d89b32c34b71a697eb1a270807ab"
7
keystone.host = controller
8
keystone.port = 35357
9
}
10
EOF
Copied!
Start and enable midonet cluster service
1
systemctl enable midonet-cluster.service
2
systemctl start midonet-cluster.service
Copied!
Install midonet CLI
1
yum install -y python-midonetclient
Copied!
Create a file at you home directory with midonet auth info
1
#vi ~/.midonetrc
2
​
3
[cli]
4
api_url = http://controller:8181/midonet-api
5
username = admin
6
password = temporal
7
project_id = admin
Copied!
Configure Compute nodes
Disable SElinux
1
setenforce 0
2
sed -i 's/SELINUX=enforcing/SELINUX=permissive/g' /etc/sysconfig/selinux
Copied!
Install OpenStack Mitaka release repository
1
sudo yum install -y centos-release-openstack-mitaka
Copied!
Add Midonet repository
1
cat <<EOF>/etc/yum.repos.d/midonet.repo
2
[midonet]
3
name=MidoNet
4
baseurl=http://builds.midonet.org/midonet-5.2/stable/el7/
5
enabled=1
6
gpgcheck=1
7
gpgkey=https://builds.midonet.org/midorepo.key
8
​
9
[midonet-openstack-integration]
10
name=MidoNet OpenStack Integration
11
baseurl=http://builds.midonet.org/openstack-mitaka/stable/el7/
12
enabled=1
13
gpgcheck=1
14
gpgkey=https://builds.midonet.org/midorepo.key
15
​
16
[midonet-misc]
17
name=MidoNet 3rd Party Tools and Libraries
18
baseurl=http://builds.midonet.org/misc/stable/el7/
19
enabled=1
20
gpgcheck=1
21
gpgkey=https://builds.midonet.org/midorepo.key
22
EOF
Copied!
Clean repos cache and update the system
1
yum clean all
2
yum update
Copied!
Edit qemu.conf
1
vi /etc/libvirt/qemu.conf
Copied!
Configure with the following options, by default all these options are commented, you can paste it all wherever you want
1
user = "root"
2
group = "root"
3
​
4
cgroup_device_acl = [
5
"/dev/null", "/dev/full", "/dev/zero",
6
"/dev/random", "/dev/urandom",
7
"/dev/ptmx", "/dev/kvm", "/dev/kqemu",
8
"/dev/rtc","/dev/hpet", "/dev/vfio/vfio",
9
"/dev/net/tun"
10
]
Copied!
Restart libvirtd service
1
systemctl restart libvirtd.service
Copied!
Install nova-network package
1
yum install -y openstack-nova-network
Copied!
Disable Nova Network service and restart Nova compute service
1
systemctl disable openstack-nova-network.service
2
systemctl restart openstack-nova-compute.service
Copied!
Install Midolman agent and java packages
1
yum install -y java-1.8.0-openjdk-headless midolman
Copied!
Configure midolman.conf
1
vi /etc/midolman/midolman.conf
Copied!
Add all nsdb nodes as zookeeper hosts
1
[zookeeper]
2
zookeeper_hosts = nsdb1:2181,nsdb2:2181,nsdb3:2181
Copied!
Configure each compute node with an appropiate flavor located at /etc/midolman/ folder, the have different hardware resources configured, use the one that better match your compute host capabilities
1
mn-conf template-set -h local -t agent-compute-medium
2
cp /etc/midolman/midolman-env.sh.compute.medium /etc/midolman/midolman-env.sh
Copied!
Configure metadata, issue the following commands only once, it will automatically populate the configuration to all midonet agents
1
echo "agent.openstack.metadata.nova_metadata_url : \"http://controller:8775\"" | mn-conf set -t default
2
echo "agent.openstack.metadata.shared_secret : 2bfeb930a90d435d" | mn-conf set -t default
3
echo "agent.openstack.metadata.enabled : true" | mn-conf set -t default
Copied!
Allow metadata trafic at iptables
1
iptables -I INPUT 1 -i metadata -j ACCEPT
Copied!
Remove br-tun bridge
1
ovs-vsctl del-br br-tun
Copied!
Start and enable midolman agent service
1
systemctl enable midolman.service
2
systemctl start midolman.service
Copied!
Gateway nodes configuration
Disable SElinux
1
setenforce 0
2
sed -i 's/SELINUX=enforcing/SELINUX=permissive/g' /etc/sysconfig/selinux
Copied!
Install OpenStack Mitaka release repository
1
sudo yum install -y centos-release-openstack-mitaka
Copied!
Add Midonet repository
1
cat <<EOF>/etc/yum.repos.d/midonet.repo
2
[midonet]
3
name=MidoNet
4
baseurl=http://builds.midonet.org/midonet-5.2/stable/el7/
5
enabled=1
6
gpgcheck=1
7
gpgkey=https://builds.midonet.org/midorepo.key
8
​
9
[midonet-openstack-integration]
10
name=MidoNet OpenStack Integration
11
baseurl=http://builds.midonet.org/openstack-mitaka/stable/el7/
12
enabled=1
13
gpgcheck=1
14
gpgkey=https://builds.midonet.org/midorepo.key
15
​
16
[midonet-misc]
17
name=MidoNet 3rd Party Tools and Libraries
18
baseurl=http://builds.midonet.org/misc/stable/el7/
19
enabled=1
20
gpgcheck=1
21
gpgkey=https://builds.midonet.org/midorepo.key
22
EOF
Copied!
Clean repos cache and update the system
1
yum clean all
2
yum update
Copied!
Install Midolman agent and java packages
1
yum install -y java-1.8.0-openjdk-headless midolman
Copied!
Configure midolman.conf
1
vi /etc/midolman/midolman.conf
Copied!
Add all nsdb nodes as zookeeper hosts
1
[zookeeper]
2
zookeeper_hosts = nsdb1:2181,nsdb2:2181,nsdb3:2181
Copied!
Configure each gateway node with an appropiate flavor located at /etc/midolman/ folder, the have different hardware resources configured, use the one that better match your gateway host capabilities
1
mn-conf template-set -h local -t agent-gateway-medium
2
cp /etc/midolman/midolman-env.sh.gateway.medium /etc/midolman/midolman-env.sh
Copied!
Grab the metadata shared secret located at nova.conf at any of your nova nodes
1
# egrep ^metadata_proxy_shared_secret /etc/nova/nova.conf
2
metadata_proxy_shared_secret =2bfeb930a90d435d
Copied!
Allow metadata trafic at iptables
1
iptables -I INPUT 1 -i metadata -j ACCEPT
Copied!
Start and enable midolman agent service
1
systemctl enable midolman.service
2
systemctl start midolman.service
Copied!
| Configure encapsulation and register nodes | Enter to midonet CLI from a controller node
1
midonet-cli
Copied!
Create the tunnel zone with VXLAN encapsulation
1
midonet> tunnel-zone create name tz type vxlan
2
tzone0
3
midonet> list tunnel-zone
4
tzone tzone0 name tz type vxlan
Copied!
List hosts discovered by midonet, should be all the nodes where you configured midonet agents(midolman)
1
midonet> list host
2
host host0 name gateway2 alive true addresses fe80:0:0:0:0:11ff:fe00:1102,169.254.123.1,fe80:0:0:0:0:11ff:fe00:1101,127.0.0.1,0:0:0:0:0:0:0:1,192.168.200.176,fe80:0:0:0:5054:ff:fef9:b2a0,169.254.169.254,fe80:0:0:0:7874:d6ff:fe5b:dea8,192.168.100.227,fe80:0:0:0:5054:ff:fed9:9cc0,fe80:0:0:0:5054:ff:fe4a:e39b,192.168.1.86 flooding-proxy-weight 1 container-weight 1 container-limit no-limit enforce-container-limit false
3
host host1 name gateway1 alive true addresses 169.254.169.254,fe80:0:0:0:3cd1:23ff:feac:a3c2,192.168.1.87,fe80:0:0:0:5054:ff:fea8:da91,127.0.0.1,0:0:0:0:0:0:0:1,fe80:0:0:0:5054:ff:feec:92c1,192.168.200.232,fe80:0:0:0:0:11ff:fe00:1102,169.254.123.1,fe80:0:0:0:0:11ff:fe00:1101,192.168.100.141,fe80:0:0:0:5054:ff:fe20:30fb flooding-proxy-weight 1 container-weight 1 container-limit no-limit enforce-container-limit false
4
host host2 name compute1 alive true addresses fe80:0:0:0:0:11ff:fe00:1101,169.254.123.1,127.0.0.1,0:0:0:0:0:0:0:1,fe80:0:0:0:0:11ff:fe00:1102,192.168.100.173,fe80:0:0:0:5054:ff:fe06:161,fe80:0:0:0:5054:ff:fee3:eb48,192.168.200.251,fe80:0:0:0:5054:ff:fe8d:d22,192.168.1.93,169.254.169.254,fe80:0:0:0:48cb:adff:fe69:f07b flooding-proxy-weight 1 container-weight 1 container-limit no-limit enforce-container-limit false
Copied!
Register each of the nodes at the VXLAN zone we created before
1
midonet> tunnel-zone tzone0 add member host host0 address 192.168.100.227
2
zone tzone0 host host0 address 192.168.100.227
3
midonet> tunnel-zone tzone0 add member host host1 address 192.168.100.141
4
zone tzone0 host host1 address 192.168.100.141
5
midonet> tunnel-zone tzone0 add member host host2 address 192.168.100.173
6
zone tzone0 host host2 address 192.168.100.173
Copied!
| Create Networks at Neutron | Create an external network
1
# neutron net-create ext-net --router:external
2
Created a new network:
3
+-----------------------+--------------------------------------+
4
| Field | Value |
5
+-----------------------+--------------------------------------+
6
| admin_state_up | True |
7
| created_at | 2016-07-03T14:47:30 |
8
| description | |
9
| id | dc15245e-4391-4514-b489-8976373046a3 |
10
| is_default | False |
11
| name | ext-net |
12
| port_security_enabled | True |
13
| provider:network_type | midonet |
14
| router:external | True |
15
| shared | False |
16
| status | ACTIVE |
17
| subnets | |
18
| tags | |
19
| tenant_id | 2f7ee2716b3b4140be57b4a5b26401e3 |
20
| updated_at | 2016-07-03T14:47:30 |
21
+-----------------------+--------------------------------------+
Copied!
Create an external subnet in the network we created before, use you own IP ranges to match your environment
1
# neutron subnet-create ext-net 192.168.200.0/24 --name ext-subnet \
2
--allocation-pool start=192.168.200.225,end=192.168.200.240 \
3
--disable-dhcp --gateway 192.168.200.1
4
Created a new subnet:
5
+-------------------+--------------------------------------------------------+
6
| Field | Value |
7
+-------------------+--------------------------------------------------------+
8
| allocation_pools | {"start": "192.168.200.225", "end": "192.168.200.240"} |
9
| cidr | 192.168.200.0/24 |
10
| created_at | 2016-07-03T14:50:46 |
11
| description | |
12
| dns_nameservers | |
13
| enable_dhcp | False |
14
| gateway_ip | 192.168.200.1 |
15
| host_routes | |
16
| id | 234dcc9a-2878-4799-b564-bf3a1bd52cad |
17
| ip_version | 4 |
18
| ipv6_address_mode | |
19
| ipv6_ra_mode | |
20
| name | ext-subnet |
21
| network_id | dc15245e-4391-4514-b489-8976373046a3 |
22
| subnetpool_id | |
23
| tenant_id | 2f7ee2716b3b4140be57b4a5b26401e3 |
24
| updated_at | 2016-07-03T14:50:46 |
25
+-------------------+--------------------------------------------------------+
Copied!
Create a tenant network and a subnet on it
1
# neutron net-create demo-net
2
Created a new network:
3
+-----------------------+--------------------------------------+
4
| Field | Value |
5
+-----------------------+--------------------------------------+
6
| admin_state_up | True |
7
| created_at | 2016-07-03T14:51:39 |
8
| description | |
9
| id | 075ba699-dc4c-4625-8e0d-0a258a9aeb7d |
10
| name | demo-net |
11
| port_security_enabled | True |
12
| provider:network_type | midonet |
13
| router:external | False |
14
| shared | False |
15
| status | ACTIVE |
16
| subnets | |
17
| tags | |
18
| tenant_id | 2f7ee2716b3b4140be57b4a5b26401e3 |
19
| updated_at | 2016-07-03T14:51:39 |
20
+-----------------------+--------------------------------------+
21
# neutron subnet-create demo-net 10.0.20.0/24 --name demo-subnet
22
Created a new subnet:
23
+-------------------+----------------------------------------------+
24
| Field | Value |
25
+-------------------+----------------------------------------------+
26
| allocation_pools | {"start": "10.0.20.2", "end": "10.0.20.254"} |
27
| cidr | 10.0.20.0/24 |
28
| created_at | 2016-07-03T14:52:32 |
29
| description | |
30
| dns_nameservers | |
31
| enable_dhcp | True |
32
| gateway_ip | 10.0.20.1 |
33
| host_routes | |
34
| id | b299d899-33a3-4bfa-aff4-fda071545bdf |
35
| ip_version | 4 |
36
| ipv6_address_mode | |
37
| ipv6_ra_mode | |
38
| name | demo-subnet |
39
| network_id | 075ba699-dc4c-4625-8e0d-0a258a9aeb7d |
40
| subnetpool_id | |
41
| tenant_id | 2f7ee2716b3b4140be57b4a5b26401e3 |
42
| updated_at | 2016-07-03T14:52:32 |
43
+-------------------+----------------------------------------------+
Copied!
Create a tenant router
1
# neutron router-create router1
2
Created a new router:
3
+-----------------------+--------------------------------------+
4
| Field | Value |
5
+-----------------------+--------------------------------------+
6
| admin_state_up | True |
7
| description | |
8
| external_gateway_info | |
9
| id | 258942d8-9d82-4ebd-b829-c7bdfcc973f5 |
10
| name | router1 |
11
| routes | |
12
| status | ACTIVE |
13
| tenant_id | 2f7ee2716b3b4140be57b4a5b26401e3 |
14
+-----------------------+--------------------------------------+
Copied!
Attach the tenant subnet interface we created before to the router
1
# neutron router-interface-add router1 demo-subnet
2
Added interface 06c85a56-368c-4d79-bbf0-4bb077f163e5 to router router1.
Copied!
Set the external network as router gateway
1
# neutron router-gateway-set router1 ext-net
2
Set gateway for router router1
Copied!
Now, you can create an instance at tenant network
1
# nova boot --flavor m1.tiny --image 80871834-29dd-4100-b038-f5f83f126204 --nic net-id=075ba699-dc4c-4625-8e0d-0a258a9aeb7d test1
2
+--------------------------------------+-----------------------------------------------------+
3
| Property | Value |
4
+--------------------------------------+-----------------------------------------------------+
5
| OS-DCF:diskConfig | MANUAL |
6
| OS-EXT-AZ:availability_zone | |
7
| OS-EXT-SRV-ATTR:host | - |
8
| OS-EXT-SRV-ATTR:hypervisor_hostname | - |
9
| OS-EXT-SRV-ATTR:instance_name | instance-0000000a |
10
| OS-EXT-STS:power_state | 0 |
11
| OS-EXT-STS:task_state | scheduling |
12
| OS-EXT-STS:vm_state | building |
13
| OS-SRV-USG:launched_at | - |
14
| OS-SRV-USG:terminated_at | - |
15
| accessIPv4 | |
16
| accessIPv6 | |
17
| adminPass | q2Cq4kxePSLL |
18
| config_drive | |
19
| created | 2016-07-03T15:46:19Z |
20
| flavor | m1.tiny (1) |
21
| hostId | |
22
| id | b8aa46f9-186c-4594-8428-f8dbb16a5e16 |
23
| image | cirros image (80871834-29dd-4100-b038-f5f83f126204) |
24
| key_name | - |
25
| metadata | {} |
26
| name | test1 |
27
| os-extended-volumes:volumes_attached | [] |
28
| progress | 0 |
29
| security_groups | default |
30
| status | BUILD |
31
| tenant_id | 2f7ee2716b3b4140be57b4a5b26401e3 |
32
| updated | 2016-07-03T15:46:20Z |
33
| user_id | a2482a91a1f14750b372445d28b07c75 |
34
+--------------------------------------+-----------------------------------------------------+
35
# nova list
36
+--------------------------------------+-------+--------+------------+-------------+---------------------+
37
| ID | Name | Status | Task State | Power State | Networks |
38
+--------------------------------------+-------+--------+------------+-------------+---------------------+
39
| b8aa46f9-186c-4594-8428-f8dbb16a5e16 | test1 | ACTIVE | - | Running | demo-net=10.0.20.11 |
40
+--------------------------------------+-------+--------+------------+-------------+---------------------+
Copied!
Ensure the instance gets IP and the metadata service is properly running
1
# nova console-log test1
2
...#Snipp from the output
3
Sending discover...
4
Sending select for 10.0.20.11...
5
Lease of 10.0.20.11 obtained, lease time 86400
6
cirros-ds 'net' up at 7.92
7
checking http://169.254.169.254/2009-04-04/instance-id
8
successful after 1/20 tries: up 8.22. iid=i-0000000a
9
...
Copied!
If you login to the instance through VNC you should be able to ping another instances
| Edge router configuration | Create a new router
1
# neutron router-create edge-router
2
Created a new router:
3
+-----------------------+--------------------------------------+
4
| Field | Value |
5
+-----------------------+--------------------------------------+
6
| admin_state_up | True |
7
| description | |
8
| external_gateway_info | |
9
| id | 5ecadb64-cb0d-4f95-a00e-aa1dd20a2012 |
10
| name | edge-router |
11
| routes | |
12
| status | ACTIVE |
13
| tenant_id | 2f7ee2716b3b4140be57b4a5b26401e3 |
14
+-----------------------+--------------------------------------+
Copied!
Attach the external subnet interface to the router
1
# neutron router-interface-add edge-router ext-subnet
2
Added interface e37f1986-c6b1-47f4-8268-02b837ceac17 to router edge-router.
Copied!
Create an uplink network
1
# neutron net-create uplink-network --tenant_id admin --provider:network_type uplink
2
Created a new network:
3
+-----------------------+--------------------------------------+
4
| Field | Value |
5
+-----------------------+--------------------------------------+
6
| admin_state_up | True |
7
| created_at | 2016-07-03T14:57:15 |
8
| description | |
9
| id | 77173ed4-6106-4515-af1c-3683897955f9 |
10
| name | uplink-network |
11
| port_security_enabled | True |
12
| provider:network_type | uplink |
13
| router:external | False |
14
| shared | False |
15
| status | ACTIVE |
16
| subnets | |
17
| tags | |
18
| tenant_id | admin |
19
| updated_at | 2016-07-03T14:57:15 |
20
+-----------------------+--------------------------------------+
Copied!
Create a subnet in the uplink network
1
# neutron subnet-create --tenant_id admin --disable-dhcp --name uplink-subnet uplink-network 192.168.1.0/24
2
Created a new subnet:
3
+-------------------+--------------------------------------------------+
4
| Field | Value |
5
+-------------------+--------------------------------------------------+
6
| allocation_pools | {"start": "192.168.1.2", "end": "192.168.1.254"} |
7
| cidr | 192.168.1.0/24 |
8
| created_at | 2016-07-03T15:06:28 |
9
| description | |
10
| dns_nameservers | |
11
| enable_dhcp | False |
12
| gateway_ip | 192.168.1.1 |
13
| host_routes | |
14
| id | 4e98e789-20d3-45fd-a3b5-9bcf02d8a832 |
15
| ip_version | 4 |
16
| ipv6_address_mode | |
17
| ipv6_ra_mode | |
18
| name | uplink-subnet |
19
| network_id | 77173ed4-6106-4515-af1c-3683897955f9 |
20
| subnetpool_id | |
21
| tenant_id | admin |
22
| updated_at | 2016-07-03T15:06:28 |
23
+-------------------+--------------------------------------------------+
Copied!
Create a port for each of the gateway nodes, interface should match with the NIC you want to use for binding the gateway nodes and a IP address for the same purposes
1
# neutron port-create uplink-network --binding:host_id gateway1 --binding:profile type=dict interface_name=eth1 --fixed-ip ip_address=192.168.1.199
2
Created a new port:
3
+-----------------------+--------------------------------------------------------------------------------------+
4
| Field | Value |
5
+-----------------------+--------------------------------------------------------------------------------------+
6
| admin_state_up | True |
7
| allowed_address_pairs | |
8
| binding:host_id | compute1 |
9
| binding:profile | {"interface_name": "eth1"} |
10
| binding:vif_details | {"port_filter": true} |
11
| binding:vif_type | midonet |
12
| binding:vnic_type | normal |
13
| created_at | 2016-07-03T15:10:06 |
14
| description | |
15
| device_id | |
16
| device_owner | |
17
| extra_dhcp_opts | |
18
| fixed_ips | {"subnet_id": "4e98e789-20d3-45fd-a3b5-9bcf02d8a832", "ip_address": "192.168.1.199"} |
19
| id | 7b4f54dd-2b41-42ba-9c5c-cda4640dc550 |
20
| mac_address | fa:16:3e:44:a8:c9 |
21
| name | |
22
| network_id | 77173ed4-6106-4515-af1c-3683897955f9 |
23
| port_security_enabled | True |
24
| security_groups | 0cf3e33e-dbd6-4b42-a0bd-6679b5eed4e1 |
25
| status | ACTIVE |
26
| tenant_id | 2f7ee2716b3b4140be57b4a5b26401e3 |
27
| updated_at | 2016-07-03T15:10:06 |
28
+-----------------------+--------------------------------------------------------------------------------------+
Copied!
Attach each of the ports to the edge router
1
# neutron router-interface-add edge-router port=7b4f54dd-2b41-42ba-9c5c-cda4640dc550
2
Added interface 7b4f54dd-2b41-42ba-9c5c-cda4640dc550 to router edge-router.
Copied!
| At this point you have to decide if use border routers with BGP enabled or static routes. | Use one of the following links to configure your use case: | https://docs.midonet.org/docs/latest/operations-guide/content/bgp_uplink_configuration.html | https://docs.midonet.org/docs/latest/operations-guide/content/static_setup.html​
Issues I faced during configuration of Midonet
| Midolman agent don\'t start: | It was caused because midolman-env.sh file has more RAM configured as the one of my server. | Edit the file to match your server resources
1
# egrep ^MAX_HEAP_SIZE /etc/midolman/midolman-env.sh
2
MAX_HEAP_SIZE="2048M"
Copied!
Instances doesn\'t boot with the following error:
1
could not open /dev/net/tun: Permission denied
Copied!
I had to remove br-tun bridges at ovs, if not, ovs locks the device and midolman cannot create the tunnel beetwen compute nodes and gateway nodes.
1
ovs-vsctl del-br br-tun
Copied!
| This post is my experience integrating Midonet into OpenStack, maybe some things are not correct, if you find any issue, please advise me to fix it. | Regards, Eduardo Gonzalez
Previous
OpenStack keystone zero downtime upgrade process newton to ocata
Next
OpenStack kolla deployment
Last modified 1yr ago
Copy link