Midonet integration with OpenStack Mitaka
MidoNet is an Open Source network virtualization software for IaaS infrastructure. | It decouples your IaaS cloud from your network hardware, creating an intelligent software abstraction layer between your end hosts and your physical network. | This network abstraction layer allows the cloud operator to move what has traditionally been hardware-based network appliances into a software-based multi-tenant virtual domain.
This definition from MidoNet documentation explains what MidoNet is and what MidoNet does.
At this I will post cover my experiences integrating MidoNet with OpenStack. | I used the following configurations:
All servers have CentOS 7.2 installed
OpenStack has been previously installed from RDO packages with multinode Packstack
x3 NSDB nodes (Casandra and Zookeeper services)
x2 Gateway Nodes (Midolman Agent)
x1 OpenStack Controller (MidoNet Cluster)
x1 OpenStack compute node (Midolman Agent)
NSDB NODES
Disable SElinux
setenforce 0
sed -i 's/SELINUX=enforcing/SELINUX=permissive/g' /etc/sysconfig/selinuxInstall OpenStack Mitaka release repository
sudo yum install -y centos-release-openstack-mitakaAdd Cassandra repository
cat <<EOF>/etc/yum.repos.d/datastax.repo
[datastax]
name = DataStax Repo for Apache Cassandra
baseurl = http://rpm.datastax.com/community
enabled = 1
gpgcheck = 1
gpgkey = https://rpm.datastax.com/rpm/repo_key
EOFAdd Midonet repository
cat <<EOF>/etc/yum.repos.d/midonet.repo
[midonet]
name=MidoNet
baseurl=http://builds.midonet.org/midonet-5.2/stable/el7/
enabled=1
gpgcheck=1
gpgkey=https://builds.midonet.org/midorepo.key
[midonet-openstack-integration]
name=MidoNet OpenStack Integration
baseurl=http://builds.midonet.org/openstack-mitaka/stable/el7/
enabled=1
gpgcheck=1
gpgkey=https://builds.midonet.org/midorepo.key
[midonet-misc]
name=MidoNet 3rd Party Tools and Libraries
baseurl=http://builds.midonet.org/misc/stable/el7/
enabled=1
gpgcheck=1
gpgkey=https://builds.midonet.org/midorepo.key
EOFClean repo cache and update packages
yum clean all
yum update| Zookeeper Configuration | Install Zookeeper, java and dependencies
yum install -y java-1.7.0-openjdk-headless zookeeper zkdump nmap-ncatEdit zookeeper configuration file
vi /etc/zookeeper/zoo.cfgAdd all NSDB nodes at the configuration file
server.1=nsdb1:2888:3888
server.2=nsdb2:2888:3888
server.3=nsdb3:2888:3888
autopurge.snapRetainCount=10
autopurge.purgeInterval =12Create zookeeper folder on which zookeeper will store data, change the owner to zookeeper user
mkdir /var/lib/zookeeper/data
chown zookeeper:zookeeper /var/lib/zookeeper/dataCreate myid file at zookeeper data folder, the ID should match with the NSDB node number, insert that number as follows:
#NSDB1
echo 1 > /var/lib/zookeeper/data/myid
#NSDB2
echo 2 > /var/lib/zookeeper/data/myid
#NSDB3
echo 3 > /var/lib/zookeeper/data/myidCreate java folder and create a softlink to it
mkdir -p /usr/java/default/bin/
ln -s /usr/lib/jvm/jre-1.7.0-openjdk/bin/java /usr/java/default/bin/javaStart and enable Zookeeper service
systemctl enable zookeeper.service
systemctl start zookeeper.serviceTest if zookeeper is working locally
echo ruok | nc 127.0.0.1 2181
imokTest if zookeeper is working at NSDB remote nodes
echo stat | nc nsdb3 2181
Zookeeper version: 3.4.5--1, built on 02/08/2013 12:25 GMT
Clients:
/192.168.100.172:35306[0](queued=0,recved=1,sent=0)
Latency min/avg/max: 0/0/0
Received: 1
Sent: 0
Connections: 1
Outstanding: 0
Zxid: 0x100000000
Mode: follower
Node count: 4| Cassandra configuration | Install Java and Cassandra dependencies
yum install -y java-1.8.0-openjdk-headless dsc22Edit cassandra yaml file
vi /etc/cassandra/conf/cassandra.yaml| Change cluster_name to midonet | Configure seed_provider seeds to match all NSDB nodes | Configure listen_address and rpc_address to match the hostname of the self node
cluster_name: 'midonet'
....
seed_provider:
- class_name: org.apache.cassandra.locator.SimpleSeedProvider
parameters:
- seeds: "nsdb1,nsdb2,nsdb3"
listen_address: nsdb1
rpc_address: nsdb1Edit cassandra\'s init script in order to fix a bug in the init script
vi /etc/init.d/cassandraAdd the next two lines after #Casandra startup
case "$1" in
start)
# Cassandra startup
echo -n "Starting Cassandra: "
mkdir -p /var/run/cassandra #Add this line
chown cassandra:cassandra /var/run/cassandra #Add this line
su $CASSANDRA_OWNR -c "$CASSANDRA_PROG -p $pid_file" > $log_file 2>&1
retval=$?
[ $retval -eq 0 ] && touch $lock_file
echo "OK"
;;Start and enable Cassandra service
systemctl enable cassandra.service
systemctl start cassandra.serviceCheck if all NSDB nodes join the cluster
nodetool --host 127.0.0.1 status
Datacenter: datacenter1
=======================
Status=Up/Down
|/ State=Normal/Leaving/Joining/Moving
-- Address Load Tokens Owns (effective) Host ID Rack
UN 192.168.100.172 89.1 KB 256 70.8% 3f1ecedd-8caf-4938-84ad-8614d2134557 rack1
UN 192.168.100.224 67.64 KB 256 60.7% cb36c999-a6e1-4d98-a4dd-d4230b41df08 rack1
UN 192.168.100.195 25.78 KB 256 68.6% 4758bae8-9300-4e57-9a61-5b1107082964 rack1Configure OpenStack Controller Nodes (On which Neutron Server is running)
Disable SElinux
setenforce 0
sed -i 's/SELINUX=enforcing/SELINUX=permissive/g' /etc/sysconfig/selinuxInstall OpenStack Mitaka release repository
sudo yum install -y centos-release-openstack-mitakaAdd Midonet Repository
cat <<EOF>/etc/yum.repos.d/midonet.repo
[midonet]
name=MidoNet
baseurl=http://builds.midonet.org/midonet-5.2/stable/el7/
enabled=1
gpgcheck=1
gpgkey=https://builds.midonet.org/midorepo.key
[midonet-openstack-integration]
name=MidoNet OpenStack Integration
baseurl=http://builds.midonet.org/openstack-mitaka/stable/el7/
enabled=1
gpgcheck=1
gpgkey=https://builds.midonet.org/midorepo.key
[midonet-misc]
name=MidoNet 3rd Party Tools and Libraries
baseurl=http://builds.midonet.org/misc/stable/el7/
enabled=1
gpgcheck=1
gpgkey=https://builds.midonet.org/midorepo.key
EOFClean repos cache and update the system
yum clean all
yum updateCreate an OpenStack user for MidoNet, change the password to match your own
# openstack user create --password temporal midonet
+----------+----------------------------------+
| Field | Value |
+----------+----------------------------------+
| email | None |
| enabled | True |
| id | ac25c5a77e7c4e4598ccadea89e09969 |
| name | midonet |
| username | midonet |
+----------+----------------------------------+Add admin role at tenant services to Midonet user
# openstack role add --project services --user midonet admin
+-----------+----------------------------------+
| Field | Value |
+-----------+----------------------------------+
| domain_id | None |
| id | bca2c6e1f3da42b0ba82aee401398a8a |
| name | admin |
+-----------+----------------------------------+Create MidoNet service at Keystone
# openstack service create --name midonet --description "MidoNet API Service" midonet
+-------------+----------------------------------+
| Field | Value |
+-------------+----------------------------------+
| description | MidoNet API Service |
| enabled | True |
| id | 499059c4a3a040cfb632411408a2be4c |
| name | midonet |
| type | midonet |
+-------------+----------------------------------+| Clean up neutron server | Stop neutron services
openstack-service stop neutronRemove neutron database and recreate it again
mysql -u root -p
DROP DATABASE neutron;
Query OK, 157 rows affected (11.50 sec)
MariaDB [(none)]> CREATE DATABASE neutron;
Query OK, 1 row affected (0.00 sec)
MariaDB [(none)]> GRANT ALL PRIVILEGES ON neutron.* TO 'neutron'@'localhost' IDENTIFIED BY 'ab4f81b1040a495e';
Query OK, 0 rows affected (0.00 sec)
MariaDB [(none)]> GRANT ALL PRIVILEGES ON neutron.* TO 'neutron'@'%' IDENTIFIED BY 'ab4f81b1040a495e';
Query OK, 0 rows affected (0.00 sec)
MariaDB [(none)]> exit
ByeRemove plugin.ini symbolic link to ml2_conf.ini
#rm /etc/neutron/plugin.ini
rm: remove symbolic link ‘/etc/neutron/plugin.ini’? yRemove br-tun tunnel used by neutron in all the nodes
ovs-vsctl del-br br-tunInstall MidoNet packages and remove ml2 package
yum install -y openstack-neutron python-networking-midonet python-neutronclient
yum remove openstack-neutron-ml2Make a backup of neutron configuration file
cp /etc/neutron/neutron.conf neutron.conf.bakEdit neutron configuration file
vi /etc/neutron/neutron.confMost of the options are already configured by our older neutron configuration, change the ones who apply to match this configuration
[DEFAULT]
core_plugin = midonet.neutron.plugin_v2.MidonetPluginV2
service_plugins = midonet.neutron.services.l3.l3_midonet.MidonetL3ServicePlugin
dhcp_agent_notification = False
allow_overlapping_ips = True
rpc_backend = rabbit
auth_strategy = keystone
notify_nova_on_port_status_changes = true
notify_nova_on_port_data_changes = true
nova_url = http://controller:8774/v2
[database]
connection = mysql+pymysql://neutron:ab4f81b1040a495e@controller/neutron
[oslo_messaging_rabbit]
rabbit_host = controller
rabbit_userid = guest
rabbit_password = guest
[keystone_authtoken]
auth_uri = http://controller:5000/v2.0
admin_user=neutron
admin_tenant_name=services
identity_uri=http://controller:35357
admin_password=d88f0bd060d64c33
[nova]
region_name = RegionOne
auth_url = http://controller:35357
auth_type = password
password = 9ca36d15e4824d93
project_domain_id = default
project_name = services
tenant_name = services
user_domain_id = default
username = nova
[oslo_concurrency]
lock_path = /var/lib/neutron/tmpAt my deployment these are the options I had to change to configure midonet
diff /etc/neutron/neutron.conf neutron.conf.bak
33c33
< core_plugin = midonet.neutron.plugin_v2.MidonetPluginV2
---
> core_plugin = neutron.plugins.ml2.plugin.Ml2Plugin
37c37
< service_plugins = midonet.neutron.services.l3.l3_midonet.MidonetL3ServicePlugin
---
> service_plugins =router
120c120
< dhcp_agent_notification = False
---
> #dhcp_agent_notification = true
1087c1087,1088
< lock_path = /var/lib/neutron/tmp
---
> lock_path = $state_path/lock
> Create midonet plugins folder
mkdir /etc/neutron/plugins/midonetCreate a file called midonet.ini
vi /etc/neutron/plugins/midonet/midonet.iniConfigure midonet.ini file to match your own configuration options
[MIDONET]
# MidoNet API URL
midonet_uri = http://controller:8181/midonet-api
# MidoNet administrative user in Keystone
username = midonet
password = temporal
# MidoNet administrative user's tenant
project_id = servicesCreate a symbolic link from midonet.ini to plugin.ini
ln -s /etc/neutron/plugins/midonet/midonet.ini /etc/neutron/plugin.iniSync and populate database tables with Midonet plugin
su -s /bin/sh -c "neutron-db-manage --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/midonet/midonet.ini upgrade head" neutron
su -s /bin/sh -c "neutron-db-manage --subproject networking-midonet upgrade head" neutronRestart nova api and neutron server services
systemctl restart openstack-nova-api.service
systemctl restart neutron-serverInstall midonet cluster package
yum install -y midonet-clusterConfigure midonet.conf file
vi /etc/midonet/midonet.confAdd all NSDB nodes at zookeeper_hosts
[zookeeper]
zookeeper_hosts = nsdb1:2181,nsdb2:2181,nsdb3:2181Configure midonet to make use of NSDB nodes as Zookeeper and cassandra hosts
cat << EOF | mn-conf set -t default
zookeeper {
zookeeper_hosts = "nsdb1:2181,nsdb2:2181,nsdb3:2181"
}
cassandra {
servers = "nsdb1,nsdb2,nsdb3"
}
EOFSet cassandra replication factor to 3
echo "cassandra.replication_factor : 3" | mn-conf set -t defaultGrab your admin token
#egrep ^admin_token /etc/keystone/keystone.conf
admin_token = 7b84d89b32c34b71a697eb1a270807abConfigure Midonet to auth with keystone
cat << EOF | mn-conf set -t default
cluster.auth {
provider_class = "org.midonet.cluster.auth.keystone.KeystoneService"
admin_role = "admin"
keystone.tenant_name = "admin"
keystone.admin_token = "7b84d89b32c34b71a697eb1a270807ab"
keystone.host = controller
keystone.port = 35357
}
EOFStart and enable midonet cluster service
systemctl enable midonet-cluster.service
systemctl start midonet-cluster.serviceInstall midonet CLI
yum install -y python-midonetclientCreate a file at you home directory with midonet auth info
#vi ~/.midonetrc
[cli]
api_url = http://controller:8181/midonet-api
username = admin
password = temporal
project_id = adminConfigure Compute nodes
Disable SElinux
setenforce 0
sed -i 's/SELINUX=enforcing/SELINUX=permissive/g' /etc/sysconfig/selinuxInstall OpenStack Mitaka release repository
sudo yum install -y centos-release-openstack-mitakaAdd Midonet repository
cat <<EOF>/etc/yum.repos.d/midonet.repo
[midonet]
name=MidoNet
baseurl=http://builds.midonet.org/midonet-5.2/stable/el7/
enabled=1
gpgcheck=1
gpgkey=https://builds.midonet.org/midorepo.key
[midonet-openstack-integration]
name=MidoNet OpenStack Integration
baseurl=http://builds.midonet.org/openstack-mitaka/stable/el7/
enabled=1
gpgcheck=1
gpgkey=https://builds.midonet.org/midorepo.key
[midonet-misc]
name=MidoNet 3rd Party Tools and Libraries
baseurl=http://builds.midonet.org/misc/stable/el7/
enabled=1
gpgcheck=1
gpgkey=https://builds.midonet.org/midorepo.key
EOFClean repos cache and update the system
yum clean all
yum updateEdit qemu.conf
vi /etc/libvirt/qemu.confConfigure with the following options, by default all these options are commented, you can paste it all wherever you want
user = "root"
group = "root"
cgroup_device_acl = [
"/dev/null", "/dev/full", "/dev/zero",
"/dev/random", "/dev/urandom",
"/dev/ptmx", "/dev/kvm", "/dev/kqemu",
"/dev/rtc","/dev/hpet", "/dev/vfio/vfio",
"/dev/net/tun"
]Restart libvirtd service
systemctl restart libvirtd.serviceInstall nova-network package
yum install -y openstack-nova-networkDisable Nova Network service and restart Nova compute service
systemctl disable openstack-nova-network.service
systemctl restart openstack-nova-compute.serviceInstall Midolman agent and java packages
yum install -y java-1.8.0-openjdk-headless midolmanConfigure midolman.conf
vi /etc/midolman/midolman.confAdd all nsdb nodes as zookeeper hosts
[zookeeper]
zookeeper_hosts = nsdb1:2181,nsdb2:2181,nsdb3:2181Configure each compute node with an appropiate flavor located at /etc/midolman/ folder, the have different hardware resources configured, use the one that better match your compute host capabilities
mn-conf template-set -h local -t agent-compute-medium
cp /etc/midolman/midolman-env.sh.compute.medium /etc/midolman/midolman-env.shConfigure metadata, issue the following commands only once, it will automatically populate the configuration to all midonet agents
echo "agent.openstack.metadata.nova_metadata_url : \"http://controller:8775\"" | mn-conf set -t default
echo "agent.openstack.metadata.shared_secret : 2bfeb930a90d435d" | mn-conf set -t default
echo "agent.openstack.metadata.enabled : true" | mn-conf set -t defaultAllow metadata trafic at iptables
iptables -I INPUT 1 -i metadata -j ACCEPTRemove br-tun bridge
ovs-vsctl del-br br-tunStart and enable midolman agent service
systemctl enable midolman.service
systemctl start midolman.serviceGateway nodes configuration
Disable SElinux
setenforce 0
sed -i 's/SELINUX=enforcing/SELINUX=permissive/g' /etc/sysconfig/selinuxInstall OpenStack Mitaka release repository
sudo yum install -y centos-release-openstack-mitakaAdd Midonet repository
cat <<EOF>/etc/yum.repos.d/midonet.repo
[midonet]
name=MidoNet
baseurl=http://builds.midonet.org/midonet-5.2/stable/el7/
enabled=1
gpgcheck=1
gpgkey=https://builds.midonet.org/midorepo.key
[midonet-openstack-integration]
name=MidoNet OpenStack Integration
baseurl=http://builds.midonet.org/openstack-mitaka/stable/el7/
enabled=1
gpgcheck=1
gpgkey=https://builds.midonet.org/midorepo.key
[midonet-misc]
name=MidoNet 3rd Party Tools and Libraries
baseurl=http://builds.midonet.org/misc/stable/el7/
enabled=1
gpgcheck=1
gpgkey=https://builds.midonet.org/midorepo.key
EOFClean repos cache and update the system
yum clean all
yum updateInstall Midolman agent and java packages
yum install -y java-1.8.0-openjdk-headless midolmanConfigure midolman.conf
vi /etc/midolman/midolman.confAdd all nsdb nodes as zookeeper hosts
[zookeeper]
zookeeper_hosts = nsdb1:2181,nsdb2:2181,nsdb3:2181Configure each gateway node with an appropiate flavor located at /etc/midolman/ folder, the have different hardware resources configured, use the one that better match your gateway host capabilities
mn-conf template-set -h local -t agent-gateway-medium
cp /etc/midolman/midolman-env.sh.gateway.medium /etc/midolman/midolman-env.shGrab the metadata shared secret located at nova.conf at any of your nova nodes
# egrep ^metadata_proxy_shared_secret /etc/nova/nova.conf
metadata_proxy_shared_secret =2bfeb930a90d435dAllow metadata trafic at iptables
iptables -I INPUT 1 -i metadata -j ACCEPTStart and enable midolman agent service
systemctl enable midolman.service
systemctl start midolman.service| Configure encapsulation and register nodes | Enter to midonet CLI from a controller node
midonet-cliCreate the tunnel zone with VXLAN encapsulation
midonet> tunnel-zone create name tz type vxlan
tzone0
midonet> list tunnel-zone
tzone tzone0 name tz type vxlanList hosts discovered by midonet, should be all the nodes where you configured midonet agents(midolman)
midonet> list host
host host0 name gateway2 alive true addresses fe80:0:0:0:0:11ff:fe00:1102,169.254.123.1,fe80:0:0:0:0:11ff:fe00:1101,127.0.0.1,0:0:0:0:0:0:0:1,192.168.200.176,fe80:0:0:0:5054:ff:fef9:b2a0,169.254.169.254,fe80:0:0:0:7874:d6ff:fe5b:dea8,192.168.100.227,fe80:0:0:0:5054:ff:fed9:9cc0,fe80:0:0:0:5054:ff:fe4a:e39b,192.168.1.86 flooding-proxy-weight 1 container-weight 1 container-limit no-limit enforce-container-limit false
host host1 name gateway1 alive true addresses 169.254.169.254,fe80:0:0:0:3cd1:23ff:feac:a3c2,192.168.1.87,fe80:0:0:0:5054:ff:fea8:da91,127.0.0.1,0:0:0:0:0:0:0:1,fe80:0:0:0:5054:ff:feec:92c1,192.168.200.232,fe80:0:0:0:0:11ff:fe00:1102,169.254.123.1,fe80:0:0:0:0:11ff:fe00:1101,192.168.100.141,fe80:0:0:0:5054:ff:fe20:30fb flooding-proxy-weight 1 container-weight 1 container-limit no-limit enforce-container-limit false
host host2 name compute1 alive true addresses fe80:0:0:0:0:11ff:fe00:1101,169.254.123.1,127.0.0.1,0:0:0:0:0:0:0:1,fe80:0:0:0:0:11ff:fe00:1102,192.168.100.173,fe80:0:0:0:5054:ff:fe06:161,fe80:0:0:0:5054:ff:fee3:eb48,192.168.200.251,fe80:0:0:0:5054:ff:fe8d:d22,192.168.1.93,169.254.169.254,fe80:0:0:0:48cb:adff:fe69:f07b flooding-proxy-weight 1 container-weight 1 container-limit no-limit enforce-container-limit falseRegister each of the nodes at the VXLAN zone we created before
midonet> tunnel-zone tzone0 add member host host0 address 192.168.100.227
zone tzone0 host host0 address 192.168.100.227
midonet> tunnel-zone tzone0 add member host host1 address 192.168.100.141
zone tzone0 host host1 address 192.168.100.141
midonet> tunnel-zone tzone0 add member host host2 address 192.168.100.173
zone tzone0 host host2 address 192.168.100.173| Create Networks at Neutron | Create an external network
# neutron net-create ext-net --router:external
Created a new network:
+-----------------------+--------------------------------------+
| Field | Value |
+-----------------------+--------------------------------------+
| admin_state_up | True |
| created_at | 2016-07-03T14:47:30 |
| description | |
| id | dc15245e-4391-4514-b489-8976373046a3 |
| is_default | False |
| name | ext-net |
| port_security_enabled | True |
| provider:network_type | midonet |
| router:external | True |
| shared | False |
| status | ACTIVE |
| subnets | |
| tags | |
| tenant_id | 2f7ee2716b3b4140be57b4a5b26401e3 |
| updated_at | 2016-07-03T14:47:30 |
+-----------------------+--------------------------------------+Create an external subnet in the network we created before, use you own IP ranges to match your environment
# neutron subnet-create ext-net 192.168.200.0/24 --name ext-subnet \
--allocation-pool start=192.168.200.225,end=192.168.200.240 \
--disable-dhcp --gateway 192.168.200.1
Created a new subnet:
+-------------------+--------------------------------------------------------+
| Field | Value |
+-------------------+--------------------------------------------------------+
| allocation_pools | {"start": "192.168.200.225", "end": "192.168.200.240"} |
| cidr | 192.168.200.0/24 |
| created_at | 2016-07-03T14:50:46 |
| description | |
| dns_nameservers | |
| enable_dhcp | False |
| gateway_ip | 192.168.200.1 |
| host_routes | |
| id | 234dcc9a-2878-4799-b564-bf3a1bd52cad |
| ip_version | 4 |
| ipv6_address_mode | |
| ipv6_ra_mode | |
| name | ext-subnet |
| network_id | dc15245e-4391-4514-b489-8976373046a3 |
| subnetpool_id | |
| tenant_id | 2f7ee2716b3b4140be57b4a5b26401e3 |
| updated_at | 2016-07-03T14:50:46 |
+-------------------+--------------------------------------------------------+Create a tenant network and a subnet on it
# neutron net-create demo-net
Created a new network:
+-----------------------+--------------------------------------+
| Field | Value |
+-----------------------+--------------------------------------+
| admin_state_up | True |
| created_at | 2016-07-03T14:51:39 |
| description | |
| id | 075ba699-dc4c-4625-8e0d-0a258a9aeb7d |
| name | demo-net |
| port_security_enabled | True |
| provider:network_type | midonet |
| router:external | False |
| shared | False |
| status | ACTIVE |
| subnets | |
| tags | |
| tenant_id | 2f7ee2716b3b4140be57b4a5b26401e3 |
| updated_at | 2016-07-03T14:51:39 |
+-----------------------+--------------------------------------+
# neutron subnet-create demo-net 10.0.20.0/24 --name demo-subnet
Created a new subnet:
+-------------------+----------------------------------------------+
| Field | Value |
+-------------------+----------------------------------------------+
| allocation_pools | {"start": "10.0.20.2", "end": "10.0.20.254"} |
| cidr | 10.0.20.0/24 |
| created_at | 2016-07-03T14:52:32 |
| description | |
| dns_nameservers | |
| enable_dhcp | True |
| gateway_ip | 10.0.20.1 |
| host_routes | |
| id | b299d899-33a3-4bfa-aff4-fda071545bdf |
| ip_version | 4 |
| ipv6_address_mode | |
| ipv6_ra_mode | |
| name | demo-subnet |
| network_id | 075ba699-dc4c-4625-8e0d-0a258a9aeb7d |
| subnetpool_id | |
| tenant_id | 2f7ee2716b3b4140be57b4a5b26401e3 |
| updated_at | 2016-07-03T14:52:32 |
+-------------------+----------------------------------------------+Create a tenant router
# neutron router-create router1
Created a new router:
+-----------------------+--------------------------------------+
| Field | Value |
+-----------------------+--------------------------------------+
| admin_state_up | True |
| description | |
| external_gateway_info | |
| id | 258942d8-9d82-4ebd-b829-c7bdfcc973f5 |
| name | router1 |
| routes | |
| status | ACTIVE |
| tenant_id | 2f7ee2716b3b4140be57b4a5b26401e3 |
+-----------------------+--------------------------------------+Attach the tenant subnet interface we created before to the router
# neutron router-interface-add router1 demo-subnet
Added interface 06c85a56-368c-4d79-bbf0-4bb077f163e5 to router router1.Set the external network as router gateway
# neutron router-gateway-set router1 ext-net
Set gateway for router router1Now, you can create an instance at tenant network
# nova boot --flavor m1.tiny --image 80871834-29dd-4100-b038-f5f83f126204 --nic net-id=075ba699-dc4c-4625-8e0d-0a258a9aeb7d test1
+--------------------------------------+-----------------------------------------------------+
| Property | Value |
+--------------------------------------+-----------------------------------------------------+
| OS-DCF:diskConfig | MANUAL |
| OS-EXT-AZ:availability_zone | |
| OS-EXT-SRV-ATTR:host | - |
| OS-EXT-SRV-ATTR:hypervisor_hostname | - |
| OS-EXT-SRV-ATTR:instance_name | instance-0000000a |
| OS-EXT-STS:power_state | 0 |
| OS-EXT-STS:task_state | scheduling |
| OS-EXT-STS:vm_state | building |
| OS-SRV-USG:launched_at | - |
| OS-SRV-USG:terminated_at | - |
| accessIPv4 | |
| accessIPv6 | |
| adminPass | q2Cq4kxePSLL |
| config_drive | |
| created | 2016-07-03T15:46:19Z |
| flavor | m1.tiny (1) |
| hostId | |
| id | b8aa46f9-186c-4594-8428-f8dbb16a5e16 |
| image | cirros image (80871834-29dd-4100-b038-f5f83f126204) |
| key_name | - |
| metadata | {} |
| name | test1 |
| os-extended-volumes:volumes_attached | [] |
| progress | 0 |
| security_groups | default |
| status | BUILD |
| tenant_id | 2f7ee2716b3b4140be57b4a5b26401e3 |
| updated | 2016-07-03T15:46:20Z |
| user_id | a2482a91a1f14750b372445d28b07c75 |
+--------------------------------------+-----------------------------------------------------+
# nova list
+--------------------------------------+-------+--------+------------+-------------+---------------------+
| ID | Name | Status | Task State | Power State | Networks |
+--------------------------------------+-------+--------+------------+-------------+---------------------+
| b8aa46f9-186c-4594-8428-f8dbb16a5e16 | test1 | ACTIVE | - | Running | demo-net=10.0.20.11 |
+--------------------------------------+-------+--------+------------+-------------+---------------------+Ensure the instance gets IP and the metadata service is properly running
# nova console-log test1
...#Snipp from the output
Sending discover...
Sending select for 10.0.20.11...
Lease of 10.0.20.11 obtained, lease time 86400
cirros-ds 'net' up at 7.92
checking http://169.254.169.254/2009-04-04/instance-id
successful after 1/20 tries: up 8.22. iid=i-0000000a
...If you login to the instance through VNC you should be able to ping another instances
| Edge router configuration | Create a new router
# neutron router-create edge-router
Created a new router:
+-----------------------+--------------------------------------+
| Field | Value |
+-----------------------+--------------------------------------+
| admin_state_up | True |
| description | |
| external_gateway_info | |
| id | 5ecadb64-cb0d-4f95-a00e-aa1dd20a2012 |
| name | edge-router |
| routes | |
| status | ACTIVE |
| tenant_id | 2f7ee2716b3b4140be57b4a5b26401e3 |
+-----------------------+--------------------------------------+Attach the external subnet interface to the router
# neutron router-interface-add edge-router ext-subnet
Added interface e37f1986-c6b1-47f4-8268-02b837ceac17 to router edge-router.Create an uplink network
# neutron net-create uplink-network --tenant_id admin --provider:network_type uplink
Created a new network:
+-----------------------+--------------------------------------+
| Field | Value |
+-----------------------+--------------------------------------+
| admin_state_up | True |
| created_at | 2016-07-03T14:57:15 |
| description | |
| id | 77173ed4-6106-4515-af1c-3683897955f9 |
| name | uplink-network |
| port_security_enabled | True |
| provider:network_type | uplink |
| router:external | False |
| shared | False |
| status | ACTIVE |
| subnets | |
| tags | |
| tenant_id | admin |
| updated_at | 2016-07-03T14:57:15 |
+-----------------------+--------------------------------------+Create a subnet in the uplink network
# neutron subnet-create --tenant_id admin --disable-dhcp --name uplink-subnet uplink-network 192.168.1.0/24
Created a new subnet:
+-------------------+--------------------------------------------------+
| Field | Value |
+-------------------+--------------------------------------------------+
| allocation_pools | {"start": "192.168.1.2", "end": "192.168.1.254"} |
| cidr | 192.168.1.0/24 |
| created_at | 2016-07-03T15:06:28 |
| description | |
| dns_nameservers | |
| enable_dhcp | False |
| gateway_ip | 192.168.1.1 |
| host_routes | |
| id | 4e98e789-20d3-45fd-a3b5-9bcf02d8a832 |
| ip_version | 4 |
| ipv6_address_mode | |
| ipv6_ra_mode | |
| name | uplink-subnet |
| network_id | 77173ed4-6106-4515-af1c-3683897955f9 |
| subnetpool_id | |
| tenant_id | admin |
| updated_at | 2016-07-03T15:06:28 |
+-------------------+--------------------------------------------------+Create a port for each of the gateway nodes, interface should match with the NIC you want to use for binding the gateway nodes and a IP address for the same purposes
# neutron port-create uplink-network --binding:host_id gateway1 --binding:profile type=dict interface_name=eth1 --fixed-ip ip_address=192.168.1.199
Created a new port:
+-----------------------+--------------------------------------------------------------------------------------+
| Field | Value |
+-----------------------+--------------------------------------------------------------------------------------+
| admin_state_up | True |
| allowed_address_pairs | |
| binding:host_id | compute1 |
| binding:profile | {"interface_name": "eth1"} |
| binding:vif_details | {"port_filter": true} |
| binding:vif_type | midonet |
| binding:vnic_type | normal |
| created_at | 2016-07-03T15:10:06 |
| description | |
| device_id | |
| device_owner | |
| extra_dhcp_opts | |
| fixed_ips | {"subnet_id": "4e98e789-20d3-45fd-a3b5-9bcf02d8a832", "ip_address": "192.168.1.199"} |
| id | 7b4f54dd-2b41-42ba-9c5c-cda4640dc550 |
| mac_address | fa:16:3e:44:a8:c9 |
| name | |
| network_id | 77173ed4-6106-4515-af1c-3683897955f9 |
| port_security_enabled | True |
| security_groups | 0cf3e33e-dbd6-4b42-a0bd-6679b5eed4e1 |
| status | ACTIVE |
| tenant_id | 2f7ee2716b3b4140be57b4a5b26401e3 |
| updated_at | 2016-07-03T15:10:06 |
+-----------------------+--------------------------------------------------------------------------------------+Attach each of the ports to the edge router
# neutron router-interface-add edge-router port=7b4f54dd-2b41-42ba-9c5c-cda4640dc550
Added interface 7b4f54dd-2b41-42ba-9c5c-cda4640dc550 to router edge-router.| At this point you have to decide if use border routers with BGP enabled or static routes. | Use one of the following links to configure your use case: | https://docs.midonet.org/docs/latest/operations-guide/content/bgp_uplink_configuration.html | https://docs.midonet.org/docs/latest/operations-guide/content/static_setup.html
Issues I faced during configuration of Midonet
| Midolman agent don\'t start: | It was caused because midolman-env.sh file has more RAM configured as the one of my server. | Edit the file to match your server resources
# egrep ^MAX_HEAP_SIZE /etc/midolman/midolman-env.sh
MAX_HEAP_SIZE="2048M"Instances doesn\'t boot with the following error:
could not open /dev/net/tun: Permission deniedI had to remove br-tun bridges at ovs, if not, ovs locks the device and midolman cannot create the tunnel beetwen compute nodes and gateway nodes.
ovs-vsctl del-br br-tun| This post is my experience integrating Midonet into OpenStack, maybe some things are not correct, if you find any issue, please advise me to fix it. | Regards, Eduardo Gonzalez
Last updated
Was this helpful?