Blog
Search…
Hacking
OpenStack
Docker and DevOps
Rally OpenStack benchmarking with Docker
OpenStack Rally is a project under the Big Tent umbrella with the mission of verify OpenStack environments to ensure SLAs under high loads or fail over scenarios, and cloud services verification. Rally can also be used to continuous integration and delivery tasks.
Why use Rally inside a Docker container? Rally is a service that is not commonly used in most environments, is a tool that is used when new infrastructure changes are made or when a SLAs review must be done, not make any sense have a service consuming infrastructure resources or block a server only for use under specific situations. Also, if your OpenStack infrastructure is automated, with a container you can have a nice integration with CI/CD tools like Jenkins.
Main reasons to use Rally inside Docker containers:
- Quick tests/deployments of Rally tasks
- Automated testing
- Cost savings
- Operators can execute tasks with their own computers, freeinginfrastructure resources
- Re-utilization of resources
Here you got my suggestions about how to use Rally inside Docker:
- Create a new container(automatized or not by another tool)
- Always use an external volume to store rally reports data
- Execute Rally tasks
- Export the reports to the volume shared with the Docker host
- Kill the container
| Let\'s start with this quick guide: | Clone the repo i created with the Dockerfile
1
[[email protected] ~]$ git clone https://github.com/egonzalez90/docker-rally.git
Copied!
Move to docker-rally directory
1
[[email protected] ~]$ cd docker-rally/
Copied!
Create the Docker image
1
[[email protected] docker-rally]$ docker build -t egonzalez90/rally-mitaka .
2
Sending build context to Docker daemon 76.8 kB
3
Step 1 : FROM centos:7
4
---> 904d6c400333
5
Step 2 : MAINTAINER Eduardo Gonzalez Gutierrez <[email protected]>
6
---> Using cache
7
---> ee93bc7747e1
8
Step 3 : RUN yum install -y https://repos.fedorapeople.org/repos/openstack/openstack-mitaka/rdo-release-mitaka-3.noarch.rpm
9
---> Using cache
10
---> 8492ab9ee261
11
Step 4 : RUN yum update -y
12
---> Using cache
13
---> 1374340eb39a
14
Step 5 : RUN yum -y install openstack-rally gcc libffi-devel python-devel openssl-devel gmp-devel libxml2-devel libxslt-devel postgresql-devel redhat-rpm-config wget openstack-selinux openstack-utils && yum clean all
15
---> Using cache
16
---> 9b65e4a281be
17
Step 6 : RUN rally-manage --config-file /etc/rally/rally.conf db recreate
18
---> Using cache
19
---> dc4f3dbc1505
20
Successfully built dc4f3dbc1505
Copied!
Start rally container with a pseudo-tty and a volume to store rally execution data
1
[[email protected] docker-rally]$ docker run -ti -v /opt/rally-data/:/rally-data:Z egonzalez90/rally-mitaka
2
[[email protected] /]#
Copied!
Create a file called deploy.json with the admin info of your OpenStack environment
1
[[email protected] /]# vi deploy.json
2
​
3
{
4
"type": "ExistingCloud",
5
"auth_url": "http://controller:5000/v2.0",
6
"region_name": "RegionOne",
7
"admin": {
8
"username": "admin",
9
"password": "my_password",
10
"tenant_name": "admin"
11
}
12
}
Copied!
Create a deployment with the json we previously created
1
[[email protected] /]# rally deployment create --file=deploy.json --name=existing
2
2016-06-15 09:42:25.428 25 INFO rally.deployment.engine [-] Deployment a5162111-02a5-458f-bb59-f822cab1aa93 | Starting: OpenStack cloud deployment.
3
2016-06-15 09:42:25.478 25 INFO rally.deployment.engine [-] Deployment a5162111-02a5-458f-bb59-f822cab1aa93 | Completed: OpenStack cloud deployment.
4
+--------------------------------------+----------------------------+----------+------------------+--------+
5
| uuid | created_at | name | status | active |
6
+--------------------------------------+----------------------------+----------+------------------+--------+
7
| a5162111-02a5-458f-bb59-f822cab1aa93 | 2016-06-15 09:42:25.391691 | existing | deploy->finished | |
8
+--------------------------------------+----------------------------+----------+------------------+--------+
9
Using deployment: a5162111-02a5-458f-bb59-f822cab1aa93
10
~/.rally/openrc was updated
11
​
12
HINTS:
13
* To get your cloud resources, run:
14
rally show [flavors|images|keypairs|networks|secgroups]
15
​
16
* To use standard OpenStack clients, set up your env by running:
17
source ~/.rally/openrc
18
OpenStack clients are now configured, e.g run:
19
glance image-list
Copied!
Source the openrc file rally has created with your user info and test if you can connect with glance
1
[[email protected] /]# source ~/.rally/openrc
2
​
3
[[email protected] /]# glance image-list
4
+--------------------------------------+--------+
5
| ID | Name |
6
+--------------------------------------+--------+
7
| 1c4fc8a6-3ea7-433c-8ece-a14bbaf861e2 | cirros |
8
+--------------------------------------+--------+
Copied!
Check deployment status
1
[[email protected] /]# rally deployment check
2
keystone endpoints are valid and following services are available:
3
+-------------+----------------+-----------+
4
| services | type | status |
5
+-------------+----------------+-----------+
6
| __unknown__ | volumev2 | Available |
7
| ceilometer | metering | Available |
8
| cinder | volume | Available |
9
| cloud | cloudformation | Available |
10
| glance | image | Available |
11
| heat | orchestration | Available |
12
| keystone | identity | Available |
13
| neutron | network | Available |
14
| nova | compute | Available |
15
+-------------+----------------+-----------+
16
NOTE: '__unknown__' service name means that Keystone service catalog doesn't return name for this service and Rally can not identify service by its type. BUT you still can use such services with api_versions context, specifying type of service (execute `rally plugin show api_versions` for more details).
Copied!
Create a test execution file, this test will check if nova can boot and delete some instances
1
[[email protected] /]# vi execution.json
2
​
3
{
4
"NovaServers.boot_and_delete_server": [
5
{
6
"runner": {
7
"type": "constant",
8
"concurrency": 2,
9
"times": 10
10
},
11
"args": {
12
"force_delete": false,
13
"flavor": {
14
"name": "m1.tiny"
15
},
16
"image": {
17
"name": "cirros"
18
}
19
},
20
"context": {
21
"users": {
22
"project_domain": "default",
23
"users_per_tenant": 2,
24
"tenants": 3,
25
"resource_management_workers": 30,
26
"user_domain": "default"
27
}
28
}
29
}
30
]
31
}
Copied!
Run the task with the following command
1
[[email protected] /]# rally task start execution.json
2
--------------------------------------------------------------------------------
3
Preparing input task
4
--------------------------------------------------------------------------------
5
​
6
Input task is:
7
{
8
"NovaServers.boot_and_delete_server": [
9
{
10
"args": {
11
"flavor": {
12
"name": "m1.tiny"
13
},
14
"image": {
15
"name": "cirros"
16
},
17
"force_delete": false
18
},
19
"runner": {
20
"type": "constant",
21
"times": 10,
22
"concurrency": 2
23
},
24
"context": {
25
"users": {
26
"tenants": 3,
27
"users_per_tenant": 2
28
}
29
}
30
}
31
]
32
}
33
​
34
Task syntax is correct :)
35
2016-06-15 09:48:11.556 101 INFO rally.task.engine [-] Task 137eb997-d1f8-4d3f-918a-8aec3db7500f | Starting: Task validation.
36
2016-06-15 09:48:11.579 101 INFO rally.task.engine [-] Task 137eb997-d1f8-4d3f-918a-8aec3db7500f | Starting: Task validation of scenarios names.
37
2016-06-15 09:48:11.581 101 INFO rally.task.engine [-] Task 137eb997-d1f8-4d3f-918a-8aec3db7500f | Completed: Task validation of scenarios names.
38
2016-06-15 09:48:11.581 101 INFO rally.task.engine [-] Task 137eb997-d1f8-4d3f-918a-8aec3db7500f | Starting: Task validation of syntax.
39
2016-06-15 09:48:11.587 101 INFO rally.task.engine [-] Task 137eb997-d1f8-4d3f-918a-8aec3db7500f | Completed: Task validation of syntax.
40
2016-06-15 09:48:11.588 101 INFO rally.task.engine [-] Task 137eb997-d1f8-4d3f-918a-8aec3db7500f | Starting: Task validation of semantic.
41
2016-06-15 09:48:11.588 101 INFO rally.task.engine [-] Task 137eb997-d1f8-4d3f-918a-8aec3db7500f | Starting: Task validation check cloud.
42
2016-06-15 09:48:11.694 101 INFO rally.task.engine [-] Task 137eb997-d1f8-4d3f-918a-8aec3db7500f | Completed: Task validation check cloud.
43
2016-06-15 09:48:11.700 101 INFO rally.plugins.openstack.context.keystone.users [-] Task 137eb997-d1f8-4d3f-918a-8aec3db7500f | Starting: Enter context: `users`
44
2016-06-15 09:48:12.004 101 INFO rally.plugins.openstack.context.keystone.users [-] Task 137eb997-d1f8-4d3f-918a-8aec3db7500f | Completed: Enter context: `users`
45
2016-06-15 09:48:12.106 101 WARNING rally.task.types [-] FlavorResourceType is deprecated in Rally v0.3.2; use the equivalent resource plugin name instead
46
2016-06-15 09:48:12.207 101 WARNING rally.task.types [-] ImageResourceType is deprecated in Rally v0.3.2; use the equivalent resource plugin name instead
47
2016-06-15 09:48:12.395 101 INFO rally.plugins.openstack.context.keystone.users [-] Task 137eb997-d1f8-4d3f-918a-8aec3db7500f | Starting: Exit context: `users`
48
2016-06-15 09:48:13.546 101 INFO rally.plugins.openstack.context.keystone.users [-] Task 137eb997-d1f8-4d3f-918a-8aec3db7500f | Completed: Exit context: `users`
49
2016-06-15 09:48:13.546 101 INFO rally.task.engine [-] Task 137eb997-d1f8-4d3f-918a-8aec3db7500f | Completed: Task validation of semantic.
50
2016-06-15 09:48:13.547 101 INFO rally.task.engine [-] Task 137eb997-d1f8-4d3f-918a-8aec3db7500f | Completed: Task validation.
51
Task config is valid :)
52
--------------------------------------------------------------------------------
53
Task 137eb997-d1f8-4d3f-918a-8aec3db7500f: started
54
--------------------------------------------------------------------------------
55
​
56
Benchmarking... This can take a while...
57
​
58
To track task status use:
59
​
60
rally task status
61
or
62
rally task detailed
63
​
64
Using task: 137eb997-d1f8-4d3f-918a-8aec3db7500f
65
2016-06-15 09:48:13.555 101 INFO rally.api [-] Benchmark Task 137eb997-d1f8-4d3f-918a-8aec3db7500f on Deployment a5162111-02a5-458f-bb59-f822cab1aa93
66
2016-06-15 09:48:13.558 101 INFO rally.task.engine [-] Task 137eb997-d1f8-4d3f-918a-8aec3db7500f | Starting: Benchmarking.
67
2016-06-15 09:48:13.586 101 INFO rally.task.engine [-] Running benchmark with key:
68
{
69
"kw": {
70
"runner": {
71
"type": "constant",
72
"concurrency": 2,
73
"times": 10
74
},
75
"args": {
76
"force_delete": false,
77
"flavor": {
78
"name": "m1.tiny"
79
},
80
"image": {
81
"name": "cirros"
82
}
83
},
84
"context": {
85
"users": {
86
"users_per_tenant": 2,
87
"tenants": 3
88
}
89
}
90
},
91
"name": "NovaServers.boot_and_delete_server",
92
"pos": 0
93
}
94
2016-06-15 09:48:13.592 101 INFO rally.plugins.openstack.context.keystone.users [-] Task 137eb997-d1f8-4d3f-918a-8aec3db7500f | Starting: Enter context: `users`
95
2016-06-15 09:48:14.994 101 INFO rally.plugins.openstack.context.keystone.users [-] Task 137eb997-d1f8-4d3f-918a-8aec3db7500f | Completed: Enter context: `users`
96
2016-06-15 09:48:15.244 292 INFO rally.task.runner [-] Task 137eb997-d1f8-4d3f-918a-8aec3db7500f | ITER: 0 START
97
2016-06-15 09:48:15.245 293 INFO rally.task.runner [-] Task 137eb997-d1f8-4d3f-918a-8aec3db7500f | ITER: 1 START
98
2016-06-15 09:48:16.975 292 WARNING rally.common.logging [-] 'wait_for' is deprecated in Rally v0.1.2: Use wait_for_status instead.
99
2016-06-15 09:48:17.095 293 WARNING rally.common.logging [-] 'wait_for' is deprecated in Rally v0.1.2: Use wait_for_status instead.
100
2016-06-15 09:49:21.024 292 INFO rally.task.runner [-] Task 137eb997-d1f8-4d3f-918a-8aec3db7500f | ITER: 0 END: OK
101
2016-06-15 09:49:21.028 292 INFO rally.task.runner [-] Task 137eb997-d1f8-4d3f-918a-8aec3db7500f | ITER: 2 START
102
2016-06-15 09:49:32.109 293 INFO rally.task.runner [-] Task 137eb997-d1f8-4d3f-918a-8aec3db7500f | ITER: 1 END: OK
103
2016-06-15 09:49:32.112 293 INFO rally.task.runner [-] Task 137eb997-d1f8-4d3f-918a-8aec3db7500f | ITER: 3 START
104
2016-06-15 09:49:41.504 292 INFO rally.task.runner [-] Task 137eb997-d1f8-4d3f-918a-8aec3db7500f | ITER: 2 END: OK
105
2016-06-15 09:49:41.508 292 INFO rally.task.runner [-] Task 137eb997-d1f8-4d3f-918a-8aec3db7500f | ITER: 4 START
106
2016-06-15 09:49:52.455 293 INFO rally.task.runner [-] Task 137eb997-d1f8-4d3f-918a-8aec3db7500f | ITER: 3 END: OK
107
2016-06-15 09:49:52.462 293 INFO rally.task.runner [-] Task 137eb997-d1f8-4d3f-918a-8aec3db7500f | ITER: 5 START
108
2016-06-15 09:50:01.907 292 INFO rally.task.runner [-] Task 137eb997-d1f8-4d3f-918a-8aec3db7500f | ITER: 4 END: OK
109
2016-06-15 09:50:01.918 292 INFO rally.task.runner [-] Task 137eb997-d1f8-4d3f-918a-8aec3db7500f | ITER: 6 START
110
2016-06-15 09:50:12.692 293 INFO rally.task.runner [-] Task 137eb997-d1f8-4d3f-918a-8aec3db7500f | ITER: 5 END: OK
111
2016-06-15 09:50:12.694 293 INFO rally.task.runner [-] Task 137eb997-d1f8-4d3f-918a-8aec3db7500f | ITER: 7 START
112
2016-06-15 09:50:23.122 292 INFO rally.task.runner [-] Task 137eb997-d1f8-4d3f-918a-8aec3db7500f | ITER: 6 END: OK
113
2016-06-15 09:50:23.131 292 INFO rally.task.runner [-] Task 137eb997-d1f8-4d3f-918a-8aec3db7500f | ITER: 8 START
114
2016-06-15 09:50:33.322 293 INFO rally.task.runner [-] Task 137eb997-d1f8-4d3f-918a-8aec3db7500f | ITER: 7 END: OK
115
2016-06-15 09:50:33.332 293 INFO rally.task.runner [-] Task 137eb997-d1f8-4d3f-918a-8aec3db7500f | ITER: 9 START
116
2016-06-15 09:50:43.285 292 INFO rally.task.runner [-] Task 137eb997-d1f8-4d3f-918a-8aec3db7500f | ITER: 8 END: OK
117
2016-06-15 09:50:53.422 293 INFO rally.task.runner [-] Task 137eb997-d1f8-4d3f-918a-8aec3db7500f | ITER: 9 END: OK
118
2016-06-15 09:50:53.436 101 INFO rally.plugins.openstack.context.cleanup.user [-] Task 137eb997-d1f8-4d3f-918a-8aec3db7500f | Starting: user resources cleanup
119
2016-06-15 09:50:55.244 101 INFO rally.plugins.openstack.context.cleanup.user [-] Task 137eb997-d1f8-4d3f-918a-8aec3db7500f | Completed: user resources cleanup
120
2016-06-15 09:50:55.245 101 INFO rally.plugins.openstack.context.keystone.users [-] Task 137eb997-d1f8-4d3f-918a-8aec3db7500f | Starting: Exit context: `users`
121
2016-06-15 09:50:57.438 101 INFO rally.plugins.openstack.context.keystone.users [-] Task 137eb997-d1f8-4d3f-918a-8aec3db7500f | Completed: Exit context: `users`
122
2016-06-15 09:50:58.023 101 INFO rally.task.engine [-] Task 137eb997-d1f8-4d3f-918a-8aec3db7500f | Completed: Benchmarking.
123
​
124
--------------------------------------------------------------------------------
125
Task 137eb997-d1f8-4d3f-918a-8aec3db7500f: finished
126
--------------------------------------------------------------------------------
127
​
128
test scenario NovaServers.boot_and_delete_server
129
args position 0
130
args values:
131
{
132
"runner": {
133
"type": "constant",
134
"concurrency": 2,
135
"times": 10
136
},
137
"args": {
138
"force_delete": false,
139
"flavor": {
140
"name": "m1.tiny"
141
},
142
"image": {
143
"name": "cirros"
144
}
145
},
146
"context": {
147
"users": {
148
"project_domain": "default",
149
"users_per_tenant": 2,
150
"tenants": 3,
151
"resource_management_workers": 30,
152
"user_domain": "default"
153
}
154
}
155
}
156
​
157
+-----------------------------------------------------------------------------------------------------------------------+
158
| Response Times (sec) |
159
+--------------------+-----------+--------------+--------------+--------------+-----------+-----------+---------+-------+
160
| Action | Min (sec) | Median (sec) | 90%ile (sec) | 95%ile (sec) | Max (sec) | Avg (sec) | Success | Count |
161
+--------------------+-----------+--------------+--------------+--------------+-----------+-----------+---------+-------+
162
| nova.boot_server | 17.84 | 18.158 | 64.433 | 69.419 | 74.405 | 28.299 | 100.0% | 10 |
163
| nova.delete_server | 2.24 | 2.275 | 2.454 | 2.456 | 2.458 | 2.317 | 100.0% | 10 |
164
| total | 20.09 | 20.437 | 66.888 | 71.875 | 76.863 | 30.616 | 100.0% | 10 |
165
+--------------------+-----------+--------------+--------------+--------------+-----------+-----------+---------+-------+
166
​
167
Load duration: 158.199862003
168
Full duration: 163.846753836
169
​
170
HINTS:
171
* To plot HTML graphics with this data, run:
172
rally task report 137eb997-d1f8-4d3f-918a-8aec3db7500f --out output.html
173
​
174
* To generate a JUnit report, run:
175
rally task report 137eb997-d1f8-4d3f-918a-8aec3db7500f --junit --out output.xml
176
​
177
* To get raw JSON output of task results, run:
178
rally task results 137eb997-d1f8-4d3f-918a-8aec3db7500f
Copied!
| After a while, you will receive an output execution resume, you can export to a report file with the following command in a pretty style report. | Use the volume we created with the Docker Host to save report files.
1
[[email protected] /]# rally task report 137eb997-d1f8-4d3f-918a-8aec3db7500f --html-static --out /rally-data/output.html
Copied!
| Open the output file form a Web browser and review the report.
| Regards
Last modified 1yr ago
Copy link