)]}'
{"/COMMIT_MSG":[{"author":{"_account_id":9708,"name":"Balazs Gibizer","display_name":"gibi","email":"gibizer@gmail.com","username":"gibi"},"change_message_id":"cbff9c69670edad82391567d0040433673e951a7","unresolved":false,"context_lines":[{"line_number":13,"context_line":""},{"line_number":14,"context_line":"1. A negative test where we inject a fault by stopping"},{"line_number":15,"context_line":"   libvirt prior to the evacuation and wait for the"},{"line_number":16,"context_line":"   server to go to ERROR status."},{"line_number":17,"context_line":""},{"line_number":18,"context_line":"2. A positive where we restart libvirt, wait for the"},{"line_number":19,"context_line":"   compute service to be enabled and then evacuate"}],"source_content_type":"text/x-gerrit-commit-message","patch_set":13,"id":"3f79a3b5_ff3d5329","line":16,"updated":"2018-10-19 14:40:11.000000000","message":"How this is capable of testing the https://bugs.launchpad.net/nova/+bug/1550919 ? I imagine for that we would need to check that the instance on the source host are still can be restarted successfully (e.g. its local disk is not deleted from the shared storage by the failed evacuation)","commit_id":"5aef13434fba96be23ab16b945c0e5ece10d7c3f"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"31a7715caf0347ac18bc4ca3ffc4f22abf084e57","unresolved":false,"context_lines":[{"line_number":13,"context_line":""},{"line_number":14,"context_line":"1. A negative test where we inject a fault by stopping"},{"line_number":15,"context_line":"   libvirt prior to the evacuation and wait for the"},{"line_number":16,"context_line":"   server to go to ERROR status."},{"line_number":17,"context_line":""},{"line_number":18,"context_line":"2. A positive where we restart libvirt, wait for the"},{"line_number":19,"context_line":"   compute service to be enabled and then evacuate"}],"source_content_type":"text/x-gerrit-commit-message","patch_set":13,"id":"3f79a3b5_bc227ea5","line":16,"in_reply_to":"3f79a3b5_ff3d5329","updated":"2018-10-19 22:40:20.000000000","message":"I\u0027m not claiming that this covers all scenarios. In this change, the script runs after the live migration tests have setup and run with ceph, so by the time this runs we have shared storage for ceph and rbd volumes. A later change in the series makes this run for both local/lvm in addition to ceph/rbd, but that still doesn\u0027t give us NFS coverage. We do have an NFS job in the experimental queue but it\u0027s only single node. The live migration job could also be setup to run NFS in addition to local/lvm and ceph/rbd, but that was turned off years ago because it wasn\u0027t working with how the live migration test scripts work (and no one has cared to work on it).\n\nSo to summarize, I wrote this because we\u0027ve had lots of evacuate-related bugs over the years and I\u0027ve always been nervous about fixes when we had no integration testing since tempest can\u0027t test this (since we have to take one of the computes down and tempest runs tests concurrently for most jobs). So before moving forward with mdbooth\u0027s shared storage evacuate fix, I wanted to make sure we had at least some basic integration testing of evacuate, which is what this series is trying to do (and also uncovered a placement-related bug as a result).\n\nIf the issue is the related-bug reference, I can remove that.","commit_id":"5aef13434fba96be23ab16b945c0e5ece10d7c3f"}],"gate/test_evacuate.sh":[{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"d9394acaacf55c7b4b7f1cbadef99dfe0d29f0ff","unresolved":false,"context_lines":[{"line_number":5,"context_line":"source ${BASE}/new/devstack/functions-common"},{"line_number":6,"context_line":""},{"line_number":7,"context_line":"# TODO(mriedem): Die if not using libvirt?"},{"line_number":8,"context_line":"echo \"Using virt driver: ${VIRT_DRIVER}\""},{"line_number":9,"context_line":""},{"line_number":10,"context_line":"echo \"Ensure we have at least two compute nodes\""},{"line_number":11,"context_line":"nodenames\u003d$(openstack hypervisor list -f value -c \u0027Hypervisor Hostname\u0027)"}],"source_content_type":"text/x-sh","patch_set":2,"id":"3f79a3b5_8205d4c3","line":8,"range":{"start_line":8,"start_character":27,"end_line":8,"end_character":38},"updated":"2018-09-13 15:08:22.000000000","message":"This isn\u0027t set:\n\nhttp://logs.openstack.org/74/602174/2/check/nova-live-migration/dde109e/logs/devstack-gate-post_test_hook.txt.gz#_2018-09-12_22_23_40_079\n\n2018-09-12 22:23:40.081 | Using virt driver:","commit_id":"e656da96632d56fbd0b6ac0e85a1e71d8fadc486"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"d9394acaacf55c7b4b7f1cbadef99dfe0d29f0ff","unresolved":false,"context_lines":[{"line_number":10,"context_line":"echo \"Ensure we have at least two compute nodes\""},{"line_number":11,"context_line":"nodenames\u003d$(openstack hypervisor list -f value -c \u0027Hypervisor Hostname\u0027)"},{"line_number":12,"context_line":"node_count\u003d$(echo ${nodenames} | wc -l)"},{"line_number":13,"context_line":"if [[ ${node_count} -lt 2 ]]; then"},{"line_number":14,"context_line":"    echo \"Evacuate requires at least two nodes\""},{"line_number":15,"context_line":"    exit 1"},{"line_number":16,"context_line":"fi"}],"source_content_type":"text/x-sh","patch_set":2,"id":"3f79a3b5_622bd832","line":13,"updated":"2018-09-13 15:08:22.000000000","message":"Apparently we\u0027re only reading one node...\n\nhttp://logs.openstack.org/74/602174/2/check/nova-live-migration/dde109e/logs/devstack-gate-post_test_hook.txt.gz#_2018-09-12_22_23_41_157\n\n2018-09-12 22:23:41.157 | + /opt/stack/new/nova/gate/test_evacuate.sh:main:L12:   node_count\u003d1\n2018-09-12 22:23:41.158 | + /opt/stack/new/nova/gate/test_evacuate.sh:main:L13:   [[ 1 -lt 2 ]]","commit_id":"e656da96632d56fbd0b6ac0e85a1e71d8fadc486"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"95bb26b29bbc5d4b6cf88766744592dd63a1434b","unresolved":false,"context_lines":[{"line_number":10,"context_line":"echo \"Ensure we have at least two compute nodes\""},{"line_number":11,"context_line":"nodenames\u003d$(openstack hypervisor list -f value -c \u0027Hypervisor Hostname\u0027)"},{"line_number":12,"context_line":"node_count\u003d$(echo ${nodenames} | wc -l)"},{"line_number":13,"context_line":"if [[ ${node_count} -lt 2 ]]; then"},{"line_number":14,"context_line":"    echo \"Evacuate requires at least two nodes\""},{"line_number":15,"context_line":"    exit 1"},{"line_number":16,"context_line":"fi"}],"source_content_type":"text/x-sh","patch_set":2,"id":"3f79a3b5_c313ce00","line":13,"in_reply_to":"3f79a3b5_622bd832","updated":"2018-09-13 16:46:38.000000000","message":"This is why:\n\n2018-09-12 22:23:40.086 | ++ /opt/stack/new/nova/gate/test_evacuate.sh:main:L11:   openstack hypervisor list -f value -c \u0027Hypervisor Hostname\u0027\n2018-09-12 22:23:41.084 | Missing value auth-url required for auth plugin password\n\n$ echo \"Missing value auth-url required for auth plugin password\" | wc -l\n1","commit_id":"e656da96632d56fbd0b6ac0e85a1e71d8fadc486"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"ccd0334b5c2fc87b6f16a64bde4b91e758945146","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":2,"id":"3f79a3b5_5620ecf8","line":70,"updated":"2018-09-12 22:33:11.000000000","message":"TODO: now bring the local node back up and evacuate and wait for it to be ACTIVE.","commit_id":"e656da96632d56fbd0b6ac0e85a1e71d8fadc486"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"17d96f012cc2d05b3009b5ee30259b9ffa40e511","unresolved":false,"context_lines":[{"line_number":15,"context_line":""},{"line_number":16,"context_line":"echo \"Ensure we have at least two compute nodes\""},{"line_number":17,"context_line":"nodenames\u003d$(openstack hypervisor list -f value -c \u0027Hypervisor Hostname\u0027)"},{"line_number":18,"context_line":"node_count\u003d$(echo ${nodenames} | wc -l)"},{"line_number":19,"context_line":"if [[ ${node_count} -lt 2 ]]; then"},{"line_number":20,"context_line":"    echo \"Evacuate requires at least two nodes\""},{"line_number":21,"context_line":"    exit 1"}],"source_content_type":"text/x-sh","patch_set":3,"id":"3f79a3b5_a984705e","line":18,"updated":"2018-09-13 20:52:00.000000000","message":"Hmm, still hitting this:\n\nhttp://logs.openstack.org/74/602174/3/check/nova-live-migration/f27e71d/logs/devstack-gate-post_test_hook.txt.gz#_2018-09-13_19_06_52_223\n\n2018-09-13 19:06:52.223 | ++ /opt/stack/new/nova/gate/test_evacuate.sh:main:17 :   openstack hypervisor list -f value -c \u0027Hypervisor Hostname\u0027\n2018-09-13 19:06:53.797 | + /opt/stack/new/nova/gate/test_evacuate.sh:main:17 :   nodenames\u003d\u0027ubuntu-xenial-vexxhost-sjc1-0002002628\n2018-09-13 19:06:53.799 | ubuntu-xenial-vexxhost-sjc1-0002002736\u0027\n2018-09-13 19:06:53.801 | ++ /opt/stack/new/nova/gate/test_evacuate.sh:main:18 :   wc -l\n2018-09-13 19:06:53.802 | ++ /opt/stack/new/nova/gate/test_evacuate.sh:main:18 :   echo ubuntu-xenial-vexxhost-sjc1-0002002628 ubuntu-xenial-vexxhost-sjc1-0002002736\n2018-09-13 19:06:53.804 | + /opt/stack/new/nova/gate/test_evacuate.sh:main:18 :   node_count\u003d1\n2018-09-13 19:06:53.806 | + /opt/stack/new/nova/gate/test_evacuate.sh:main:19 :   [[ 1 -lt 2 ]]","commit_id":"5d0036b212043f6f356581f0d599202ce5f7be82"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"1819c9798f8bd697e7f606c6ea2cc11c4f70ff3e","unresolved":false,"context_lines":[{"line_number":22,"context_line":"fi"},{"line_number":23,"context_line":""},{"line_number":24,"context_line":"echo \"Finding the subnode\""},{"line_number":25,"context_line":"subnode \u003d \u0027\u0027"},{"line_number":26,"context_line":"# TODO(mriedem): If this is flaky, use virsh hostname?"},{"line_number":27,"context_line":"local_hostname\u003d$(hostname -s)"},{"line_number":28,"context_line":"for nodename in ${nodenames}; do"}],"source_content_type":"text/x-sh","patch_set":4,"id":"3f79a3b5_6a52bce0","line":25,"range":{"start_line":25,"start_character":7,"end_line":25,"end_character":10},"updated":"2018-09-14 14:51:20.000000000","message":"blarg","commit_id":"26c1cc6d599c73a7d43557e9b0799abd5ded58b9"},{"author":{"_account_id":9555,"name":"Matthew Booth","email":"mbooth@redhat.com","username":"MatthewBooth"},"change_message_id":"630ba517a0283035583a0c5407878c9db148e65a","unresolved":false,"context_lines":[{"line_number":48,"context_line":"openstack --os-compute-api-version 2.11 compute service set --down ${subnode} nova-compute"},{"line_number":49,"context_line":""},{"line_number":50,"context_line":"echo \"Killing libvirtd on the localhost before evacuating to trigger failure\""},{"line_number":51,"context_line":"libvirtd_pid\u003d$(ps axjf | grep libvirtd | awk \u0027NR\u003d\u003d1{print $1}\u0027)"},{"line_number":52,"context_line":"kill -9 ${libvirtd_pid}"},{"line_number":53,"context_line":""},{"line_number":54,"context_line":"# Now force the evacuation to *this* host; we have to force to bypass the"}],"source_content_type":"text/x-sh","patch_set":5,"id":"3f79a3b5_717e2bb0","line":51,"updated":"2018-09-14 18:04:26.000000000","message":"Based on the failure in the logs, this seems to be returning pid 1, which we obviously fail to kill on the next line, causing failure.","commit_id":"38d4a99b0f22ec12693b84147af51d99c8e4b388"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"b04582bdd89854f78584b3d57b05cc70b071f110","unresolved":false,"context_lines":[{"line_number":10,"context_line":"source ${BASE}/new/devstack/openrc admin"},{"line_number":11,"context_line":"set -x"},{"line_number":12,"context_line":""},{"line_number":13,"context_line":"# TODO(mriedem): Die if not using libvirt?"},{"line_number":14,"context_line":"echo \"Using virt driver: ${VIRT_DRIVER}\""},{"line_number":15,"context_line":""},{"line_number":16,"context_line":"echo \"Ensure we have at least two compute nodes\""}],"source_content_type":"text/x-sh","patch_set":6,"id":"3f79a3b5_d2fdbe6e","line":13,"updated":"2018-09-14 19:37:13.000000000","message":"Given the libvirtd.pid read below, yes.","commit_id":"50e13c9064a0dc3b7ab111ed746d82e93a314eee"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"de831d840c76c68f7556d2902c55f04d7286002d","unresolved":false,"context_lines":[{"line_number":48,"context_line":"openstack --os-compute-api-version 2.11 compute service set --down ${subnode} nova-compute"},{"line_number":49,"context_line":""},{"line_number":50,"context_line":"echo \"Killing libvirtd on the localhost before evacuating to trigger failure\""},{"line_number":51,"context_line":"read libvirtd_pid \u003c /var/run/libvirtd.pid"},{"line_number":52,"context_line":"kill -9 ${libvirtd_pid}"},{"line_number":53,"context_line":""},{"line_number":54,"context_line":"# Now force the evacuation to *this* host; we have to force to bypass the"}],"source_content_type":"text/x-sh","patch_set":6,"id":"3f79a3b5_12b6d6dd","line":51,"updated":"2018-09-14 19:31:42.000000000","message":"I don\u0027t think this is going to work:\n\nstack@stein:~$ cat /var/run/libvirtd.pid\n16770stack@stein:~$ read libvirtd_pid \u003c /var/run/libvirtd.pid\nstack@stein:~$ echo $libvirt_pid\n\nstack@stein:~$","commit_id":"50e13c9064a0dc3b7ab111ed746d82e93a314eee"},{"author":{"_account_id":9555,"name":"Matthew Booth","email":"mbooth@redhat.com","username":"MatthewBooth"},"change_message_id":"04f3067331878fbff4d8f85f7c4bc41c3eb2ece6","unresolved":false,"context_lines":[{"line_number":48,"context_line":"openstack --os-compute-api-version 2.11 compute service set --down ${subnode} nova-compute"},{"line_number":49,"context_line":""},{"line_number":50,"context_line":"echo \"Killing libvirtd on the localhost before evacuating to trigger failure\""},{"line_number":51,"context_line":"read libvirtd_pid \u003c /var/run/libvirtd.pid"},{"line_number":52,"context_line":"kill -9 ${libvirtd_pid}"},{"line_number":53,"context_line":""},{"line_number":54,"context_line":"# Now force the evacuation to *this* host; we have to force to bypass the"}],"source_content_type":"text/x-sh","patch_set":6,"id":"3f79a3b5_75e934e1","line":51,"in_reply_to":"3f79a3b5_1204967d","updated":"2018-09-14 20:16:33.000000000","message":"It failed again. My guess is either permissions, or the pid file doesn\u0027t exist (at least in that location).","commit_id":"50e13c9064a0dc3b7ab111ed746d82e93a314eee"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"65c0e2eea6c4c2d06ae644de678423b110fd4516","unresolved":false,"context_lines":[{"line_number":48,"context_line":"openstack --os-compute-api-version 2.11 compute service set --down ${subnode} nova-compute"},{"line_number":49,"context_line":""},{"line_number":50,"context_line":"echo \"Killing libvirtd on the localhost before evacuating to trigger failure\""},{"line_number":51,"context_line":"read libvirtd_pid \u003c /var/run/libvirtd.pid"},{"line_number":52,"context_line":"kill -9 ${libvirtd_pid}"},{"line_number":53,"context_line":""},{"line_number":54,"context_line":"# Now force the evacuation to *this* host; we have to force to bypass the"}],"source_content_type":"text/x-sh","patch_set":6,"id":"3f79a3b5_b2c4c242","line":51,"in_reply_to":"3f79a3b5_12b6d6dd","updated":"2018-09-14 19:33:38.000000000","message":"This might have:\n\nstack@stein:~$ ps axjf | grep libvirtd\n11933 12235 12234 11912 pts/0    12234 S+    1001   0:00                          \\_ grep --color\u003dauto libvirtd\n    1 16770 16770 16770 ?           -1 Ssl      0   1:27 /usr/sbin/libvirtd\nstack@stein:~$ ps axjf | grep libvirtd | awk \u0027NR\u003d\u003d2{print $2}\u0027\n16770\nstack@stein:~$ \n\nObviously reading from the pid file is ideal though. We should probably check to make sure it exists first.","commit_id":"50e13c9064a0dc3b7ab111ed746d82e93a314eee"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"3e4fe7063ab9feafa94b080b2923d590325518cf","unresolved":false,"context_lines":[{"line_number":48,"context_line":"openstack --os-compute-api-version 2.11 compute service set --down ${subnode} nova-compute"},{"line_number":49,"context_line":""},{"line_number":50,"context_line":"echo \"Killing libvirtd on the localhost before evacuating to trigger failure\""},{"line_number":51,"context_line":"read libvirtd_pid \u003c /var/run/libvirtd.pid"},{"line_number":52,"context_line":"kill -9 ${libvirtd_pid}"},{"line_number":53,"context_line":""},{"line_number":54,"context_line":"# Now force the evacuation to *this* host; we have to force to bypass the"}],"source_content_type":"text/x-sh","patch_set":6,"id":"3f79a3b5_1204967d","line":51,"in_reply_to":"3f79a3b5_b2c4c242","updated":"2018-09-14 19:36:48.000000000","message":"Oh nevermind I guess read is a bash thing I\u0027m not aware of. Works fine in a bash script, just not on command line.","commit_id":"50e13c9064a0dc3b7ab111ed746d82e93a314eee"},{"author":{"_account_id":9555,"name":"Matthew Booth","email":"mbooth@redhat.com","username":"MatthewBooth"},"change_message_id":"c20e1875f62ece50300def61856765e64cd97161","unresolved":false,"context_lines":[{"line_number":48,"context_line":"openstack --os-compute-api-version 2.11 compute service set --down ${subnode} nova-compute"},{"line_number":49,"context_line":""},{"line_number":50,"context_line":"echo \"Killing libvirtd on the localhost before evacuating to trigger failure\""},{"line_number":51,"context_line":"read libvirtd_pid \u003c /var/run/libvirtd.pid"},{"line_number":52,"context_line":"kill -9 ${libvirtd_pid}"},{"line_number":53,"context_line":""},{"line_number":54,"context_line":"# Now force the evacuation to *this* host; we have to force to bypass the"}],"source_content_type":"text/x-sh","patch_set":6,"id":"3f79a3b5_f2e79ae2","line":51,"in_reply_to":"3f79a3b5_b2c4c242","updated":"2018-09-14 19:35:51.000000000","message":"You had a typo in your test above: libvirtd_pid/libvirt_pid\n\nIt works for me locally.","commit_id":"50e13c9064a0dc3b7ab111ed746d82e93a314eee"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"9904a471d14ef42157016111f31d232b78b8bfac","unresolved":false,"context_lines":[{"line_number":48,"context_line":"openstack --os-compute-api-version 2.11 compute service set --down ${subnode} nova-compute"},{"line_number":49,"context_line":""},{"line_number":50,"context_line":"echo \"Killing libvirtd on the localhost before evacuating to trigger failure\""},{"line_number":51,"context_line":"libvirtd_pid\u003d$(ps axjf | grep libvirtd | awk \u0027$1\u003d\u003d1{print $2}\u0027)"},{"line_number":52,"context_line":"kill -9 ${libvirtd_pid}"},{"line_number":53,"context_line":""},{"line_number":54,"context_line":"# Now force the evacuation to *this* host; we have to force to bypass the"}],"source_content_type":"text/x-sh","patch_set":7,"id":"3f79a3b5_a676c43e","line":51,"updated":"2018-09-14 22:53:47.000000000","message":"Could also maybe use pkill here.","commit_id":"e227b1484ffafa43ed658d039b0c7403e4c53b76"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"853da05bbe8abeb0a4283ff658b0e03dec44810f","unresolved":false,"context_lines":[{"line_number":60,"context_line":"# TODO(mriedem): Use OSC when it supports evacuate."},{"line_number":61,"context_line":"echo \"Forcing evacuate to local host\""},{"line_number":62,"context_line":"nova evacuate --force evacuate-test ${local_hostname}"},{"line_number":63,"context_line":"# Wait for the instance to go into ERROR state from the failed evacuate."},{"line_number":64,"context_line":"count\u003d0"},{"line_number":65,"context_line":"status\u003d$(openstack server show evacuate-test -f value -c status)"},{"line_number":66,"context_line":"while [ \"${status}\" !\u003d \"ERROR\" ]"}],"source_content_type":"text/x-sh","patch_set":9,"id":"3f79a3b5_c7617013","line":63,"updated":"2018-09-16 15:28:23.000000000","message":"This works now:\n\nhttp://logs.openstack.org/74/602174/9/check/nova-live-migration/0ece220/logs/devstack-gate-post_test_hook.txt.gz#_2018-09-15_03_24_22_202","commit_id":"4eca502f4ff2ad0271e6a451448e47c92122c4eb"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"853da05bbe8abeb0a4283ff658b0e03dec44810f","unresolved":false,"context_lines":[{"line_number":78,"context_line":"sudo systemctl start libvirt-bin"},{"line_number":79,"context_line":"sleep 10"},{"line_number":80,"context_line":""},{"line_number":81,"context_line":"nova evacuate evacuate-test"},{"line_number":82,"context_line":"# Wait for the instance to go into ACTIVE state from the evacuate."},{"line_number":83,"context_line":"count\u003d0"},{"line_number":84,"context_line":"status\u003d$(openstack server show evacuate-test -f value -c status)"}],"source_content_type":"text/x-sh","patch_set":9,"id":"3f79a3b5_6772dce5","line":81,"updated":"2018-09-16 15:28:23.000000000","message":"But this doesn\u0027t:\n\nhttp://logs.openstack.org/74/602174/9/check/nova-live-migration/0ece220/logs/devstack-gate-post_test_hook.txt.gz#_2018-09-15_03_24_34_298\n\nBecause the compute service says it\u0027s down:\n\nhttp://logs.openstack.org/74/602174/9/check/nova-live-migration/0ece220/logs/screen-n-sch.txt.gz#_Sep_15_03_24_34_519592\n\nSo we probably need a loop to wait for the local compute service to report as up.","commit_id":"4eca502f4ff2ad0271e6a451448e47c92122c4eb"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"6ea21ba25399e41db99241396c585a8a91db9797","unresolved":false,"context_lines":[{"line_number":78,"context_line":"sudo systemctl start libvirt-bin"},{"line_number":79,"context_line":"sleep 10"},{"line_number":80,"context_line":""},{"line_number":81,"context_line":"nova evacuate evacuate-test"},{"line_number":82,"context_line":"# Wait for the instance to go into ACTIVE state from the evacuate."},{"line_number":83,"context_line":"count\u003d0"},{"line_number":84,"context_line":"status\u003d$(openstack server show evacuate-test -f value -c status)"}],"source_content_type":"text/x-sh","patch_set":9,"id":"3f79a3b5_ea023309","line":81,"in_reply_to":"3f79a3b5_6772dce5","updated":"2018-09-16 15:29:52.000000000","message":"Looks like if we would have waited another 2 seconds it would have been OK:\n\nhttp://logs.openstack.org/74/602174/9/check/nova-live-migration/0ece220/logs/screen-n-cpu.txt.gz#_Sep_15_03_25_23_176525","commit_id":"4eca502f4ff2ad0271e6a451448e47c92122c4eb"},{"author":{"_account_id":9555,"name":"Matthew Booth","email":"mbooth@redhat.com","username":"MatthewBooth"},"change_message_id":"0d9a0d9d35f3eab92d0dbc0f03f62e9e1f57d392","unresolved":false,"context_lines":[{"line_number":10,"context_line":"source ${BASE}/new/devstack/openrc admin"},{"line_number":11,"context_line":"set -x"},{"line_number":12,"context_line":""},{"line_number":13,"context_line":"if [[ ${VIRT_DRIVER} !\u003d libvirt ]]; then"},{"line_number":14,"context_line":"   echo \"Only the libvirt driver is supported by this script\""},{"line_number":15,"context_line":"   exit 1"},{"line_number":16,"context_line":"fi"}],"source_content_type":"text/x-sh","patch_set":11,"id":"3f79a3b5_8842dc67","line":13,"updated":"2018-09-26 15:55:45.000000000","message":"At some point in the past (but certainly within my career at Red Hat) this would have failed if VIRT_DRIVER was blank/undefined, but it seems to work on the bash on my F28 system. The idiom for doing this was always:\n\n  if [[ \"x${VIRT_DRIVER}\" !\u003d \"xlibvirt\" ]]; then\n\nIt also works on my RHEL 7.5 system, so I guess this can go the way of the dodo.\n\nLeaving this here in case anybody else has the same thought.","commit_id":"2c8e83b2c99cd6e0ed7bf207cf2c8fd01daec216"},{"author":{"_account_id":9555,"name":"Matthew Booth","email":"mbooth@redhat.com","username":"MatthewBooth"},"change_message_id":"0d9a0d9d35f3eab92d0dbc0f03f62e9e1f57d392","unresolved":false,"context_lines":[{"line_number":12,"context_line":""},{"line_number":13,"context_line":"if [[ ${VIRT_DRIVER} !\u003d libvirt ]]; then"},{"line_number":14,"context_line":"   echo \"Only the libvirt driver is supported by this script\""},{"line_number":15,"context_line":"   exit 1"},{"line_number":16,"context_line":"fi"},{"line_number":17,"context_line":""},{"line_number":18,"context_line":"echo \"Ensure we have at least two compute nodes\""}],"source_content_type":"text/x-sh","patch_set":11,"id":"3f79a3b5_08e1ec9b","line":15,"updated":"2018-09-26 15:55:45.000000000","message":"As discussed on IRC, this script is \u0027opt-in\u0027 for CI, so failing here (rather than skipping) is the correct behaviour.","commit_id":"2c8e83b2c99cd6e0ed7bf207cf2c8fd01daec216"},{"author":{"_account_id":9555,"name":"Matthew Booth","email":"mbooth@redhat.com","username":"MatthewBooth"},"change_message_id":"0d9a0d9d35f3eab92d0dbc0f03f62e9e1f57d392","unresolved":false,"context_lines":[{"line_number":40,"context_line":"fi"},{"line_number":41,"context_line":""},{"line_number":42,"context_line":"echo \"Creating test server on subnode\""},{"line_number":43,"context_line":"image\u003d$(openstack image list -f value -c Name | awk \u0027NR\u003d\u003d1{print $1}\u0027)"},{"line_number":44,"context_line":"flavor\u003d$(openstack flavor list -f value -c Name | awk \u0027NR\u003d\u003d1{print $1}\u0027)"},{"line_number":45,"context_line":"openstack server create --image ${image} --flavor ${flavor} \\"},{"line_number":46,"context_line":"--availability-zone nova:${subnode} --wait evacuate-test"},{"line_number":47,"context_line":""}],"source_content_type":"text/x-sh","patch_set":11,"id":"3f79a3b5_8ed18559","line":44,"range":{"start_line":43,"start_character":0,"end_line":44,"end_character":72},"updated":"2018-09-26 15:55:45.000000000","message":"Changed these in the next patch to be image_id and flavor_id. Probably not worth rewriting history.","commit_id":"2c8e83b2c99cd6e0ed7bf207cf2c8fd01daec216"},{"author":{"_account_id":9555,"name":"Matthew Booth","email":"mbooth@redhat.com","username":"MatthewBooth"},"change_message_id":"0d9a0d9d35f3eab92d0dbc0f03f62e9e1f57d392","unresolved":false,"context_lines":[{"line_number":76,"context_line":""},{"line_number":77,"context_line":"echo \"Now restart libvirt and perform a successful evacuation\""},{"line_number":78,"context_line":"sudo systemctl start libvirt-bin"},{"line_number":79,"context_line":"sleep 10"},{"line_number":80,"context_line":""},{"line_number":81,"context_line":"# Wait for the compute service to be enabled."},{"line_number":82,"context_line":"count\u003d0"}],"source_content_type":"text/x-sh","patch_set":11,"id":"3f79a3b5_0ef015da","line":79,"updated":"2018-09-26 15:55:45.000000000","message":"This seems redundant: the following poll should take care of this, and I wouldn\u0027t be surprised if this typically took less than 10 seconds anyway.\n\nNot a blocker.","commit_id":"2c8e83b2c99cd6e0ed7bf207cf2c8fd01daec216"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"086e728476a98cfec1907ba8eda67ca120a9de40","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":11,"id":"3f79a3b5_30559c50","line":109,"updated":"2018-09-20 15:41:58.000000000","message":"Another thing we could do here is test a volume-backed server getting evacuated.","commit_id":"2c8e83b2c99cd6e0ed7bf207cf2c8fd01daec216"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"2d8ffcc622b932a16e536d1a4f834f82726d8ac6","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":11,"id":"3f79a3b5_22fbf97a","line":109,"in_reply_to":"3f79a3b5_30559c50","updated":"2018-09-21 15:55:15.000000000","message":"Done in a follow up.","commit_id":"2c8e83b2c99cd6e0ed7bf207cf2c8fd01daec216"},{"author":{"_account_id":4393,"name":"Dan Smith","email":"dms@danplanet.com","username":"danms"},"change_message_id":"72ffba94ddea72aed292a9cc62f9425610906aaf","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":13,"id":"3f79a3b5_076b01f0","line":109,"updated":"2018-10-22 18:12:59.000000000","message":"Should we verify that it moved? I know it probably won\u0027t go back to happy state if it doesn\u0027t, but seems like a worthwhile check.","commit_id":"5aef13434fba96be23ab16b945c0e5ece10d7c3f"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"777bf4b1e2deedb8a5ae384a6d18d07549578775","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":13,"id":"3f79a3b5_eaa56cb5","line":109,"in_reply_to":"3f79a3b5_076b01f0","updated":"2018-10-22 18:36:53.000000000","message":"I can add that.","commit_id":"5aef13434fba96be23ab16b945c0e5ece10d7c3f"}]}