)]}'
{"/COMMIT_MSG":[{"author":{"_account_id":9555,"name":"Matthew Booth","email":"mbooth@redhat.com","username":"MatthewBooth"},"change_message_id":"2f5973c66b9551e645fa4c88fb7a8ad1db878b17","unresolved":false,"context_lines":[{"line_number":18,"context_line":"guest already exists on host B so _check_instance_exists()"},{"line_number":19,"context_line":"raises an InstanceExists error."},{"line_number":20,"context_line":""},{"line_number":21,"context_line":"So if we\u0027ve successfully spawned the guest but failed in"},{"line_number":22,"context_line":"some other way, this adds a rollback method to destroy"},{"line_number":23,"context_line":"the guest on the dest host whilst preserving disks on shared"},{"line_number":24,"context_line":"storage and updates the port bindings to point back at the"},{"line_number":25,"context_line":"source host."}],"source_content_type":"text/x-gerrit-commit-message","patch_set":1,"id":"5f7c97a3_d56bdc84","line":22,"range":{"start_line":21,"start_character":0,"end_line":22,"end_character":43},"updated":"2018-08-02 13:54:00.000000000","message":"Is it appropriate to rollback at this point? I would expect the driver\u0027s rebuild implementation (or _rebuild_default_impl) to either give us a running instance or clean up after itself. _rebuild_default_impl calls driver.spawn(), which *should* do this.\n\nLooking at _do_rebuild_instance we\u0027ve got 1 of 3 cases:\n\nFail before driver.rebuild(): ComputeManager fails the migration, rolls everything back.\n\nFail during driver.rebuild(): Driver is responsible for hypervisor resource cleanup on dest. ComputeManager fails the migration, rolls everything back.\n\nFail after driver.rebuild(): There\u0027s (by design) only less consequential stuff here like updating instance metadata, notifications. Should probably succeed migration, put instance in error state? You\u0027d know better than me what\u0027s most appropriate for this state. But... the instance will be fully operational on the destination, so rolling it back seems inappropriate.","commit_id":"d20fb07c6a043b2ae733b6e8b681ffc4ec49cf84"},{"author":{"_account_id":9555,"name":"Matthew Booth","email":"mbooth@redhat.com","username":"MatthewBooth"},"change_message_id":"8e45d3be1dea4c01db4ed5a0418709ae1e616c7a","unresolved":false,"context_lines":[{"line_number":18,"context_line":"guest already exists on host B so _check_instance_exists()"},{"line_number":19,"context_line":"raises an InstanceExists error."},{"line_number":20,"context_line":""},{"line_number":21,"context_line":"So if we\u0027ve successfully spawned the guest but failed in"},{"line_number":22,"context_line":"some other way, this adds a rollback method to destroy"},{"line_number":23,"context_line":"the guest on the dest host whilst preserving disks on shared"},{"line_number":24,"context_line":"storage and updates the port bindings to point back at the"},{"line_number":25,"context_line":"source host."}],"source_content_type":"text/x-gerrit-commit-message","patch_set":1,"id":"5f7c97a3_98e839e1","line":22,"range":{"start_line":21,"start_character":0,"end_line":22,"end_character":43},"in_reply_to":"5f7c97a3_35e490df","updated":"2018-08-02 14:21:31.000000000","message":"I agree with all of that.","commit_id":"d20fb07c6a043b2ae733b6e8b681ffc4ec49cf84"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"1ed0daf4bf66c9d664c271b0b640d8cc373122cf","unresolved":false,"context_lines":[{"line_number":18,"context_line":"guest already exists on host B so _check_instance_exists()"},{"line_number":19,"context_line":"raises an InstanceExists error."},{"line_number":20,"context_line":""},{"line_number":21,"context_line":"So if we\u0027ve successfully spawned the guest but failed in"},{"line_number":22,"context_line":"some other way, this adds a rollback method to destroy"},{"line_number":23,"context_line":"the guest on the dest host whilst preserving disks on shared"},{"line_number":24,"context_line":"storage and updates the port bindings to point back at the"},{"line_number":25,"context_line":"source host."}],"source_content_type":"text/x-gerrit-commit-message","patch_set":1,"id":"5f7c97a3_35e490df","line":22,"range":{"start_line":21,"start_character":0,"end_line":22,"end_character":43},"in_reply_to":"5f7c97a3_d56bdc84","updated":"2018-08-02 14:06:11.000000000","message":"\u003e Fail after driver.rebuild(): There\u0027s (by design) only less consequential stuff here like updating instance metadata, notifications. Should probably succeed migration, put instance in error state? You\u0027d know better than me what\u0027s most appropriate for this state. But... the instance will be fully operational on the destination, so rolling it back seems inappropriate.\n\nThis is the case the bug is talking about. I think if we don\u0027t rollback here, what\u0027d probably do is:\n\n1. set the migration status to error (like we do today)\n2. update the instance host to the dest and status to error so the user can rebuild and when they do rebuild, it\u0027s a rebuild on the dest (not an evacuate from the source)\n\nThe tricky thing is if we failed post-spawn b/c of an UnexpectedTaskStateError or something when saving the instance status, then changing the instance status to ERROR and setting the host might also fail, idk - we might need to catch that error and refresh the instance status or something. I suspect this might be a result of a bug which Lee already fixed:\n\nhttps://bugs.launchpad.net/nova/+bug/1764883\n\nBut in the latest comments on this bug, the reporter says that driver.spawn() failed:\n\nFirst time,instance \"vmtest\" evacuated failed on B with libvirt error.\n\nKey word from log is: libvirtError: Did not receive a reply. Possible causes include: the remote application did not send a reply, the message bus security policy blocked the reply, the reply timeout expired, or the network connection was broken.\n\n\n\nLog\n\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\n2018-07-31 18:20:01.464 30831 INFO nova.compute.manager [req-0f721cbe-f8d4-43fb-a4c5-944fe13baff4 d8aeb1794cbb47259ad126a7310c65ef 9f86737686af478487424ebb31ea2be6 - - -] [instanc\ne: 6470fe19-f58e-4bca-a8a8-0048165f2dc4] Successfully reverted task state from rebuild_spawning on failure for instance.\n2018-07-31 18:20:01.469 30831 ERROR oslo_messaging.rpc.dispatcher [req-0f721cbe-f8d4-43fb-a4c5-944fe13baff4 d8aeb1794cbb47259ad126a7310c65ef 9f86737686af478487424ebb31ea2be6 - - -\n] Exception during message handling: Did not receive a reply. Possible causes include: the remote application did not send a reply, the message bus security policy blocked the rep\nly, the reply timeout expired, or the network connection was broken.\n2018-07-31 18:20:01.469 30831 ERROR oslo_messaging.rpc.dispatcher Traceback (most recent call last):\n2018-07-31 18:20:01.469 30831 ERROR oslo_messaging.rpc.dispatcher   File \"/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py\", line 138, in _dispatch_and_reply\n2018-07-31 18:20:01.469 30831 ERROR oslo_messaging.rpc.dispatcher     incoming.message))\n2018-07-31 18:20:01.469 30831 ERROR oslo_messaging.rpc.dispatcher   File \"/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py\", line 183, in _dispatch\n2018-07-31 18:20:01.469 30831 ERROR oslo_messaging.rpc.dispatcher     return self._do_dispatch(endpoint, method, ctxt, args)\n2018-07-31 18:20:01.469 30831 ERROR oslo_messaging.rpc.dispatcher   File \"/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py\", line 127, in _do_dispatch\n2018-07-31 18:20:01.469 30831 ERROR oslo_messaging.rpc.dispatcher     result \u003d func(ctxt, **new_args)\n2018-07-31 18:20:01.469 30831 ERROR oslo_messaging.rpc.dispatcher   File \"/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py\", line 150, in inner\n2018-07-31 18:20:01.469 30831 ERROR oslo_messaging.rpc.dispatcher     return func(*args, **kwargs)\n2018-07-31 18:20:01.469 30831 ERROR oslo_messaging.rpc.dispatcher   File \"/usr/lib/python2.7/site-packages/nova/exception.py\", line 110, in wrapped\n2018-07-31 18:20:01.469 30831 ERROR oslo_messaging.rpc.dispatcher     payload)\n2018-07-31 18:20:01.469 30831 ERROR oslo_messaging.rpc.dispatcher   File \"/usr/lib/python2.7/site-packages/oslo_utils/excutils.py\", line 220, in __exit__\n2018-07-31 18:20:01.469 30831 ERROR oslo_messaging.rpc.dispatcher     self.force_reraise()\n2018-07-31 18:20:01.469 30831 ERROR oslo_messaging.rpc.dispatcher   File \"/usr/lib/python2.7/site-packages/oslo_utils/excutils.py\", line 196, in force_reraise\n2018-07-31 18:20:01.469 30831 ERROR oslo_messaging.rpc.dispatcher     six.reraise(self.type_, self.value, self.tb)\n2018-07-31 18:20:01.469 30831 ERROR oslo_messaging.rpc.dispatcher   File \"/usr/lib/python2.7/site-packages/nova/exception.py\", line 89, in wrapped\n2018-07-31 18:20:01.469 30831 ERROR oslo_messaging.rpc.dispatcher     return f(self, context, *args, **kw)\n2018-07-31 18:20:01.469 30831 ERROR oslo_messaging.rpc.dispatcher   File \"/usr/lib/python2.7/site-packages/nova/compute/manager.py\", line 359, in decorated_function\n2018-07-31 18:20:01.469 30831 ERROR oslo_messaging.rpc.dispatcher     LOG.warning(msg, e, instance\u003dinstance)\n2018-07-31 18:20:01.469 30831 ERROR oslo_messaging.rpc.dispatcher   File \"/usr/lib/python2.7/site-packages/oslo_utils/excutils.py\", line 220, in __exit__\n2018-07-31 18:20:01.469 30831 ERROR oslo_messaging.rpc.dispatcher     self.force_reraise()\n2018-07-31 18:20:01.469 30831 ERROR oslo_messaging.rpc.dispatcher   File \"/usr/lib/python2.7/site-packages/oslo_utils/excutils.py\", line 196, in force_reraise\n2018-07-31 18:20:01.469 30831 ERROR oslo_messaging.rpc.dispatcher     six.reraise(self.type_, self.value, self.tb)\n2018-07-31 18:20:01.469 30831 ERROR oslo_messaging.rpc.dispatcher   File \"/usr/lib/python2.7/site-packages/nova/compute/manager.py\", line 328, in decorated_function\n2018-07-31 18:20:01.469 30831 ERROR oslo_messaging.rpc.dispatcher     return function(self, context, *args, **kwargs)\n2018-07-31 18:20:01.469 30831 ERROR oslo_messaging.rpc.dispatcher   File \"/usr/lib/python2.7/site-packages/nova/compute/manager.py\", line 409, in decorated_function\n2018-07-31 18:20:01.469 30831 ERROR oslo_messaging.rpc.dispatcher     return function(self, context, *args, **kwargs)\n2018-07-31 18:20:01.469 30831 ERROR oslo_messaging.rpc.dispatcher   File \"/usr/lib/python2.7/site-packages/nova/compute/manager.py\", line 387, in decorated_function\n2018-07-31 18:20:01.469 30831 ERROR oslo_messaging.rpc.dispatcher     kwargs[\u0027instance\u0027], e, sys.exc_info())\n2018-07-31 18:20:01.469 30831 ERROR oslo_messaging.rpc.dispatcher   File \"/usr/lib/python2.7/site-packages/oslo_utils/excutils.py\", line 220, in __exit__\n2018-07-31 18:20:01.469 30831 ERROR oslo_messaging.rpc.dispatcher     self.force_reraise()\n2018-07-31 18:20:01.469 30831 ERROR oslo_messaging.rpc.dispatcher   File \"/usr/lib/python2.7/site-packages/oslo_utils/excutils.py\", line 196, in force_reraise\n2018-07-31 18:20:01.469 30831 ERROR oslo_messaging.rpc.dispatcher     six.reraise(self.type_, self.value, self.tb)\n2018-07-31 18:20:01.469 30831 ERROR oslo_messaging.rpc.dispatcher   File \"/usr/lib/python2.7/site-packages/nova/compute/manager.py\", line 375, in decorated_function\n2018-07-31 18:20:01.469 30831 ERROR oslo_messaging.rpc.dispatcher     return function(self, context, *args, **kwargs)\n2018-07-31 18:20:01.469 30831 ERROR oslo_messaging.rpc.dispatcher   File \"/usr/lib/python2.7/site-packages/nova/compute/manager.py\", line 2809, in rebuild_instance\n2018-07-31 18:20:01.469 30831 ERROR oslo_messaging.rpc.dispatcher     bdms, recreate, on_shared_storage, preserve_ephemeral)\n2018-07-31 18:20:01.469 30831 ERROR oslo_messaging.rpc.dispatcher   File \"/usr/lib/python2.7/site-packages/nova/compute/manager.py\", line 2853, in _do_rebuild_instance_with_claim\n2018-07-31 18:20:01.469 30831 ERROR oslo_messaging.rpc.dispatcher     self._do_rebuild_instance(*args, **kwargs)\n2018-07-31 18:20:01.469 30831 ERROR oslo_messaging.rpc.dispatcher   File \"/usr/lib/python2.7/site-packages/nova/compute/manager.py\", line 2969, in _do_rebuild_instance\n2018-07-31 18:20:01.469 30831 ERROR oslo_messaging.rpc.dispatcher     self._rebuild_default_impl(**kwargs)\n2018-07-31 18:20:01.469 30831 ERROR oslo_messaging.rpc.dispatcher   File \"/usr/lib/python2.7/site-packages/nova/compute/manager.py\", line 2734, in _rebuild_default_impl\n2018-07-31 18:20:01.469 30831 ERROR oslo_messaging.rpc.dispatcher     block_device_info\u003dnew_block_device_info)\n2018-07-31 18:20:01.469 30831 ERROR oslo_messaging.rpc.dispatcher   File \"/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py\", line 2779, in spawn\n2018-07-31 18:20:01.469 30831 ERROR oslo_messaging.rpc.dispatcher     block_device_info\u003dblock_device_info)\n2018-07-31 18:20:01.469 30831 ERROR oslo_messaging.rpc.dispatcher   File \"/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py\", line 4923, in _create_domain_and_network\n2018-07-31 18:20:01.469 30831 ERROR oslo_messaging.rpc.dispatcher     xml, pause\u003dpause, power_on\u003dpower_on)\n2018-07-31 18:20:01.469 30831 ERROR oslo_messaging.rpc.dispatcher   File \"/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py\", line 4854, in _create_domain\n2018-07-31 18:20:01.469 30831 ERROR oslo_messaging.rpc.dispatcher     guest.launch(pause\u003dpause)\n2018-07-31 18:20:01.469 30831 ERROR oslo_messaging.rpc.dispatcher   File \"/usr/lib/python2.7/site-packages/nova/virt/libvirt/guest.py\", line 142, in launch\n2018-07-31 18:20:01.469 30831 ERROR oslo_messaging.rpc.dispatcher     self._encoded_xml, errors\u003d\u0027ignore\u0027)\n2018-07-31 18:20:01.469 30831 ERROR oslo_messaging.rpc.dispatcher   File \"/usr/lib/python2.7/site-packages/oslo_utils/excutils.py\", line 220, in __exit__\n2018-07-31 18:20:01.469 30831 ERROR oslo_messaging.rpc.dispatcher     self.force_reraise()\n2018-07-31 18:20:01.469 30831 ERROR oslo_messaging.rpc.dispatcher   File \"/usr/lib/python2.7/site-packages/oslo_utils/excutils.py\", line 196, in force_reraise\n2018-07-31 18:20:01.469 30831 ERROR oslo_messaging.rpc.dispatcher     six.reraise(self.type_, self.value, self.tb)\n2018-07-31 18:20:01.469 30831 ERROR oslo_messaging.rpc.dispatcher   File \"/usr/lib/python2.7/site-packages/nova/virt/libvirt/guest.py\", line 137, in launch\n2018-07-31 18:20:01.469 30831 ERROR oslo_messaging.rpc.dispatcher     return self._domain.createWithFlags(flags)\n2018-07-31 18:20:01.469 30831 ERROR oslo_messaging.rpc.dispatcher   File \"/usr/lib/python2.7/site-packages/eventlet/tpool.py\", line 186, in doit\n2018-07-31 18:20:01.469 30831 ERROR oslo_messaging.rpc.dispatcher     result \u003d proxy_call(self._autowrap, f, *args, **kwargs)\n2018-07-31 18:20:01.469 30831 ERROR oslo_messaging.rpc.dispatcher   File \"/usr/lib/python2.7/site-packages/eventlet/tpool.py\", line 144, in proxy_call\n2018-07-31 18:20:01.469 30831 ERROR oslo_messaging.rpc.dispatcher     rv \u003d execute(f, *args, **kwargs)\n2018-07-31 18:20:01.469 30831 ERROR oslo_messaging.rpc.dispatcher   File \"/usr/lib/python2.7/site-packages/eventlet/tpool.py\", line 125, in execute\n2018-07-31 18:20:01.469 30831 ERROR oslo_messaging.rpc.dispatcher     six.reraise(c, e, tb)\n2018-07-31 18:20:01.469 30831 ERROR oslo_messaging.rpc.dispatcher   File \"/usr/lib/python2.7/site-packages/eventlet/tpool.py\", line 83, in tworker\n2018-07-31 18:20:01.469 30831 ERROR oslo_messaging.rpc.dispatcher     rv \u003d meth(*args, **kwargs)\n2018-07-31 18:20:01.469 30831 ERROR oslo_messaging.rpc.dispatcher   File \"/usr/lib64/python2.7/site-packages/libvirt.py\", line 1059, in createWithFlags\n2018-07-31 18:20:01.469 30831 ERROR oslo_messaging.rpc.dispatcher     if ret \u003d\u003d -1: raise libvirtError (\u0027virDomainCreateWithFlags() failed\u0027, dom\u003dself)\n2018-07-31 18:20:01.469 30831 ERROR oslo_messaging.rpc.dispatcher libvirtError: Did not receive a reply. Possible causes include: the remote application did not send a reply, the \nmessage bus security policy blocked the reply, the reply timeout expired, or the network connection was broken.\n\nSo I guess in that case we need to rollback in the driver.","commit_id":"d20fb07c6a043b2ae733b6e8b681ffc4ec49cf84"}],"nova/compute/manager.py":[{"author":{"_account_id":9555,"name":"Matthew Booth","email":"mbooth@redhat.com","username":"MatthewBooth"},"change_message_id":"dc26c5caecd38d9385f372d3f4bad954be55e660","unresolved":false,"context_lines":[{"line_number":2943,"context_line":"        :param recreate: True if the instance is being recreated (e.g. the"},{"line_number":2944,"context_line":"            hypervisor it was on failed) - cleanup of old state will be"},{"line_number":2945,"context_line":"            skipped."},{"line_number":2946,"context_line":"        :param on_shared_storage: True if instance files on shared storage."},{"line_number":2947,"context_line":"                                  If not provided then information from the"},{"line_number":2948,"context_line":"                                  driver will be used to decide if the instance"},{"line_number":2949,"context_line":"                                  files are available or not on the target host"}],"source_content_type":"text/x-python","patch_set":1,"id":"5f7c97a3_3c1d5e60","line":2946,"range":{"start_line":2946,"start_character":15,"end_line":2946,"end_character":32},"updated":"2018-08-02 11:24:37.000000000","message":"I think any use of this parameter is a bug. Details below.","commit_id":"d20fb07c6a043b2ae733b6e8b681ffc4ec49cf84"},{"author":{"_account_id":11604,"name":"sean mooney","email":"smooney@redhat.com","username":"sean-k-mooney"},"change_message_id":"81b8070f54f50301c466ae6e8fa065dd48825d48","unresolved":false,"context_lines":[{"line_number":2943,"context_line":"        :param recreate: True if the instance is being recreated (e.g. the"},{"line_number":2944,"context_line":"            hypervisor it was on failed) - cleanup of old state will be"},{"line_number":2945,"context_line":"            skipped."},{"line_number":2946,"context_line":"        :param on_shared_storage: True if instance files on shared storage."},{"line_number":2947,"context_line":"                                  If not provided then information from the"},{"line_number":2948,"context_line":"                                  driver will be used to decide if the instance"},{"line_number":2949,"context_line":"                                  files are available or not on the target host"}],"source_content_type":"text/x-python","patch_set":1,"id":"5f7c97a3_02d92682","line":2946,"range":{"start_line":2946,"start_character":15,"end_line":2946,"end_character":32},"updated":"2018-08-01 22:42:02.000000000","message":"this is expected to be either true of false not none right?","commit_id":"d20fb07c6a043b2ae733b6e8b681ffc4ec49cf84"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"dbae3ac77be4b032f7246128c342c0455e1756dc","unresolved":false,"context_lines":[{"line_number":2943,"context_line":"        :param recreate: True if the instance is being recreated (e.g. the"},{"line_number":2944,"context_line":"            hypervisor it was on failed) - cleanup of old state will be"},{"line_number":2945,"context_line":"            skipped."},{"line_number":2946,"context_line":"        :param on_shared_storage: True if instance files on shared storage."},{"line_number":2947,"context_line":"                                  If not provided then information from the"},{"line_number":2948,"context_line":"                                  driver will be used to decide if the instance"},{"line_number":2949,"context_line":"                                  files are available or not on the target host"}],"source_content_type":"text/x-python","patch_set":1,"id":"5f7c97a3_c5b01844","line":2946,"range":{"start_line":2946,"start_character":15,"end_line":2946,"end_character":32},"in_reply_to":"5f7c97a3_02d92682","updated":"2018-08-01 23:01:41.000000000","message":"No, it can be None to mean \u0027you figure it out for me\u0027:\n\nhttps://docs.openstack.org/nova/latest/reference/api-microversion-history.html#id12","commit_id":"d20fb07c6a043b2ae733b6e8b681ffc4ec49cf84"},{"author":{"_account_id":11604,"name":"sean mooney","email":"smooney@redhat.com","username":"sean-k-mooney"},"change_message_id":"81b8070f54f50301c466ae6e8fa065dd48825d48","unresolved":false,"context_lines":[{"line_number":2999,"context_line":"            else:"},{"line_number":3000,"context_line":"                scheduled_node \u003d instance.node"},{"line_number":3001,"context_line":""},{"line_number":3002,"context_line":"        if evacuate and on_shared_storage is None:"},{"line_number":3003,"context_line":"            LOG.debug(\u0027on_shared_storage is not provided, using driver \u0027"},{"line_number":3004,"context_line":"                      \u0027information to decide if the instance needs to \u0027"},{"line_number":3005,"context_line":"                      \u0027be evacuated\u0027)"}],"source_content_type":"text/x-python","patch_set":1,"id":"5f7c97a3_c2b00e44","line":3002,"range":{"start_line":3002,"start_character":42,"end_line":3002,"end_character":50},"updated":"2018-08-01 22:42:02.000000000","message":"is this needed? the name on_shared_storage reads as if its a bool which would seam to agree with line 2946 above","commit_id":"d20fb07c6a043b2ae733b6e8b681ffc4ec49cf84"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"dbae3ac77be4b032f7246128c342c0455e1756dc","unresolved":false,"context_lines":[{"line_number":2999,"context_line":"            else:"},{"line_number":3000,"context_line":"                scheduled_node \u003d instance.node"},{"line_number":3001,"context_line":""},{"line_number":3002,"context_line":"        if evacuate and on_shared_storage is None:"},{"line_number":3003,"context_line":"            LOG.debug(\u0027on_shared_storage is not provided, using driver \u0027"},{"line_number":3004,"context_line":"                      \u0027information to decide if the instance needs to \u0027"},{"line_number":3005,"context_line":"                      \u0027be evacuated\u0027)"}],"source_content_type":"text/x-python","patch_set":1,"id":"5f7c97a3_a5839c54","line":3002,"range":{"start_line":3002,"start_character":42,"end_line":3002,"end_character":50},"in_reply_to":"5f7c97a3_c2b00e44","updated":"2018-08-01 23:01:41.000000000","message":"same","commit_id":"d20fb07c6a043b2ae733b6e8b681ffc4ec49cf84"},{"author":{"_account_id":11604,"name":"sean mooney","email":"smooney@redhat.com","username":"sean-k-mooney"},"change_message_id":"81b8070f54f50301c466ae6e8fa065dd48825d48","unresolved":false,"context_lines":[{"line_number":3057,"context_line":"                    rt.delete_allocation_for_evacuated_instance("},{"line_number":3058,"context_line":"                        context, instance, scheduled_node,"},{"line_number":3059,"context_line":"                        node_type\u003d\u0027destination\u0027)"},{"line_number":3060,"context_line":"                    # If we actually spawned a guest on the hypervisor, remove"},{"line_number":3061,"context_line":"                    # it otherwise trying to evacuate to this host again for"},{"line_number":3062,"context_line":"                    # the same instance could fail in _check_instance_exists."},{"line_number":3063,"context_line":"                    if self.driver.instance_exists(instance):"}],"source_content_type":"text/x-python","patch_set":1,"id":"5f7c97a3_a2d15258","line":3060,"range":{"start_line":3060,"start_character":37,"end_line":3060,"end_character":44},"updated":"2018-08-01 22:42:02.000000000","message":"the guest would be in the paused state right not running if it did exisit at this point?","commit_id":"d20fb07c6a043b2ae733b6e8b681ffc4ec49cf84"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"dbae3ac77be4b032f7246128c342c0455e1756dc","unresolved":false,"context_lines":[{"line_number":3057,"context_line":"                    rt.delete_allocation_for_evacuated_instance("},{"line_number":3058,"context_line":"                        context, instance, scheduled_node,"},{"line_number":3059,"context_line":"                        node_type\u003d\u0027destination\u0027)"},{"line_number":3060,"context_line":"                    # If we actually spawned a guest on the hypervisor, remove"},{"line_number":3061,"context_line":"                    # it otherwise trying to evacuate to this host again for"},{"line_number":3062,"context_line":"                    # the same instance could fail in _check_instance_exists."},{"line_number":3063,"context_line":"                    if self.driver.instance_exists(instance):"}],"source_content_type":"text/x-python","patch_set":1,"id":"5f7c97a3_0594d08a","line":3060,"range":{"start_line":3060,"start_character":37,"end_line":3060,"end_character":44},"in_reply_to":"5f7c97a3_a2d15258","updated":"2018-08-01 23:01:41.000000000","message":"Why wouldn\u0027t it be running?","commit_id":"d20fb07c6a043b2ae733b6e8b681ffc4ec49cf84"},{"author":{"_account_id":9555,"name":"Matthew Booth","email":"mbooth@redhat.com","username":"MatthewBooth"},"change_message_id":"0322e0f9ced3643cdbd3522cc3ed7985d0959e51","unresolved":false,"context_lines":[{"line_number":3057,"context_line":"                    rt.delete_allocation_for_evacuated_instance("},{"line_number":3058,"context_line":"                        context, instance, scheduled_node,"},{"line_number":3059,"context_line":"                        node_type\u003d\u0027destination\u0027)"},{"line_number":3060,"context_line":"                    # If we actually spawned a guest on the hypervisor, remove"},{"line_number":3061,"context_line":"                    # it otherwise trying to evacuate to this host again for"},{"line_number":3062,"context_line":"                    # the same instance could fail in _check_instance_exists."},{"line_number":3063,"context_line":"                    if self.driver.instance_exists(instance):"}],"source_content_type":"text/x-python","patch_set":1,"id":"5f7c97a3_feefdef6","line":3060,"range":{"start_line":3060,"start_character":37,"end_line":3060,"end_character":44},"in_reply_to":"5f7c97a3_a2d15258","updated":"2018-08-02 10:21:53.000000000","message":"You might be mixing up live migration here? This is evacuation, so we just create the domain normally.","commit_id":"d20fb07c6a043b2ae733b6e8b681ffc4ec49cf84"},{"author":{"_account_id":11604,"name":"sean mooney","email":"smooney@redhat.com","username":"sean-k-mooney"},"change_message_id":"81b8070f54f50301c466ae6e8fa065dd48825d48","unresolved":false,"context_lines":[{"line_number":3092,"context_line":"                context, instance.uuid)"},{"line_number":3093,"context_line":"        block_device_info \u003d self._get_instance_block_device_info("},{"line_number":3094,"context_line":"            context, instance, bdms\u003dbdms)"},{"line_number":3095,"context_line":"        self.driver.destroy("},{"line_number":3096,"context_line":"            context, instance, network_info,"},{"line_number":3097,"context_line":"            block_device_info\u003dblock_device_info,"},{"line_number":3098,"context_line":"            destroy_disks\u003dnot on_shared_storage)"}],"source_content_type":"text/x-python","patch_set":1,"id":"5f7c97a3_0285a6ad","line":3095,"range":{"start_line":3095,"start_character":20,"end_line":3095,"end_character":27},"updated":"2018-08-01 22:42:02.000000000","message":"will this do unplug on the dest host implicitly?\n\ni think yes but if not we have to unplug before we bind the ports back to the source on 3099","commit_id":"d20fb07c6a043b2ae733b6e8b681ffc4ec49cf84"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"dbae3ac77be4b032f7246128c342c0455e1756dc","unresolved":false,"context_lines":[{"line_number":3092,"context_line":"                context, instance.uuid)"},{"line_number":3093,"context_line":"        block_device_info \u003d self._get_instance_block_device_info("},{"line_number":3094,"context_line":"            context, instance, bdms\u003dbdms)"},{"line_number":3095,"context_line":"        self.driver.destroy("},{"line_number":3096,"context_line":"            context, instance, network_info,"},{"line_number":3097,"context_line":"            block_device_info\u003dblock_device_info,"},{"line_number":3098,"context_line":"            destroy_disks\u003dnot on_shared_storage)"}],"source_content_type":"text/x-python","patch_set":1,"id":"5f7c97a3_c589f86f","line":3095,"range":{"start_line":3095,"start_character":20,"end_line":3095,"end_character":27},"in_reply_to":"5f7c97a3_0285a6ad","updated":"2018-08-01 23:01:41.000000000","message":"Yes it will destroy_vifs (unplug) by default. For the libvirt driver, anyway, destroy() calls cleanup() which unplugs by default:\n\nhttps://github.com/openstack/nova/blob/d4dbb42593893c1d1ed51a127b7183a314bcac2c/nova/virt/libvirt/driver.py#L998","commit_id":"d20fb07c6a043b2ae733b6e8b681ffc4ec49cf84"},{"author":{"_account_id":9555,"name":"Matthew Booth","email":"mbooth@redhat.com","username":"MatthewBooth"},"change_message_id":"dc26c5caecd38d9385f372d3f4bad954be55e660","unresolved":false,"context_lines":[{"line_number":3095,"context_line":"        self.driver.destroy("},{"line_number":3096,"context_line":"            context, instance, network_info,"},{"line_number":3097,"context_line":"            block_device_info\u003dblock_device_info,"},{"line_number":3098,"context_line":"            destroy_disks\u003dnot on_shared_storage)"},{"line_number":3099,"context_line":"        # Update the port bindings to point back at the source host."},{"line_number":3100,"context_line":"        self.network_api.setup_instance_network_on_host("},{"line_number":3101,"context_line":"            context, instance, instance.host, migration)"}],"source_content_type":"text/x-python","patch_set":1,"id":"5f7c97a3_c1d7a3bc","line":3098,"range":{"start_line":3098,"start_character":12,"end_line":3098,"end_character":47},"updated":"2018-08-02 11:24:37.000000000","message":"If the deployment uses ceph this still isn\u0027t going to cleanup the instance directory on the failed target. I don\u0027t *think* this results in a major failure if re-evacuating to the same host, but I\u0027ll bet there\u0027s weird side-effects and really obtuse bugs if we look hard enough.\n\nThe problem is that we calculated on_shared_storage earlier as LibvirtDriver.instance_on_disk(instance) which, if we\u0027re on ceph, always returns true (I\u0027ll get back to this). LibvirtDriver.cleanup() (called via destroy) will only remove the instance directory if either destroy_disks\u003dTrue, or we passed migrate_data with is_shared_block_storage\u003dTrue (why isn\u0027t this testing shared_instance\u003dFalse?).\n\nThe fundamental issue here is that, for the libvirt driver, the type of \u0027on_shared_storage\u0027 should be (instance_shared:bool, block_shared:bool) which fundamentally can\u0027t be represented as (shared:bool). So essentially anywhere outside driver code which refers to \u0027shared storage\u0027 as a boolean, there\u0027s a reasonable chance it\u0027s a bug. Other drivers may have different requirements which, incidentally, is also why LiveMigrateData is driver-specific.","commit_id":"d20fb07c6a043b2ae733b6e8b681ffc4ec49cf84"}]}