)]}'
{"/COMMIT_MSG":[{"author":{"_account_id":8556,"name":"Ghanshyam Maan","display_name":"Ghanshyam Maan","email":"gmaan.os14@gmail.com","username":"ghanshyam"},"change_message_id":"2c1552fed1f74bfbc79f26ca89972df526d75913","unresolved":true,"context_lines":[{"line_number":11,"context_line":"returns. A message waiting there is not an in-progress task yet: it is"},{"line_number":12,"context_line":"only tracked once dispatched to the manager."},{"line_number":13,"context_line":""},{"line_number":14,"context_line":"The scheduler and conductor managers shut the thread pool executors"},{"line_number":15,"context_line":"down at the end of graceful_shutdown(), which Service.stop() runs"},{"line_number":16,"context_line":"between the RPC server stop() and wait(). _wait_for_in_progress_tasks()"},{"line_number":17,"context_line":"can return while such messages are pending, and once one reaches the"},{"line_number":18,"context_line":"manager, get_executor() raises:"},{"line_number":19,"context_line":""},{"line_number":20,"context_line":"  RuntimeError: Cannot create the cell_worker thread pool executor"},{"line_number":21,"context_line":"  because shutdown has been started."},{"line_number":22,"context_line":""},{"line_number":23,"context_line":"For select_destinations, conductor then sets the instance to ERROR"},{"line_number":24,"context_line":"without rescheduling."}],"source_content_type":"text/x-gerrit-commit-message","patch_set":1,"id":"e6d9dbac_554f4d9e","line":21,"range":{"start_line":14,"start_character":0,"end_line":21,"end_character":36},"updated":"2026-10-08 03:05:01.000000000","message":"executor raising the error for the new task submission is the protection we have and if manager is shutdown then we should not expect new task to arrive and executor execute it. i added my comment in the reproted bug, I think we are missing the main root cause of the issue and moving executors shutdown late will hide such issue? https://bugs.launchpad.net/nova/+bug/2169804/comments/4","commit_id":"9a02d39edd99a72e7909662dacacc1be137bac42"},{"author":{"_account_id":8556,"name":"Ghanshyam Maan","display_name":"Ghanshyam Maan","email":"gmaan.os14@gmail.com","username":"ghanshyam"},"change_message_id":"cff8ed498d2eeb5e0083476a103b3ceb65511bf0","unresolved":false,"context_lines":[{"line_number":11,"context_line":"returns. A message waiting there is not an in-progress task yet: it is"},{"line_number":12,"context_line":"only tracked once dispatched to the manager."},{"line_number":13,"context_line":""},{"line_number":14,"context_line":"The scheduler and conductor managers shut the thread pool executors"},{"line_number":15,"context_line":"down at the end of graceful_shutdown(), which Service.stop() runs"},{"line_number":16,"context_line":"between the RPC server stop() and wait(). _wait_for_in_progress_tasks()"},{"line_number":17,"context_line":"can return while such messages are pending, and once one reaches the"},{"line_number":18,"context_line":"manager, get_executor() raises:"},{"line_number":19,"context_line":""},{"line_number":20,"context_line":"  RuntimeError: Cannot create the cell_worker thread pool executor"},{"line_number":21,"context_line":"  because shutdown has been started."},{"line_number":22,"context_line":""},{"line_number":23,"context_line":"For select_destinations, conductor then sets the instance to ERROR"},{"line_number":24,"context_line":"without rescheduling."}],"source_content_type":"text/x-gerrit-commit-message","patch_set":1,"id":"e455670d_146da253","line":21,"range":{"start_line":14,"start_character":0,"end_line":21,"end_character":36},"in_reply_to":"e6d9dbac_554f4d9e","updated":"2026-10-09 03:09:16.000000000","message":"after discussing on bug, i agree that if rpc thread executor is slow to dispatch the message then manager can miss those new tasks to track and shutdown early.","commit_id":"9a02d39edd99a72e7909662dacacc1be137bac42"}],"/PATCHSET_LEVEL":[{"author":{"_account_id":8556,"name":"Ghanshyam Maan","display_name":"Ghanshyam Maan","email":"gmaan.os14@gmail.com","username":"ghanshyam"},"change_message_id":"2c1552fed1f74bfbc79f26ca89972df526d75913","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":1,"id":"c008a082_c5f0fc24","updated":"2026-10-08 03:05:01.000000000","message":"I do not deny the bug but we need to find the root cause. moving executor shutdown can solve it but then it break the task tracking design.","commit_id":"9a02d39edd99a72e7909662dacacc1be137bac42"},{"author":{"_account_id":8556,"name":"Ghanshyam Maan","display_name":"Ghanshyam Maan","email":"gmaan.os14@gmail.com","username":"ghanshyam"},"change_message_id":"cff8ed498d2eeb5e0083476a103b3ceb65511bf0","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":1,"id":"e83ef985_6bddfe64","updated":"2026-10-09 03:09:16.000000000","message":"also, can you rebase this on top of my reproducer test by https://review.opendev.org/c/openstack/nova/+/1009630\n\nand with fix we can modify that reproducer to show build requires are passing now.\n\nand let\u0027s fix it in this single change so that we can backport this in stable/2026.2 eaisly","commit_id":"9a02d39edd99a72e7909662dacacc1be137bac42"},{"author":{"_account_id":39509,"name":"Lucas HENRY","display_name":"Lucas HENRY","email":"lucas.henry@ovhcloud.com","username":"polyedre"},"change_message_id":"4c1cd94be5dbf1ac0338e63060430df58e60a542","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":1,"id":"b57882b9_4064114d","in_reply_to":"c008a082_c5f0fc24","updated":"2026-10-12 13:41:07.000000000","message":"I think the root cause is that while we stop receiving new RabbitMQ messages, we do not wait for all the already-received messages to be converted to tasks before starting the shutdown of the Manager.\n\nI was able to test","commit_id":"9a02d39edd99a72e7909662dacacc1be137bac42"},{"author":{"_account_id":8556,"name":"Ghanshyam Maan","display_name":"Ghanshyam Maan","email":"gmaan.os14@gmail.com","username":"ghanshyam"},"change_message_id":"4096d3c09005ec854251019926780f3e3f7f6158","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":2,"id":"5496126c_edefb4eb","updated":"2026-10-09 17:51:19.000000000","message":"Thanks Lucas for updating and squashing it in single change. a few comments inline.","commit_id":"f17e39967e72e18e2f5369f1a06d7cb28c17b807"},{"author":{"_account_id":39509,"name":"Lucas HENRY","display_name":"Lucas HENRY","email":"lucas.henry@ovhcloud.com","username":"polyedre"},"change_message_id":"4c1cd94be5dbf1ac0338e63060430df58e60a542","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":3,"id":"4614b33f_a3119115","updated":"2026-10-12 13:41:07.000000000","message":"I reworked the PatchSet according to your comments. It was no longer required to move the executor shutdown outside of the manager shutdown, so I dropped the move. Waiting for the RPCserver to finish its shutdown is enough.","commit_id":"f8fbef15ebe1ded0c0b6562572d3465c078a5235"}],"nova/manager.py":[{"author":{"_account_id":8556,"name":"Ghanshyam Maan","display_name":"Ghanshyam Maan","email":"gmaan.os14@gmail.com","username":"ghanshyam"},"change_message_id":"4096d3c09005ec854251019926780f3e3f7f6158","unresolved":true,"context_lines":[{"line_number":291,"context_line":"        deadline \u003d time.monotonic() + timeout"},{"line_number":292,"context_line":""},{"line_number":293,"context_line":"        with self._in_progress_task_cond:"},{"line_number":294,"context_line":"            if self._in_progress_tasks:"},{"line_number":295,"context_line":"                LOG.info("},{"line_number":296,"context_line":"                    \u0027Graceful shutdown initiated with %d in-progress tasks: \u0027"},{"line_number":297,"context_line":"                    \u0027%s\u0027,"},{"line_number":298,"context_line":"                    len(self._in_progress_tasks),"},{"line_number":299,"context_line":"                    self._summarize_in_progress_tasks())"},{"line_number":300,"context_line":""},{"line_number":301,"context_line":"            while self._in_progress_tasks:"},{"line_number":302,"context_line":"                remaining_time \u003d deadline - time.monotonic()"}],"source_content_type":"text/x-python","patch_set":2,"id":"a78952ba_8eb21acf","line":299,"range":{"start_line":294,"start_character":0,"end_line":299,"end_character":56},"updated":"2026-10-09 17:51:19.000000000","message":"I will say let\u0027s not log here because:\n1. Graceful shutdown is initiated well before and rpcserver.wait() already started finishing the in-progress tasks so saying \"Graceful shutdown initiated\" can be confusing here.\n2. we can achieve the first log of in-progress tasks here if we move L316 logs before L314 (self._in_progress_task_cond.wait).\n\nThat way we will be able to log them at start and at regular interval.","commit_id":"f17e39967e72e18e2f5369f1a06d7cb28c17b807"},{"author":{"_account_id":39509,"name":"Lucas HENRY","display_name":"Lucas HENRY","email":"lucas.henry@ovhcloud.com","username":"polyedre"},"change_message_id":"4c1cd94be5dbf1ac0338e63060430df58e60a542","unresolved":false,"context_lines":[{"line_number":291,"context_line":"        deadline \u003d time.monotonic() + timeout"},{"line_number":292,"context_line":""},{"line_number":293,"context_line":"        with self._in_progress_task_cond:"},{"line_number":294,"context_line":"            if self._in_progress_tasks:"},{"line_number":295,"context_line":"                LOG.info("},{"line_number":296,"context_line":"                    \u0027Graceful shutdown initiated with %d in-progress tasks: \u0027"},{"line_number":297,"context_line":"                    \u0027%s\u0027,"},{"line_number":298,"context_line":"                    len(self._in_progress_tasks),"},{"line_number":299,"context_line":"                    self._summarize_in_progress_tasks())"},{"line_number":300,"context_line":""},{"line_number":301,"context_line":"            while self._in_progress_tasks:"},{"line_number":302,"context_line":"                remaining_time \u003d deadline - time.monotonic()"}],"source_content_type":"text/x-python","patch_set":2,"id":"aacadae6_c8630af5","line":299,"range":{"start_line":294,"start_character":0,"end_line":299,"end_character":56},"in_reply_to":"a78952ba_8eb21acf","updated":"2026-10-12 13:41:07.000000000","message":"Acknowledged","commit_id":"f17e39967e72e18e2f5369f1a06d7cb28c17b807"}],"nova/service.py":[{"author":{"_account_id":8556,"name":"Ghanshyam Maan","display_name":"Ghanshyam Maan","email":"gmaan.os14@gmail.com","username":"ghanshyam"},"change_message_id":"cff8ed498d2eeb5e0083476a103b3ceb65511bf0","unresolved":true,"context_lines":[{"line_number":465,"context_line":"        if self.rpcserver is not None:"},{"line_number":466,"context_line":"            rpcserver_stopped \u003d self._stop_rpc_server("},{"line_number":467,"context_line":"                self.rpcserver, self.topic)"},{"line_number":468,"context_line":""},{"line_number":469,"context_line":"        # Run manager graceful_shutdown() before waiting for the RPC server"},{"line_number":470,"context_line":"        # to shutdown so that tasks already in progress on self.rpcserver can"},{"line_number":471,"context_line":"        # also be tracked and logged before RPC server wait() finish them."},{"line_number":472,"context_line":"        manager_shutdown_finished \u003d self._run_manager_graceful_shutdown()"},{"line_number":473,"context_line":""},{"line_number":474,"context_line":"        if rpcserver_stopped:"},{"line_number":475,"context_line":"            self._wait_rpc_server(self.rpcserver, self.topic)"}],"source_content_type":"text/x-python","patch_set":1,"id":"a3a555af_4f7d61ae","line":472,"range":{"start_line":468,"start_character":0,"end_line":472,"end_character":73},"updated":"2026-10-09 03:09:16.000000000","message":"let\u0027s move this after rpcserver.wait(). so that we will be able to properly wait and log the remaining in-progress tasks continuously.\n\nall tasks are tracked in the the _record_task_start() and _record_task_end() log for any task is completed or failed. At the end _run_manager_graceful_shutdown() will log the summary of total completed and failed tasks. So tracking and logging of tasks are good even we move it after rpcserver.wait(). Only one case we can improve:\n\nin _wait_for_in_progress_tasks() we can summarize the log when manager does not find any in-progress tasks (means rpcserver.wait() finish them before manager _wait_for_in_progress_tasks)\n\n_wait_for_in_progress_tasks() can be like:\n\n    def _wait_for_in_progress_tasks(self, timeout):\n        \"\"\"Wait for all in-progress tasks to complete.\n\n        Wait until all tasks are finished or timeout.\n\n        :param timeout: Maximum number of seconds to wait.\n        \"\"\"\n        log_interval \u003d 10\n        deadline \u003d time.monotonic() + timeout\n\n        with self._in_progress_task_cond:\n            while self._in_progress_tasks:\n                remaining_time \u003d deadline - time.monotonic()\n                if remaining_time \u003c\u003d 0:\n                    LOG.warning(\n                        \u0027Graceful shutdown timed out with %d tasks still \u0027\n                        \u0027in progress: %s (completed tasks: %d, failed \u0027\n                        \u0027tasks: %d%s)\u0027,\n                        len(self._in_progress_tasks),\n                        self._summarize_in_progress_tasks(),\n                        self._completed_task_count, len(self._failed_tasks),\n                        \u0027: %s\u0027 % self._summarize_failed_tasks()\n                        if self._failed_tasks else \u0027\u0027)\n                    return\n                self._in_progress_task_cond.wait(\n                    timeout\u003dmin(remaining_time, log_interval))\n                if self._in_progress_tasks:\n                    LOG.info(\n                        \u0027Graceful shutdown still waiting for %d in progress \u0027\n                        \u0027tasks: %s\u0027,\n                        len(self._in_progress_tasks),\n                        self._summarize_in_progress_tasks())\n\n        LOG.info(\n            \u0027Graceful shutdown: all in-progress tasks finished. \u0027\n            \u0027Number of completed tasks: %d, failed tasks: %d%s\u0027,\n            self._completed_task_count, len(self._failed_tasks),\n            \u0027: %s\u0027 % self._summarize_failed_tasks()\n            if self._failed_tasks else \u0027\u0027)","commit_id":"9a02d39edd99a72e7909662dacacc1be137bac42"},{"author":{"_account_id":39509,"name":"Lucas HENRY","display_name":"Lucas HENRY","email":"lucas.henry@ovhcloud.com","username":"polyedre"},"change_message_id":"c5853f189dff92b0d46e6ae26483740a440444f4","unresolved":false,"context_lines":[{"line_number":469,"context_line":"        # Run manager graceful_shutdown() before waiting for the RPC server"},{"line_number":470,"context_line":"        # to shutdown so that tasks already in progress on self.rpcserver can"},{"line_number":471,"context_line":"        # also be tracked and logged before RPC server wait() finish them."},{"line_number":472,"context_line":"        manager_shutdown_finished \u003d self._run_manager_graceful_shutdown()"},{"line_number":473,"context_line":""},{"line_number":474,"context_line":"        if rpcserver_stopped:"},{"line_number":475,"context_line":"            self._wait_rpc_server(self.rpcserver, self.topic)"}],"source_content_type":"text/x-python","patch_set":1,"id":"3e40b9cc_ba8f964e","line":472,"in_reply_to":"a3a555af_4f7d61ae","updated":"2026-10-09 12:25:10.000000000","message":"Done in PS2.","commit_id":"9a02d39edd99a72e7909662dacacc1be137bac42"},{"author":{"_account_id":8556,"name":"Ghanshyam Maan","display_name":"Ghanshyam Maan","email":"gmaan.os14@gmail.com","username":"ghanshyam"},"change_message_id":"cff8ed498d2eeb5e0083476a103b3ceb65511bf0","unresolved":true,"context_lines":[{"line_number":456,"context_line":"        LOG.info(\u0027%s service graceful shutdown started.\u0027, self.binary)"},{"line_number":457,"context_line":""},{"line_number":458,"context_line":"        self.manager.set_shutdown_in_progress()"},{"line_number":459,"context_line":""},{"line_number":460,"context_line":"        # This RPC server handles new requests during normal operation. During"},{"line_number":461,"context_line":"        # graceful shutdown, we limit the RPC requests the service can handle."},{"line_number":462,"context_line":"        # So we stop the main RPC server here and let the alternative RPC"},{"line_number":463,"context_line":"        # server handle the remaining requests for the ongoing operations."},{"line_number":464,"context_line":"        rpcserver_stopped \u003d False"},{"line_number":465,"context_line":"        if self.rpcserver is not None:"},{"line_number":466,"context_line":"            rpcserver_stopped \u003d self._stop_rpc_server("},{"line_number":467,"context_line":"                self.rpcserver, self.topic)"},{"line_number":468,"context_line":""},{"line_number":469,"context_line":"        # Run manager graceful_shutdown() before waiting for the RPC server"},{"line_number":470,"context_line":"        # to shutdown so that tasks already in progress on self.rpcserver can"},{"line_number":471,"context_line":"        # also be tracked and logged before RPC server wait() finish them."},{"line_number":472,"context_line":"        manager_shutdown_finished \u003d self._run_manager_graceful_shutdown()"},{"line_number":473,"context_line":""},{"line_number":474,"context_line":"        if rpcserver_stopped:"},{"line_number":475,"context_line":"            self._wait_rpc_server(self.rpcserver, self.topic)"},{"line_number":476,"context_line":""},{"line_number":477,"context_line":"        if self.rpcserver_alt is not None:"},{"line_number":478,"context_line":"            # During graceful shutdown, manager will use this RPC server to"}],"source_content_type":"text/x-python","patch_set":1,"id":"78bdcff8_c495d2b7","line":475,"range":{"start_line":459,"start_character":0,"end_line":475,"end_character":61},"updated":"2026-10-09 03:09:16.000000000","message":"based on above comment, this order can be:\n\n\n        # This RPC server handles new requests during normal operation. During\n        # graceful shutdown, we limit the RPC requests the service can handle.\n        # So we stop and wait the main RPC server here and let the alternative RPC\n        # server handle the remaining requests for the ongoing operations.\n        rpcserver_stopped \u003d False\n        if self.rpcserver is not None:\n            self._shutdown_rpc_server(\n                    self.rpcserver, self.topic)\n\n        self._run_manager_graceful_shutdown()","commit_id":"9a02d39edd99a72e7909662dacacc1be137bac42"},{"author":{"_account_id":39509,"name":"Lucas HENRY","display_name":"Lucas HENRY","email":"lucas.henry@ovhcloud.com","username":"polyedre"},"change_message_id":"c5853f189dff92b0d46e6ae26483740a440444f4","unresolved":false,"context_lines":[{"line_number":472,"context_line":"        manager_shutdown_finished \u003d self._run_manager_graceful_shutdown()"},{"line_number":473,"context_line":""},{"line_number":474,"context_line":"        if rpcserver_stopped:"},{"line_number":475,"context_line":"            self._wait_rpc_server(self.rpcserver, self.topic)"},{"line_number":476,"context_line":""},{"line_number":477,"context_line":"        if self.rpcserver_alt is not None:"},{"line_number":478,"context_line":"            # During graceful shutdown, manager will use this RPC server to"}],"source_content_type":"text/x-python","patch_set":1,"id":"e9bd6e01_35d06291","line":475,"in_reply_to":"78bdcff8_c495d2b7","updated":"2026-10-09 12:25:10.000000000","message":"Done in PS2.","commit_id":"9a02d39edd99a72e7909662dacacc1be137bac42"},{"author":{"_account_id":8556,"name":"Ghanshyam Maan","display_name":"Ghanshyam Maan","email":"gmaan.os14@gmail.com","username":"ghanshyam"},"change_message_id":"cff8ed498d2eeb5e0083476a103b3ceb65511bf0","unresolved":true,"context_lines":[{"line_number":487,"context_line":"        # can need the executors. A manager graceful shutdown still running"},{"line_number":488,"context_line":"        # past its timeout can need them too. The bound is the time the"},{"line_number":489,"context_line":"        # managers reserve out of manager_shutdown_timeout for it."},{"line_number":490,"context_line":"        if manager_shutdown_finished:"},{"line_number":491,"context_line":"            self._run_with_timeout("},{"line_number":492,"context_line":"                \u0027thread pool executors shutdown\u0027,"},{"line_number":493,"context_line":"                thread_pool_factory.shutdown_all_executors,"},{"line_number":494,"context_line":"                min(20, CONF.manager_shutdown_timeout))"},{"line_number":495,"context_line":""},{"line_number":496,"context_line":"        LOG.info(\u0027%s service graceful shutdown finished.\u0027, self.binary)"},{"line_number":497,"context_line":"        super(Service, self).stop()"}],"source_content_type":"text/x-python","patch_set":1,"id":"7aa1dd9f_f9b74082","line":494,"range":{"start_line":490,"start_character":0,"end_line":494,"end_character":55},"updated":"2026-10-09 03:09:16.000000000","message":"i do not think we need to keep timer for this. let\u0027s executors shutdown cleanly and if they takes more time then graceful_shutdown_timeout will make sure that process is terminated by oslo.service stop. we can do it in try and catch block as done in https://review.opendev.org/c/openstack/nova/+/1000211/4/nova/service.py","commit_id":"9a02d39edd99a72e7909662dacacc1be137bac42"},{"author":{"_account_id":39509,"name":"Lucas HENRY","display_name":"Lucas HENRY","email":"lucas.henry@ovhcloud.com","username":"polyedre"},"change_message_id":"c5853f189dff92b0d46e6ae26483740a440444f4","unresolved":false,"context_lines":[{"line_number":491,"context_line":"            self._run_with_timeout("},{"line_number":492,"context_line":"                \u0027thread pool executors shutdown\u0027,"},{"line_number":493,"context_line":"                thread_pool_factory.shutdown_all_executors,"},{"line_number":494,"context_line":"                min(20, CONF.manager_shutdown_timeout))"},{"line_number":495,"context_line":""},{"line_number":496,"context_line":"        LOG.info(\u0027%s service graceful shutdown finished.\u0027, self.binary)"},{"line_number":497,"context_line":"        super(Service, self).stop()"}],"source_content_type":"text/x-python","patch_set":1,"id":"0637299f_d1f672c2","line":494,"in_reply_to":"7aa1dd9f_f9b74082","updated":"2026-10-09 12:25:10.000000000","message":"Done in PS2: plain try/except like 1000211, no timer.","commit_id":"9a02d39edd99a72e7909662dacacc1be137bac42"}],"nova/tests/functional/regressions/test_bug_2169804.py":[{"author":{"_account_id":8556,"name":"Ghanshyam Maan","display_name":"Ghanshyam Maan","email":"gmaan.os14@gmail.com","username":"ghanshyam"},"change_message_id":"4096d3c09005ec854251019926780f3e3f7f6158","unresolved":false,"context_lines":[{"line_number":119,"context_line":"        self.assertTrue("},{"line_number":120,"context_line":"            rpc_wait_started.wait(timeout\u003d30),"},{"line_number":121,"context_line":"            \u0027Timed out waiting for the RPC server to be stopped\u0027)"},{"line_number":122,"context_line":"        self.assertFalse(graceful_shutdown_started.is_set())"},{"line_number":123,"context_line":"        self.assertFalse(executors_shutdown.is_set())"},{"line_number":124,"context_line":""},{"line_number":125,"context_line":"        rpc_proceed.set()"},{"line_number":126,"context_line":"        stop_thread.join(timeout\u003d60)"}],"source_content_type":"text/x-python","patch_set":2,"id":"b9b61434_33145af6","line":123,"range":{"start_line":122,"start_character":0,"end_line":123,"end_character":53},"updated":"2026-10-09 17:51:19.000000000","message":"++","commit_id":"f17e39967e72e18e2f5369f1a06d7cb28c17b807"}],"nova/tests/unit/compute/test_compute_mgr.py":[{"author":{"_account_id":8556,"name":"Ghanshyam Maan","display_name":"Ghanshyam Maan","email":"gmaan.os14@gmail.com","username":"ghanshyam"},"change_message_id":"4096d3c09005ec854251019926780f3e3f7f6158","unresolved":true,"context_lines":[{"line_number":1425,"context_line":""},{"line_number":1426,"context_line":"            self.compute.cleanup_host()"},{"line_number":1427,"context_line":""},{"line_number":1428,"context_line":"        # shutdown_executor() must run after the driver cleanup and"},{"line_number":1429,"context_line":"        # before the live migration pool cleanup."},{"line_number":1430,"context_line":"        self.assertEqual(["},{"line_number":1431,"context_line":"            mock.call.driver_cleanup_host(host\u003dself.compute.host),"}],"source_content_type":"text/x-python","patch_set":2,"id":"28a9677f_76c6f7a0","line":1428,"range":{"start_line":1428,"start_character":10,"end_line":1428,"end_character":29},"updated":"2026-10-09 17:51:19.000000000","message":"let\u0027s me clear about live migration executor shutdown\n\n    # shutdown_executor(ExecutorType.LIVE_MIGRATION)...","commit_id":"f17e39967e72e18e2f5369f1a06d7cb28c17b807"},{"author":{"_account_id":8556,"name":"Ghanshyam Maan","display_name":"Ghanshyam Maan","email":"gmaan.os14@gmail.com","username":"ghanshyam"},"change_message_id":"4096d3c09005ec854251019926780f3e3f7f6158","unresolved":false,"context_lines":[{"line_number":1433,"context_line":"                thread_pool_factory.ExecutorType.LIVE_MIGRATION),"},{"line_number":1434,"context_line":"            mock.call.cleanup_migrations_pool(),"},{"line_number":1435,"context_line":"        ], tracker.mock_calls)"},{"line_number":1436,"context_line":"        mock_shutdown_all_executors.assert_not_called()"},{"line_number":1437,"context_line":""},{"line_number":1438,"context_line":"    @mock.patch.object(manager.ComputeManager,"},{"line_number":1439,"context_line":"                        \u0027_cleanup_live_migrations_in_pool\u0027)"}],"source_content_type":"text/x-python","patch_set":2,"id":"26ab105f_7d1ee3b9","line":1436,"range":{"start_line":1436,"start_character":0,"end_line":1436,"end_character":55},"updated":"2026-10-09 17:51:19.000000000","message":"++","commit_id":"f17e39967e72e18e2f5369f1a06d7cb28c17b807"}],"nova/thread_pool_factory.py":[{"author":{"_account_id":8556,"name":"Ghanshyam Maan","display_name":"Ghanshyam Maan","email":"gmaan.os14@gmail.com","username":"ghanshyam"},"change_message_id":"4096d3c09005ec854251019926780f3e3f7f6158","unresolved":true,"context_lines":[{"line_number":249,"context_line":"            if mark_shutdown:"},{"line_number":250,"context_line":"                self._shutdown \u003d True"},{"line_number":251,"context_line":"            executors \u003d list(self._all_executors.values())"},{"line_number":252,"context_line":"            self._all_executors.clear()"},{"line_number":253,"context_line":""},{"line_number":254,"context_line":"        for executor in executors:"},{"line_number":255,"context_line":"            self._shutdown_executor(executor)"}],"source_content_type":"text/x-python","patch_set":2,"id":"ecc66826_8acfd166","line":252,"range":{"start_line":252,"start_character":38,"end_line":252,"end_character":39},"updated":"2026-10-09 17:51:19.000000000","message":"let\u0027s clear self._shutdown_executor_types also\n\nself._shutdown_executor_types.clear()","commit_id":"f17e39967e72e18e2f5369f1a06d7cb28c17b807"}],"releasenotes/notes/graceful-shutdown-executors-after-rpc-drain-d05cb7cddfec6456.yaml":[{"author":{"_account_id":8556,"name":"Ghanshyam Maan","display_name":"Ghanshyam Maan","email":"gmaan.os14@gmail.com","username":"ghanshyam"},"change_message_id":"4096d3c09005ec854251019926780f3e3f7f6158","unresolved":true,"context_lines":[{"line_number":1,"context_line":"---"},{"line_number":2,"context_line":"fixes:"},{"line_number":3,"context_line":"  - |"},{"line_number":4,"context_line":"    Restarting ``nova-scheduler``, ``nova-conductor`` or ``nova-compute`` no"},{"line_number":5,"context_line":"    longer fails the requests their RPC server had already consumed when the"},{"line_number":6,"context_line":"    graceful shutdown started, which could set servers being created to"},{"line_number":7,"context_line":"    ``ERROR`` with ``Cannot create the cell_worker thread pool executor"},{"line_number":8,"context_line":"    because shutdown has been started``. The manager graceful shutdown now"},{"line_number":9,"context_line":"    starts once the RPC server dispatched those requests, and the thread pool"},{"line_number":10,"context_line":"    executors are shut down once every RPC server is stopped."},{"line_number":11,"context_line":"    See `bug 2169804 \u003chttps://bugs.launchpad.net/nova/+bug/2169804\u003e`_."}],"source_content_type":"text/x-yaml","patch_set":2,"id":"2c1a4098_6177029f","line":11,"range":{"start_line":4,"start_character":0,"end_line":11,"end_character":70},"updated":"2026-10-09 17:51:19.000000000","message":"let\u0027s be more user perspective here instead of adding error message details, how about:\n\n    Restarting ``nova-scheduler``, ``nova-conductor`` or ``nova-compute`` no\n    longer fails the requests their RPC server had already consumed when the\n    graceful shutdown started. This can occur when the RPC server is slow to dispatch\n    the message to the service manager and the manager shuts down before that. Now\n    the manager waits until the RPC server has dispatched all requests and then proceeds.\n    See `bug 2169804 \u003chttps://bugs.launchpad.net/nova/+bug/2169804\u003e`_.","commit_id":"f17e39967e72e18e2f5369f1a06d7cb28c17b807"}]}