)]}'
{"/PATCHSET_LEVEL":[{"author":{"_account_id":4393,"name":"Dan Smith","email":"dms@danplanet.com","username":"danms"},"change_message_id":"047ad036a658caf3ed97d4138cc21edeb617a6d1","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":2,"id":"35fe4282_cc0a36b7","updated":"2026-06-11 19:17:11.000000000","message":"Only made it through the first tests. Had a few questions/suggestions that might make sense for the whole thing before I move on.","commit_id":"3310676e8844c410803c61a425f9a085656951f8"},{"author":{"_account_id":4690,"name":"melanie witt","display_name":"melwitt","email":"melwittt@gmail.com","username":"melwitt"},"change_message_id":"1ab052341fe05385c395fefcc7b27cd68d3500c8","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":4,"id":"369e5701_f12dd298","updated":"2026-06-17 23:16:15.000000000","message":"Nice technique with the tests, LGTM","commit_id":"652c9f8b70fb808f8d290cb3076d95150e599e9b"},{"author":{"_account_id":4393,"name":"Dan Smith","email":"dms@danplanet.com","username":"danms"},"change_message_id":"f2a9b729dc387216dc51d2794aa84695a6f247ea","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":4,"id":"695c9368_ed32a8bb","updated":"2026-06-16 18:19:13.000000000","message":"Thanks, I think these are pretty tight now!","commit_id":"652c9f8b70fb808f8d290cb3076d95150e599e9b"}],"nova/tests/functional/test_graceful_shutdown.py":[{"author":{"_account_id":4393,"name":"Dan Smith","email":"dms@danplanet.com","username":"danms"},"change_message_id":"047ad036a658caf3ed97d4138cc21edeb617a6d1","unresolved":true,"context_lines":[{"line_number":134,"context_line":"        stop_thread \u003d self._stop_compute_gracefully(self.computes[\u0027src\u0027])"},{"line_number":135,"context_line":""},{"line_number":136,"context_line":"        # Allow the migration to proceed."},{"line_number":137,"context_line":"        proceed.set()"},{"line_number":138,"context_line":""},{"line_number":139,"context_line":"        # Verify migration completed successfully."},{"line_number":140,"context_line":"        server \u003d self._wait_for_state_change(server, \u0027ACTIVE\u0027)"}],"source_content_type":"text/x-python","patch_set":2,"id":"f059a550_e40e864b","line":137,"updated":"2026-06-11 19:17:11.000000000","message":"I think you want to assert that we haven\u0027t done the post operation  here, and only have done it after you set the event. To make sure we hadn\u0027t yet finished before we even tried to cancel.\n\nI also wonder if we can check to see how many threads are waiting on this event ot make sure we\u0027ve made it this far in the worker and it hasn\u0027t been slow to start or something.","commit_id":"3310676e8844c410803c61a425f9a085656951f8"},{"author":{"_account_id":8556,"name":"Ghanshyam Maan","display_name":"Ghanshyam Maan","email":"gmaan.os14@gmail.com","username":"ghanshyam"},"change_message_id":"139676de71d16992a7653e00782690b378131cab","unresolved":false,"context_lines":[{"line_number":134,"context_line":"        stop_thread \u003d self._stop_compute_gracefully(self.computes[\u0027src\u0027])"},{"line_number":135,"context_line":""},{"line_number":136,"context_line":"        # Allow the migration to proceed."},{"line_number":137,"context_line":"        proceed.set()"},{"line_number":138,"context_line":""},{"line_number":139,"context_line":"        # Verify migration completed successfully."},{"line_number":140,"context_line":"        server \u003d self._wait_for_state_change(server, \u0027ACTIVE\u0027)"}],"source_content_type":"text/x-python","patch_set":2,"id":"cd72bcfc_3f6c350e","line":137,"in_reply_to":"f059a550_e40e864b","updated":"2026-06-15 18:44:17.000000000","message":"sure, done. That is better and more explicit to test what this test is meant for. I could not find any way to count the thread waiting for a particular event until I check the interal variable _waiters. I think asserting on operation is not completed will complete the purpose here.","commit_id":"3310676e8844c410803c61a425f9a085656951f8"},{"author":{"_account_id":4393,"name":"Dan Smith","email":"dms@danplanet.com","username":"danms"},"change_message_id":"047ad036a658caf3ed97d4138cc21edeb617a6d1","unresolved":true,"context_lines":[{"line_number":142,"context_line":"        self._wait_for_migration_status(server, [\u0027completed\u0027])"},{"line_number":143,"context_line":""},{"line_number":144,"context_line":"        # Mark operation complete and signal graceful_shutdown() to proceed"},{"line_number":145,"context_line":"        operation_complete_event.set()"},{"line_number":146,"context_line":"        self._join_stop_thread(stop_thread)"},{"line_number":147,"context_line":""},{"line_number":148,"context_line":"        self._restart_compute(\u0027src\u0027)"}],"source_content_type":"text/x-python","patch_set":2,"id":"31156550_7aedf7b9","line":145,"updated":"2026-06-11 19:17:11.000000000","message":"Don\u0027t we also want to, somehow, make sure that the manager was actually waiting? I mean I guess if it exits cleanly then it probably was, but.. this is potentially very racy so it seems best to have these tests very clean on the assertions.","commit_id":"3310676e8844c410803c61a425f9a085656951f8"},{"author":{"_account_id":8556,"name":"Ghanshyam Maan","display_name":"Ghanshyam Maan","email":"gmaan.os14@gmail.com","username":"ghanshyam"},"change_message_id":"139676de71d16992a7653e00782690b378131cab","unresolved":false,"context_lines":[{"line_number":142,"context_line":"        self._wait_for_migration_status(server, [\u0027completed\u0027])"},{"line_number":143,"context_line":""},{"line_number":144,"context_line":"        # Mark operation complete and signal graceful_shutdown() to proceed"},{"line_number":145,"context_line":"        operation_complete_event.set()"},{"line_number":146,"context_line":"        self._join_stop_thread(stop_thread)"},{"line_number":147,"context_line":""},{"line_number":148,"context_line":"        self._restart_compute(\u0027src\u0027)"}],"source_content_type":"text/x-python","patch_set":2,"id":"a958dd08_987957de","line":145,"in_reply_to":"31156550_7aedf7b9","updated":"2026-06-15 18:44:17.000000000","message":"Done","commit_id":"3310676e8844c410803c61a425f9a085656951f8"},{"author":{"_account_id":4393,"name":"Dan Smith","email":"dms@danplanet.com","username":"danms"},"change_message_id":"047ad036a658caf3ed97d4138cc21edeb617a6d1","unresolved":true,"context_lines":[{"line_number":145,"context_line":"        operation_complete_event.set()"},{"line_number":146,"context_line":"        self._join_stop_thread(stop_thread)"},{"line_number":147,"context_line":""},{"line_number":148,"context_line":"        self._restart_compute(\u0027src\u0027)"},{"line_number":149,"context_line":"        self._delete_server(server)"},{"line_number":150,"context_line":""},{"line_number":151,"context_line":"    def test_live_migration_dest_compute_graceful_shutdown(self):"}],"source_content_type":"text/x-python","patch_set":2,"id":"49c9593b_34d066cc","line":148,"updated":"2026-06-11 19:17:11.000000000","message":"Hmm, why restart? Oh, so that you can delete the test server?","commit_id":"3310676e8844c410803c61a425f9a085656951f8"},{"author":{"_account_id":8556,"name":"Ghanshyam Maan","display_name":"Ghanshyam Maan","email":"gmaan.os14@gmail.com","username":"ghanshyam"},"change_message_id":"139676de71d16992a7653e00782690b378131cab","unresolved":false,"context_lines":[{"line_number":145,"context_line":"        operation_complete_event.set()"},{"line_number":146,"context_line":"        self._join_stop_thread(stop_thread)"},{"line_number":147,"context_line":""},{"line_number":148,"context_line":"        self._restart_compute(\u0027src\u0027)"},{"line_number":149,"context_line":"        self._delete_server(server)"},{"line_number":150,"context_line":""},{"line_number":151,"context_line":"    def test_live_migration_dest_compute_graceful_shutdown(self):"}],"source_content_type":"text/x-python","patch_set":2,"id":"6acc0980_e7aaedbe","line":148,"in_reply_to":"49c9593b_34d066cc","updated":"2026-06-15 18:44:17.000000000","message":"yeah, delete server and also can make sure service can start normally after shutdown.","commit_id":"3310676e8844c410803c61a425f9a085656951f8"},{"author":{"_account_id":4393,"name":"Dan Smith","email":"dms@danplanet.com","username":"danms"},"change_message_id":"63345822c410fe16bc00664321649336603773b4","unresolved":true,"context_lines":[{"line_number":117,"context_line":"                context, instance, dest, post_method, recover_method,"},{"line_number":118,"context_line":"                block_migration\u003dFalse, migrate_data\u003dNone):"},{"line_number":119,"context_line":"            started.set()"},{"line_number":120,"context_line":"            proceed.wait(timeout\u003dself.OPERATION_TIMEOUT)"},{"line_number":121,"context_line":"            post_method(context, instance, dest, block_migration, migrate_data)"},{"line_number":122,"context_line":""},{"line_number":123,"context_line":"        self.useFixture(fixtures.MockPatchObject("}],"source_content_type":"text/x-python","patch_set":3,"id":"9d438406_80f12750","line":120,"updated":"2026-06-15 21:07:44.000000000","message":"If I comment this out, the test still passes. This means we didn\u0027t wait here and already completed, and thus didn\u0027t confirm that it was actually running when we tried to do graceful shutdown, right? When I said confirm it was waiting, I meant some way to check that there\u0027s a waiter before we signal it to proceed. I thought `threading.Event` had this, but apparently not. Perhaps we need a wrapper that gives us that accounting?","commit_id":"e0623621392a41ad03bd7973feb54a55a121fadf"},{"author":{"_account_id":4393,"name":"Dan Smith","email":"dms@danplanet.com","username":"danms"},"change_message_id":"f2a9b729dc387216dc51d2794aa84695a6f247ea","unresolved":false,"context_lines":[{"line_number":117,"context_line":"                context, instance, dest, post_method, recover_method,"},{"line_number":118,"context_line":"                block_migration\u003dFalse, migrate_data\u003dNone):"},{"line_number":119,"context_line":"            started.set()"},{"line_number":120,"context_line":"            proceed.wait(timeout\u003dself.OPERATION_TIMEOUT)"},{"line_number":121,"context_line":"            post_method(context, instance, dest, block_migration, migrate_data)"},{"line_number":122,"context_line":""},{"line_number":123,"context_line":"        self.useFixture(fixtures.MockPatchObject("}],"source_content_type":"text/x-python","patch_set":3,"id":"a74e570b_80a73914","line":120,"in_reply_to":"75b8c9a4_7541cf03","updated":"2026-06-16 18:19:13.000000000","message":"Yep, this looks better and can confirm it fails locally now without.","commit_id":"e0623621392a41ad03bd7973feb54a55a121fadf"},{"author":{"_account_id":8556,"name":"Ghanshyam Maan","display_name":"Ghanshyam Maan","email":"gmaan.os14@gmail.com","username":"ghanshyam"},"change_message_id":"9af4b9b929ed5f6cba7e2b276d4eb3b4e1bb8bb0","unresolved":true,"context_lines":[{"line_number":117,"context_line":"                context, instance, dest, post_method, recover_method,"},{"line_number":118,"context_line":"                block_migration\u003dFalse, migrate_data\u003dNone):"},{"line_number":119,"context_line":"            started.set()"},{"line_number":120,"context_line":"            proceed.wait(timeout\u003dself.OPERATION_TIMEOUT)"},{"line_number":121,"context_line":"            post_method(context, instance, dest, block_migration, migrate_data)"},{"line_number":122,"context_line":""},{"line_number":123,"context_line":"        self.useFixture(fixtures.MockPatchObject("}],"source_content_type":"text/x-python","patch_set":3,"id":"75b8c9a4_7541cf03","line":120,"in_reply_to":"9d438406_80f12750","updated":"2026-06-16 04:23:09.000000000","message":"It might have passed because live migration is slow and it still running during graceful shutdown. We have two asserts to confirm that. 1. Assert at L143 make sure shutdown is started 2. assert at L148 make sure live migration is not completed before shutdown is done. If live migration is completed before shutdown then L148 will fail. I try running it by commenting the proceed.wait and 2nd time itself got failure:\n\n    nova.tests.functional.test_graceful_shutdown.TestGracefulShutdown.test_live_migration_source_compute_graceful_shutdown\n    ----------------------------------------------------------------------------------------------------------------------\n\n    Captured traceback:\n    ~~~~~~~~~~~~~~~~~~~\n    Traceback (most recent call last):\n\n      File \"/opt/stack/nova/nova/tests/functional/test_graceful_shutdown.py\", line 147, in test_live_migration_source_compute_graceful_shutdown\n    self.assertEqual(\n\n      File \"/opt/stack/nova/.tox/functional/lib/python3.12/site-packages/testtools/testcase.py\", line 513, in assertEqual\n    self.assertThat(observed, matcher, message)\n\n      File \"/opt/stack/nova/.tox/functional/lib/python3.12/site-packages/testtools/testcase.py\", line 704, in assertThat\n    raise mismatch_error\n\n    testtools.matchers._impl.MismatchError: \u0027MIGRATING\u0027 !\u003d \u0027ACTIVE\u0027\n\n\n\nproceed.wait() adds an explicit wait here if live migration is fast and finishing before the service shutdown is started. I am sure it might not be 100% success if you comment this wait.\n\nI can add a wrapper here to count the thread waiting for this event and assert on that as 3rd assert to confirm that migration is waiting.","commit_id":"e0623621392a41ad03bd7973feb54a55a121fadf"},{"author":{"_account_id":4393,"name":"Dan Smith","email":"dms@danplanet.com","username":"danms"},"change_message_id":"63345822c410fe16bc00664321649336603773b4","unresolved":true,"context_lines":[{"line_number":161,"context_line":"            \u0027graceful shutdown should still be waiting for operation\u0027)"},{"line_number":162,"context_line":"        # Mark operation complete and signal graceful_shutdown() to proceed"},{"line_number":163,"context_line":"        operation_complete_event.set()"},{"line_number":164,"context_line":"        self._join_stop_thread(stop_thread)"},{"line_number":165,"context_line":""},{"line_number":166,"context_line":"        # compute service is restarted to check if after shutdown, service"},{"line_number":167,"context_line":"        # comes up normally and test server is deleted."}],"source_content_type":"text/x-python","patch_set":3,"id":"0179a8af_31b81e72","line":164,"updated":"2026-06-15 21:07:44.000000000","message":"So, this asserts that we called stop in a thread and that thread is finished, but I guess it doesn\u0027t seem totally convincing to me that it\u0027s actually stopped. I guess we\u0027re not really starting these services as in starting a thread - more they\u0027re just marked as started so we can make synchronous \"RPC\" calls to them, right? I guess I\u0027ve forgotten how all this functional machinery works for the tests. Probably I\u0027m being too pedantic, I just worry about not really having validated a lot of this shutdown stuff I guess.","commit_id":"e0623621392a41ad03bd7973feb54a55a121fadf"},{"author":{"_account_id":8556,"name":"Ghanshyam Maan","display_name":"Ghanshyam Maan","email":"gmaan.os14@gmail.com","username":"ghanshyam"},"change_message_id":"9af4b9b929ed5f6cba7e2b276d4eb3b4e1bb8bb0","unresolved":true,"context_lines":[{"line_number":161,"context_line":"            \u0027graceful shutdown should still be waiting for operation\u0027)"},{"line_number":162,"context_line":"        # Mark operation complete and signal graceful_shutdown() to proceed"},{"line_number":163,"context_line":"        operation_complete_event.set()"},{"line_number":164,"context_line":"        self._join_stop_thread(stop_thread)"},{"line_number":165,"context_line":""},{"line_number":166,"context_line":"        # compute service is restarted to check if after shutdown, service"},{"line_number":167,"context_line":"        # comes up normally and test server is deleted."}],"source_content_type":"text/x-python","patch_set":3,"id":"5a000533_d0f710d6","line":164,"in_reply_to":"0179a8af_31b81e72","updated":"2026-06-16 04:23:09.000000000","message":"Yeah, instead of checking whether the thread is finished, I can check whether the service is actually stopped. At least that will verify the shutdown is finished.","commit_id":"e0623621392a41ad03bd7973feb54a55a121fadf"},{"author":{"_account_id":4393,"name":"Dan Smith","email":"dms@danplanet.com","username":"danms"},"change_message_id":"f2a9b729dc387216dc51d2794aa84695a6f247ea","unresolved":false,"context_lines":[{"line_number":161,"context_line":"            \u0027graceful shutdown should still be waiting for operation\u0027)"},{"line_number":162,"context_line":"        # Mark operation complete and signal graceful_shutdown() to proceed"},{"line_number":163,"context_line":"        operation_complete_event.set()"},{"line_number":164,"context_line":"        self._join_stop_thread(stop_thread)"},{"line_number":165,"context_line":""},{"line_number":166,"context_line":"        # compute service is restarted to check if after shutdown, service"},{"line_number":167,"context_line":"        # comes up normally and test server is deleted."}],"source_content_type":"text/x-python","patch_set":3,"id":"ea04731d_6615b086","line":164,"in_reply_to":"5a000533_d0f710d6","updated":"2026-06-16 18:19:13.000000000","message":"Acknowledged","commit_id":"e0623621392a41ad03bd7973feb54a55a121fadf"},{"author":{"_account_id":4393,"name":"Dan Smith","email":"dms@danplanet.com","username":"danms"},"change_message_id":"63345822c410fe16bc00664321649336603773b4","unresolved":true,"context_lines":[{"line_number":263,"context_line":"            stop_thread.is_alive(),"},{"line_number":264,"context_line":"            \u0027graceful shutdown should still be waiting for operation\u0027)"},{"line_number":265,"context_line":"        operation_complete_event.set()"},{"line_number":266,"context_line":"        self._join_stop_thread(stop_thread)"},{"line_number":267,"context_line":""},{"line_number":268,"context_line":"        # compute service is restarted to check if after shutdown, service"},{"line_number":269,"context_line":"        # comes up normally and test server is deleted after confirm rezie."}],"source_content_type":"text/x-python","patch_set":3,"id":"29f8b168_17d210f2","line":266,"updated":"2026-06-15 21:07:44.000000000","message":"Hmm... maybe related to the above... if we\u0027re going to catch the shutdown request and handle it \"gracefully\" shouldn\u0027t we explicitly mark ourselves \"down\" in the database? Thinking about this, a server \"finishes\" resizing and sits in VERIFY_RESIZE state, then the compute service immediately terminates. They go and try to confirm, but can\u0027t because the host is down. Now, that exists today of course, but if we\u0027re trying to make this cleaner, it would be best if we mark ourselves as down so we _could_ do a check and reject at the API layer right?\n\nUnrelated to these tests (until it\u0027s done) of course, just thinking.","commit_id":"e0623621392a41ad03bd7973feb54a55a121fadf"},{"author":{"_account_id":8556,"name":"Ghanshyam Maan","display_name":"Ghanshyam Maan","email":"gmaan.os14@gmail.com","username":"ghanshyam"},"change_message_id":"9af4b9b929ed5f6cba7e2b276d4eb3b4e1bb8bb0","unresolved":true,"context_lines":[{"line_number":263,"context_line":"            stop_thread.is_alive(),"},{"line_number":264,"context_line":"            \u0027graceful shutdown should still be waiting for operation\u0027)"},{"line_number":265,"context_line":"        operation_complete_event.set()"},{"line_number":266,"context_line":"        self._join_stop_thread(stop_thread)"},{"line_number":267,"context_line":""},{"line_number":268,"context_line":"        # compute service is restarted to check if after shutdown, service"},{"line_number":269,"context_line":"        # comes up normally and test server is deleted after confirm rezie."}],"source_content_type":"text/x-python","patch_set":3,"id":"c8297df7_3e2f2165","line":266,"in_reply_to":"29f8b168_17d210f2","updated":"2026-06-16 04:23:09.000000000","message":"\u003e Hmm... maybe related to the above... if we\u0027re going to catch the shutdown request and handle it \"gracefully\" shouldn\u0027t we explicitly mark ourselves \"down\" in the database? \n\nYeah, I\u0027ll update these assertions to reflect the service\u0027s actual status in the DB.\n\n\n\u003e Thinking about this, a server \"finishes\" resizing and sits in VERIFY_RESIZE state, then the compute service immediately terminates. They go and try to confirm, but can\u0027t because the host is down. Now, that exists today of course, but if we\u0027re trying to make this cleaner, it would be best if we mark ourselves as down so we _could_ do a check and reject at the API layer right?\n\u003e \n\u003e Unrelated to these tests (until it\u0027s done) of course, just thinking.\n\nMaybe I thought about it during part-1 implementation, but then I thought that we can do the compute service check at the API layer for every operation right now, not just CONFIRM/REVERT resize? And that will be unnecessary, or maybe overheard, at the API layer when the service is up.\n\nAnother drawback of checking and rejecting at the API layer is that if the service is up again, the user operation is lost. If we do not reject at the API layer, it can still be in RPC, and the service can serve the RPC requests once it is up. I know that it has its own cons when RPC requests can be stale and can end up with unwanted operation/state on the server. I am not sure which is a better experience for users, asking them to perform the operation again or having a risk of stale operation to perform (which can be protected in code with some state/server\u0027s current host checks but we are missing those currently)","commit_id":"e0623621392a41ad03bd7973feb54a55a121fadf"},{"author":{"_account_id":8556,"name":"Ghanshyam Maan","display_name":"Ghanshyam Maan","email":"gmaan.os14@gmail.com","username":"ghanshyam"},"change_message_id":"6624d3c5f5866eb210788f7b6ad9b4ed47a9b680","unresolved":true,"context_lines":[{"line_number":263,"context_line":"            stop_thread.is_alive(),"},{"line_number":264,"context_line":"            \u0027graceful shutdown should still be waiting for operation\u0027)"},{"line_number":265,"context_line":"        operation_complete_event.set()"},{"line_number":266,"context_line":"        self._join_stop_thread(stop_thread)"},{"line_number":267,"context_line":""},{"line_number":268,"context_line":"        # compute service is restarted to check if after shutdown, service"},{"line_number":269,"context_line":"        # comes up normally and test server is deleted after confirm rezie."}],"source_content_type":"text/x-python","patch_set":3,"id":"9838a77c_d292b554","line":266,"in_reply_to":"7879cf16_b9df5307","updated":"2026-06-16 22:36:07.000000000","message":"yeah, it will be good if we can somehow differentiate the restart vs shutdown. let me check it during phase-3.","commit_id":"e0623621392a41ad03bd7973feb54a55a121fadf"},{"author":{"_account_id":8556,"name":"Ghanshyam Maan","display_name":"Ghanshyam Maan","email":"gmaan.os14@gmail.com","username":"ghanshyam"},"change_message_id":"451d498212cd0c8a67ac5d8b4ee3a396c345206e","unresolved":true,"context_lines":[{"line_number":263,"context_line":"            stop_thread.is_alive(),"},{"line_number":264,"context_line":"            \u0027graceful shutdown should still be waiting for operation\u0027)"},{"line_number":265,"context_line":"        operation_complete_event.set()"},{"line_number":266,"context_line":"        self._join_stop_thread(stop_thread)"},{"line_number":267,"context_line":""},{"line_number":268,"context_line":"        # compute service is restarted to check if after shutdown, service"},{"line_number":269,"context_line":"        # comes up normally and test server is deleted after confirm rezie."}],"source_content_type":"text/x-python","patch_set":3,"id":"820badaa_e67a119d","line":266,"in_reply_to":"7cba4f43_2e7b7e2d","updated":"2026-06-16 18:07:17.000000000","message":"You mean marking service down in DB as soon shutdown is initiated? I think that is an option and can help scheduler also not to schedule new instance itself to down compute (currently it is going via RPC and timeout if shutting down compute is selected).\n\nOne concern/open question, does any of the tasks which compute will handle during shutdown checks the service DB status? if yes then we need to wait for DB service \u0027down\u0027 update until it finish the in-progress tasks but then it will be shutdown finish itself so we would not much benefit in that case.","commit_id":"e0623621392a41ad03bd7973feb54a55a121fadf"},{"author":{"_account_id":4393,"name":"Dan Smith","email":"dms@danplanet.com","username":"danms"},"change_message_id":"84fd23e52fc8cb5cd1e0f33cdfb3c76af6871940","unresolved":true,"context_lines":[{"line_number":263,"context_line":"            stop_thread.is_alive(),"},{"line_number":264,"context_line":"            \u0027graceful shutdown should still be waiting for operation\u0027)"},{"line_number":265,"context_line":"        operation_complete_event.set()"},{"line_number":266,"context_line":"        self._join_stop_thread(stop_thread)"},{"line_number":267,"context_line":""},{"line_number":268,"context_line":"        # compute service is restarted to check if after shutdown, service"},{"line_number":269,"context_line":"        # comes up normally and test server is deleted after confirm rezie."}],"source_content_type":"text/x-python","patch_set":3,"id":"7879cf16_b9df5307","line":266,"in_reply_to":"820badaa_e67a119d","updated":"2026-06-16 18:15:47.000000000","message":"\u003e You mean marking service down in DB as soon shutdown is initiated?\n\nWell, not necessarily as soon as it\u0027s initiated, but basically as soon as we\u0027re going to shutdown the main RPC queue (which I guess is immediate).\n\n\u003e I think that is an option and can help scheduler also not to schedule new instance itself to down compute (currently it is going via RPC and timeout if shutting down compute is selected).\n\nRight. On the one hand, someone doing a service restart may prefer just to have builds (etc) queued for a second while the service comes back up. But in most cases, shutdown for any period of time should be something we help by marking ourselves down first.\n\nPerhaps a different signal to tell the service not to mark itself down because we\u0027re doing a restart vs. a shutdown for a longer time?\n\nAnyway, this is not important for this patch, I\u0027m just thinking about it in the context of these tests.\n\n\u003e One concern/open question, does any of the tasks which compute will handle during shutdown checks the service DB status? if yes then we need to wait for DB service \u0027down\u0027 update until it finish the in-progress tasks but then it will be shutdown finish itself so we would not much benefit in that case.\n\nMeaning where the compute would check its own \"down\" state? I can\u0027t think of anything that would do that (or any good reason why it would). The service state and disabled flag are really just for the scheduler/API (and thus the user/admin).","commit_id":"e0623621392a41ad03bd7973feb54a55a121fadf"},{"author":{"_account_id":4393,"name":"Dan Smith","email":"dms@danplanet.com","username":"danms"},"change_message_id":"22d002f54c61216dd55a82f2457e9d2c2836ed9f","unresolved":true,"context_lines":[{"line_number":263,"context_line":"            stop_thread.is_alive(),"},{"line_number":264,"context_line":"            \u0027graceful shutdown should still be waiting for operation\u0027)"},{"line_number":265,"context_line":"        operation_complete_event.set()"},{"line_number":266,"context_line":"        self._join_stop_thread(stop_thread)"},{"line_number":267,"context_line":""},{"line_number":268,"context_line":"        # compute service is restarted to check if after shutdown, service"},{"line_number":269,"context_line":"        # comes up normally and test server is deleted after confirm rezie."}],"source_content_type":"text/x-python","patch_set":3,"id":"7cba4f43_2e7b7e2d","line":266,"in_reply_to":"c8297df7_3e2f2165","updated":"2026-06-16 17:01:12.000000000","message":"Yeah, I don\u0027t want to add an \"is it up\" check before every API operation for sure. I was just sort of using this as an example to say that we _could_ make ourselves look \"down\" as soon as we shutdown instead of the API continuing to think that compute services are up for a few minutes after shutdown until the timer expires.","commit_id":"e0623621392a41ad03bd7973feb54a55a121fadf"}]}