)]}'
{"/COMMIT_MSG":[{"author":{"_account_id":1179,"name":"Clay Gerrard","email":"clay.gerrard@gmail.com","username":"clay-gerrard"},"change_message_id":"a13e4c85ba439cb84d2b8fff27b4942b6f3e67bc","unresolved":true,"context_lines":[{"line_number":8,"context_line":""},{"line_number":9,"context_line":"TODO:"},{"line_number":10,"context_line":""},{"line_number":11,"context_line":"* needs unit test, and fixes to existing assertions"},{"line_number":12,"context_line":""},{"line_number":13,"context_line":"* needs refactor to deal with early returns from _get_from_shards"},{"line_number":14,"context_line":""}],"source_content_type":"text/x-gerrit-commit-message","patch_set":1,"id":"0c0d32f4_c5af8b2e","line":11,"updated":"2026-03-04 16:48:36.000000000","message":"there\u0027s some existing tests that cover the mixed_policies cases\n\n```\nvagrant@saio:~$ pytest swift/test/unit/proxy/controllers/test_container.py -k mixed_policies --collect-only --quiet\n\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d test session starts \u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\nplatform linux -- Python 3.10.19, pytest-9.0.2, pluggy-1.6.0\nrootdir: /home/vagrant/swift\nconfigfile: tox.ini\nplugins: cov-7.0.0\ncollected 114 items / 110 deselected / 4 selected                                                                                                                                                                                                      \n\n\u003cDir swift\u003e\n  \u003cPackage test\u003e\n    \u003cPackage unit\u003e\n      \u003cPackage proxy\u003e\n        \u003cPackage controllers\u003e\n          \u003cModule test_container.py\u003e\n            \u003cUnitTestCase TestGetShardedContainer\u003e\n              \u003cTestCaseFunction test_GET_sharded_container_mixed_policies_error\u003e\n              \u003cTestCaseFunction test_GET_sharded_container_sharding_shard_mixed_policies\u003e\n            \u003cUnitTestCase TestGetShardedContainerLegacy\u003e\n              \u003cTestCaseFunction test_GET_sharded_container_mixed_policies_error\u003e\n              \u003cTestCaseFunction test_GET_sharded_container_sharding_shard_mixed_policies\u003e\n\n\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d warnings summary \u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\nswift/test/__init__.py:38\n  /home/vagrant/swift/test/__init__.py:38: DeprecationWarning: \n  Eventlet is deprecated. It is currently being maintained in bugfix mode, and\n  we strongly recommend against using it for new projects.\n  \n  If you are already using Eventlet, we recommend migrating to a different\n  framework.  For more detail see\n  https://eventlet.readthedocs.io/en/latest/asyncio/migration.html\n  \n    from eventlet.green import socket\n\n-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html\n\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d 4/114 tests collected (110 deselected) in 0.35s \u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\n```\n\n... maybe they can be adapted to demonstrate the sharded+mixed_policy case with versions?","commit_id":"e509cf9f71075313fd1c3fd7ae5224a24f320663"},{"author":{"_account_id":1179,"name":"Clay Gerrard","email":"clay.gerrard@gmail.com","username":"clay-gerrard"},"change_message_id":"a13e4c85ba439cb84d2b8fff27b4942b6f3e67bc","unresolved":true,"context_lines":[{"line_number":10,"context_line":""},{"line_number":11,"context_line":"* needs unit test, and fixes to existing assertions"},{"line_number":12,"context_line":""},{"line_number":13,"context_line":"* needs refactor to deal with early returns from _get_from_shards"},{"line_number":14,"context_line":""},{"line_number":15,"context_line":"* an alternative/additional fix would be for object_versioning to"},{"line_number":16,"context_line":"  explicitly set the x-backend-policy-index header on the versions"}],"source_content_type":"text/x-gerrit-commit-message","patch_set":1,"id":"bef13a26_71e20039","line":13,"updated":"2026-03-04 16:48:36.000000000","message":"I would prefer to fix the bug and defer this to a follow-up","commit_id":"e509cf9f71075313fd1c3fd7ae5224a24f320663"},{"author":{"_account_id":1179,"name":"Clay Gerrard","email":"clay.gerrard@gmail.com","username":"clay-gerrard"},"change_message_id":"a13e4c85ba439cb84d2b8fff27b4942b6f3e67bc","unresolved":true,"context_lines":[{"line_number":16,"context_line":"  explicitly set the x-backend-policy-index header on the versions"},{"line_number":17,"context_line":"  listing subrequest, if it is thought that the versions *should* be"},{"line_number":18,"context_line":"  in the same policy. IMHO the bleeding of swift.shard_listing_history"},{"line_number":19,"context_line":"  should still be addressed."},{"line_number":20,"context_line":""},{"line_number":21,"context_line":"This patch ensures that every listing request to a shard container has"},{"line_number":22,"context_line":"the \u0027X-Backend-Storage-Policy-Index\u0027 header of the root container"}],"source_content_type":"text/x-gerrit-commit-message","patch_set":1,"id":"268a1b0d_383ea57d","line":19,"updated":"2026-03-04 16:48:36.000000000","message":"I think that data structure is supposed to be preserved across all sub-requests using the root request environ dict, I would expect it should be able to keep the root and version shards separate but presumably still never recurse?","commit_id":"e509cf9f71075313fd1c3fd7ae5224a24f320663"},{"author":{"_account_id":7847,"name":"Alistair Coles","email":"alistairncoles@gmail.com","username":"acoles"},"change_message_id":"0668832bd2e71daf71b9e179af85a737974aed01","unresolved":true,"context_lines":[{"line_number":16,"context_line":"  explicitly set the x-backend-policy-index header on the versions"},{"line_number":17,"context_line":"  listing subrequest, if it is thought that the versions *should* be"},{"line_number":18,"context_line":"  in the same policy. IMHO the bleeding of swift.shard_listing_history"},{"line_number":19,"context_line":"  should still be addressed."},{"line_number":20,"context_line":""},{"line_number":21,"context_line":"This patch ensures that every listing request to a shard container has"},{"line_number":22,"context_line":"the \u0027X-Backend-Storage-Policy-Index\u0027 header of the root container"}],"source_content_type":"text/x-gerrit-commit-message","patch_set":1,"id":"ec0b04c7_e88d51fd","line":19,"in_reply_to":"268a1b0d_383ea57d","updated":"2026-03-04 19:10:42.000000000","message":"\u003e that data structure is supposed to be preserved across all sub-requests using the root request environ dict\n\nI think that is the intention, but where \u0027root\u0027 request  is the request as first received from the middleware pipeline by the container controller. I don\u0027t think it was ever the intention for the history to be preserved across multiple sequential requests received from the middleware pipeline as is happening with versions listings.\n\nIt\u0027s probably ok so long as those sequential requests are never going to list from the same shards (so, ok in the versions scenario). I just think it\u0027s undesirable to have that state left in the request with the potential to break a sequential request.","commit_id":"e509cf9f71075313fd1c3fd7ae5224a24f320663"},{"author":{"_account_id":1179,"name":"Clay Gerrard","email":"clay.gerrard@gmail.com","username":"clay-gerrard"},"change_message_id":"a13e4c85ba439cb84d2b8fff27b4942b6f3e67bc","unresolved":true,"context_lines":[{"line_number":37,"context_line":"implies that the request is the \u0027root\u0027 request (i.e. the first in the"},{"line_number":38,"context_line":"recursion) and therefore the authority for the policy index to be"},{"line_number":39,"context_line":"used. When swift.shard_listing_history is not found in the environ, the"},{"line_number":40,"context_line":"proxy ensures a policy index header is set in the request environ."},{"line_number":41,"context_line":""},{"line_number":42,"context_line":"When a versions container is listed, the object_versioning middleware"},{"line_number":43,"context_line":"first lists the user container. If this is sharded, the request"}],"source_content_type":"text/x-gerrit-commit-message","patch_set":1,"id":"38d828b7_fd89b5d0","line":40,"updated":"2026-03-04 16:48:36.000000000","message":"\u003e When swift.shard_listing_history is not found in the environ, the\nproxy ensures a policy index header is set in the request environ.\n\nthe proxy should probably have a better way to tell if it should set the spi in the listing request - I feel like \"once I have a root resp all subsequent requests should use THAT spi\" might be better strat?","commit_id":"e509cf9f71075313fd1c3fd7ae5224a24f320663"},{"author":{"_account_id":7847,"name":"Alistair Coles","email":"alistairncoles@gmail.com","username":"acoles"},"change_message_id":"0668832bd2e71daf71b9e179af85a737974aed01","unresolved":true,"context_lines":[{"line_number":37,"context_line":"implies that the request is the \u0027root\u0027 request (i.e. the first in the"},{"line_number":38,"context_line":"recursion) and therefore the authority for the policy index to be"},{"line_number":39,"context_line":"used. When swift.shard_listing_history is not found in the environ, the"},{"line_number":40,"context_line":"proxy ensures a policy index header is set in the request environ."},{"line_number":41,"context_line":""},{"line_number":42,"context_line":"When a versions container is listed, the object_versioning middleware"},{"line_number":43,"context_line":"first lists the user container. If this is sharded, the request"}],"source_content_type":"text/x-gerrit-commit-message","patch_set":1,"id":"8fdc8a84_aeebfa1c","line":40,"in_reply_to":"38d828b7_fd89b5d0","updated":"2026-03-04 19:10:42.000000000","message":"\u003e I feel like \"once I have a root resp all subsequent requests should use THAT spi\" might be better strat\n\nI think that is exactly what the code on master was intended to do by copying the *root* resp spi into request and then copying it into all subrequests. But for some reason the header copy is ALSO conditional upon the shard_listing_history. I\u0027m not sure why.","commit_id":"e509cf9f71075313fd1c3fd7ae5224a24f320663"},{"author":{"_account_id":1179,"name":"Clay Gerrard","email":"clay.gerrard@gmail.com","username":"clay-gerrard"},"change_message_id":"a13e4c85ba439cb84d2b8fff27b4942b6f3e67bc","unresolved":true,"context_lines":[{"line_number":45,"context_line":"middleware then lists the versions container using a subrequest cloned"},{"line_number":46,"context_line":"from the original request. This subrequest inherits the"},{"line_number":47,"context_line":"swift.shard_listing_history from the original request. As result, the"},{"line_number":48,"context_line":"proxy does not think this is a \u0027root\u0027 request, and does not ensure"},{"line_number":49,"context_line":"that the request has an X-Backend-Storage-Policy-Index header (which"},{"line_number":50,"context_line":"is not copied into the subrequest). A KeyError would then occur when"},{"line_number":51,"context_line":"this header is checked against a response from a versions container"}],"source_content_type":"text/x-gerrit-commit-message","patch_set":1,"id":"1dfe4003_4e291063","line":48,"updated":"2026-03-04 16:48:36.000000000","message":"\u003e proxy does not think this is a \u0027root\u0027 request\n\nit *is* getting a list of the versions containers shards tho?  It does *make* a root container request - it just doesn\u0027t KNOW it!?","commit_id":"e509cf9f71075313fd1c3fd7ae5224a24f320663"},{"author":{"_account_id":7847,"name":"Alistair Coles","email":"alistairncoles@gmail.com","username":"acoles"},"change_message_id":"0668832bd2e71daf71b9e179af85a737974aed01","unresolved":true,"context_lines":[{"line_number":45,"context_line":"middleware then lists the versions container using a subrequest cloned"},{"line_number":46,"context_line":"from the original request. This subrequest inherits the"},{"line_number":47,"context_line":"swift.shard_listing_history from the original request. As result, the"},{"line_number":48,"context_line":"proxy does not think this is a \u0027root\u0027 request, and does not ensure"},{"line_number":49,"context_line":"that the request has an X-Backend-Storage-Policy-Index header (which"},{"line_number":50,"context_line":"is not copied into the subrequest). A KeyError would then occur when"},{"line_number":51,"context_line":"this header is checked against a response from a versions container"}],"source_content_type":"text/x-gerrit-commit-message","patch_set":1,"id":"698406c4_1c023fcf","line":48,"in_reply_to":"1dfe4003_4e291063","updated":"2026-03-04 19:10:42.000000000","message":"it doesn\u0027t know it in the sense that the condition to set the spi header to the root spi evaluates False","commit_id":"e509cf9f71075313fd1c3fd7ae5224a24f320663"},{"author":{"_account_id":1179,"name":"Clay Gerrard","email":"clay.gerrard@gmail.com","username":"clay-gerrard"},"change_message_id":"a13e4c85ba439cb84d2b8fff27b4942b6f3e67bc","unresolved":true,"context_lines":[{"line_number":51,"context_line":"this header is checked against a response from a versions container"},{"line_number":52,"context_line":"shard."},{"line_number":53,"context_line":""},{"line_number":54,"context_line":"The solution is to ensure that the swift.shard_listing_history is"},{"line_number":55,"context_line":"cleared from the request environ when the recursive listing from shards"},{"line_number":56,"context_line":"has completed, so that a subsequent subrequest is not polluted with an"},{"line_number":57,"context_line":"unrelated swift.shard_listing_history."}],"source_content_type":"text/x-gerrit-commit-message","patch_set":1,"id":"36e8084f_cacfedeb","line":54,"updated":"2026-03-04 16:48:36.000000000","message":"s/the/a/","commit_id":"e509cf9f71075313fd1c3fd7ae5224a24f320663"},{"author":{"_account_id":1179,"name":"Clay Gerrard","email":"clay.gerrard@gmail.com","username":"clay-gerrard"},"change_message_id":"a13e4c85ba439cb84d2b8fff27b4942b6f3e67bc","unresolved":true,"context_lines":[{"line_number":54,"context_line":"The solution is to ensure that the swift.shard_listing_history is"},{"line_number":55,"context_line":"cleared from the request environ when the recursive listing from shards"},{"line_number":56,"context_line":"has completed, so that a subsequent subrequest is not polluted with an"},{"line_number":57,"context_line":"unrelated swift.shard_listing_history."},{"line_number":58,"context_line":""},{"line_number":59,"context_line":"Change-Id: Ia2f731b471d24b114d0675445f53d54815ca0c4c"},{"line_number":60,"context_line":"Closes-Bug: #2143220"}],"source_content_type":"text/x-gerrit-commit-message","patch_set":1,"id":"9976b237_806641aa","line":57,"updated":"2026-03-04 16:48:36.000000000","message":"oic, so there may be reason to keep the recusrive listing prevention mechanism \"per-root\" as opposed to \"per-request\"","commit_id":"e509cf9f71075313fd1c3fd7ae5224a24f320663"},{"author":{"_account_id":7847,"name":"Alistair Coles","email":"alistairncoles@gmail.com","username":"acoles"},"change_message_id":"0668832bd2e71daf71b9e179af85a737974aed01","unresolved":true,"context_lines":[{"line_number":54,"context_line":"The solution is to ensure that the swift.shard_listing_history is"},{"line_number":55,"context_line":"cleared from the request environ when the recursive listing from shards"},{"line_number":56,"context_line":"has completed, so that a subsequent subrequest is not polluted with an"},{"line_number":57,"context_line":"unrelated swift.shard_listing_history."},{"line_number":58,"context_line":""},{"line_number":59,"context_line":"Change-Id: Ia2f731b471d24b114d0675445f53d54815ca0c4c"},{"line_number":60,"context_line":"Closes-Bug: #2143220"}],"source_content_type":"text/x-gerrit-commit-message","patch_set":1,"id":"1c7f2838_3b1ce552","line":57,"in_reply_to":"9976b237_806641aa","updated":"2026-03-04 19:10:42.000000000","message":"we definitely want it per-recursive-listing","commit_id":"e509cf9f71075313fd1c3fd7ae5224a24f320663"},{"author":{"_account_id":7847,"name":"Alistair Coles","email":"alistairncoles@gmail.com","username":"acoles"},"change_message_id":"0668832bd2e71daf71b9e179af85a737974aed01","unresolved":true,"context_lines":[{"line_number":55,"context_line":"cleared from the request environ when the recursive listing from shards"},{"line_number":56,"context_line":"has completed, so that a subsequent subrequest is not polluted with an"},{"line_number":57,"context_line":"unrelated swift.shard_listing_history."},{"line_number":58,"context_line":""},{"line_number":59,"context_line":"Change-Id: Ia2f731b471d24b114d0675445f53d54815ca0c4c"},{"line_number":60,"context_line":"Closes-Bug: #2143220"},{"line_number":61,"context_line":"Signed-off-by: Alistair Coles \u003calistairncoles@gmail.com\u003e"}],"source_content_type":"text/x-gerrit-commit-message","patch_set":1,"id":"8d2bf928_7d72072b","line":58,"updated":"2026-03-04 19:10:42.000000000","message":"Related-Change:  I026b699fc5f0fba619cf524093632d67ca38d32f","commit_id":"e509cf9f71075313fd1c3fd7ae5224a24f320663"}],"/PATCHSET_LEVEL":[{"author":{"_account_id":1179,"name":"Clay Gerrard","email":"clay.gerrard@gmail.com","username":"clay-gerrard"},"change_message_id":"a13e4c85ba439cb84d2b8fff27b4942b6f3e67bc","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":1,"id":"ee888b33_43a44644","updated":"2026-03-04 16:48:36.000000000","message":"what a gold mine of tomfoolery!  the probe test is awesome!!!\n\n978907: sq? look at the resp | https://review.opendev.org/c/openstack/swift/+/978907","commit_id":"e509cf9f71075313fd1c3fd7ae5224a24f320663"},{"author":{"_account_id":7847,"name":"Alistair Coles","email":"alistairncoles@gmail.com","username":"acoles"},"change_message_id":"9a50c562f7eefe2afed52cd466f047a08f7d6b51","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":3,"id":"845c2cfc_c0a80aa0","updated":"2026-03-05 15:00:45.000000000","message":"another bug related to this https://review.opendev.org/c/openstack/swift/+/979010/1","commit_id":"e9400d7d61906af956c366b0f307e97f726a4c3f"},{"author":{"_account_id":7233,"name":"Matthew Oliver","email":"matt@oliver.net.au","username":"mattoliverau"},"change_message_id":"d8a182e84b2254e7bbe01927f2d930e127b79b1c","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":3,"id":"4b5b433e_0ece333a","updated":"2026-03-11 03:24:43.000000000","message":"recheck","commit_id":"e9400d7d61906af956c366b0f307e97f726a4c3f"}],"swift/proxy/controllers/container.py":[{"author":{"_account_id":7847,"name":"Alistair Coles","email":"alistairncoles@gmail.com","username":"acoles"},"change_message_id":"0668832bd2e71daf71b9e179af85a737974aed01","unresolved":true,"context_lines":[{"line_number":465,"context_line":"        shard_listing_history \u003d req.environ.setdefault("},{"line_number":466,"context_line":"            \u0027swift.shard_listing_history\u0027, [])"},{"line_number":467,"context_line":"        policy_key \u003d \u0027X-Backend-Storage-Policy-Index\u0027"},{"line_number":468,"context_line":"        if not (shard_listing_history or policy_key in req.headers):"},{"line_number":469,"context_line":"            # We\u0027re handling the original request to the root container: set"},{"line_number":470,"context_line":"            # the root policy index in the request, unless it is already set,"},{"line_number":471,"context_line":"            # so that shards will return listings for that policy index."}],"source_content_type":"text/x-python","patch_set":1,"id":"ede9a4a1_b34b97e0","line":468,"updated":"2026-03-04 19:10:42.000000000","message":"I\u0027m not understanding this condition.\n\nWe want an X-Backend-Storage-Policy-Index in requests, and we want it to be the same in all shard sub-requests.\n\nSo if it is not already set in the request then use the container\u0027s policy index from the initial response. This may happen for the \u0027root\u0027 request, but shard subrequest inherit the headers from the \u0027root\u0027 request so they will have ``policy_key in headers``.\n\nAnd yes, if the spi header is already set the don\u0027t change it - this is the policy we consistently want to request from all shards.\n\nBut why do we want/need a truthy shard_listing_history to stop us setting it? Why is *that* condition even needed since every subrequest WILL already have the spi header so it won\u0027t get set again. \n\nWe even test it (artificially) here test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_get_from_shards_add_root_spi (added in https://review.opendev.org/c/openstack/swift/+/803423)\n\nBut I\u0027m thinking that it may be sufficient to have:\n\n```\nif policy_key not in req.headers:\n    req.headers[policy_key] \u003d resp.headers[policy_key]\n```\n or \n \n```\nreq.headers.setdefault(policy_key, resp.headers[policy_key])\n```\n\nand that would fix the KeyError bug","commit_id":"e509cf9f71075313fd1c3fd7ae5224a24f320663"},{"author":{"_account_id":1179,"name":"Clay Gerrard","email":"clay.gerrard@gmail.com","username":"clay-gerrard"},"change_message_id":"a13e4c85ba439cb84d2b8fff27b4942b6f3e67bc","unresolved":true,"context_lines":[{"line_number":573,"context_line":"                \u0027X-Backend-Record-Storage-Policy-Index\u0027,"},{"line_number":574,"context_line":"                shard_resp.headers[policy_key]"},{"line_number":575,"context_line":"            )"},{"line_number":576,"context_line":"            if shard_policy !\u003d req.headers[policy_key]:"},{"line_number":577,"context_line":"                self.logger.error("},{"line_number":578,"context_line":"                    \u0027Aborting listing from shards due to bad shard policy \u0027"},{"line_number":579,"context_line":"                    \u0027index: %s (expected %s)\u0027,"}],"source_content_type":"text/x-python","patch_set":1,"id":"fdb5b69c_ef526ff5","line":576,"updated":"2026-03-04 16:48:36.000000000","message":"IMHO the bug is *here* - where we get the KeyError\n\n... if we think the bug is \"we\u0027re not setting the policy_key in the request\" we can add a test to demonstrate *that* ... but so far I haven\u0027t found, maybe b/c \"mixed policy and sharded and versioned\" is kind of niche?\n\nIn the meantime, I\u0027d prefer the happy path to work as opposed to blow-up\n\n978907: sq? look at the resp | https://review.opendev.org/c/openstack/swift/+/978907","commit_id":"e509cf9f71075313fd1c3fd7ae5224a24f320663"},{"author":{"_account_id":7847,"name":"Alistair Coles","email":"alistairncoles@gmail.com","username":"acoles"},"change_message_id":"0668832bd2e71daf71b9e179af85a737974aed01","unresolved":true,"context_lines":[{"line_number":573,"context_line":"                \u0027X-Backend-Record-Storage-Policy-Index\u0027,"},{"line_number":574,"context_line":"                shard_resp.headers[policy_key]"},{"line_number":575,"context_line":"            )"},{"line_number":576,"context_line":"            if shard_policy !\u003d req.headers[policy_key]:"},{"line_number":577,"context_line":"                self.logger.error("},{"line_number":578,"context_line":"                    \u0027Aborting listing from shards due to bad shard policy \u0027"},{"line_number":579,"context_line":"                    \u0027index: %s (expected %s)\u0027,"}],"source_content_type":"text/x-python","patch_set":1,"id":"b0261072_13c9d4ff","line":576,"in_reply_to":"fdb5b69c_ef526ff5","updated":"2026-03-04 19:10:42.000000000","message":"I don\u0027t agree. I think we need the spi header to be in the request, we\u0027ve just failed to ensure that it is at line 468. Maybe just sematics, but the bug is we don\u0027t set the header, and it manifests as a KeyError here.","commit_id":"e509cf9f71075313fd1c3fd7ae5224a24f320663"},{"author":{"_account_id":15343,"name":"Tim Burke","email":"tburke@nvidia.com","username":"tburke"},"change_message_id":"6d417665c74c0a447de4fa175d41c6155fb2a40e","unresolved":true,"context_lines":[{"line_number":465,"context_line":"        shard_listing_history \u003d req.environ.setdefault("},{"line_number":466,"context_line":"            \u0027swift.shard_listing_history\u0027, [])"},{"line_number":467,"context_line":"        policy_key \u003d \u0027X-Backend-Storage-Policy-Index\u0027"},{"line_number":468,"context_line":"        if not (shard_listing_history or policy_key in req.headers):"},{"line_number":469,"context_line":"            # We\u0027re handling the original request to the root container: set"},{"line_number":470,"context_line":"            # the root policy index in the request, unless it is already set,"},{"line_number":471,"context_line":"            # so that shards will return listings for that policy index."}],"source_content_type":"text/x-python","patch_set":3,"id":"e5c341d6_44b67d2d","side":"PARENT","line":468,"range":{"start_line":468,"start_character":41,"end_line":468,"end_character":66},"updated":"2026-03-05 18:00:06.000000000","message":"So we used to overwrite policy when it *was* present -- and now we only set it when it *is*... I\u0027m trying to convince myself that the fix **shouldn\u0027t** be\n```\nreq.headers[policy_key] \u003d resp.headers[policy_key]\n```","commit_id":"51b068309a626d2228ab69b5158ee857e96d101d"},{"author":{"_account_id":15343,"name":"Tim Burke","email":"tburke@nvidia.com","username":"tburke"},"change_message_id":"c7261cbef9a958ee331f12c8f0146d46c4646084","unresolved":false,"context_lines":[{"line_number":465,"context_line":"        shard_listing_history \u003d req.environ.setdefault("},{"line_number":466,"context_line":"            \u0027swift.shard_listing_history\u0027, [])"},{"line_number":467,"context_line":"        policy_key \u003d \u0027X-Backend-Storage-Policy-Index\u0027"},{"line_number":468,"context_line":"        if not (shard_listing_history or policy_key in req.headers):"},{"line_number":469,"context_line":"            # We\u0027re handling the original request to the root container: set"},{"line_number":470,"context_line":"            # the root policy index in the request, unless it is already set,"},{"line_number":471,"context_line":"            # so that shards will return listings for that policy index."}],"source_content_type":"text/x-python","patch_set":3,"id":"fe853d56_c2b1633f","side":"PARENT","line":468,"range":{"start_line":468,"start_character":41,"end_line":468,"end_character":66},"in_reply_to":"92013215_3ee5e0f3","updated":"2026-03-06 23:59:43.000000000","message":"I think I must\u0027ve misread this as\n```\nif not shard_listing_history or policy_key in req.headers:\n```\nI blame sleepiness from the pre-treatment meds ;-)","commit_id":"51b068309a626d2228ab69b5158ee857e96d101d"},{"author":{"_account_id":7847,"name":"Alistair Coles","email":"alistairncoles@gmail.com","username":"acoles"},"change_message_id":"b1afffee76bd2b3d5e53e2a27e76cadb8d6c5086","unresolved":true,"context_lines":[{"line_number":465,"context_line":"        shard_listing_history \u003d req.environ.setdefault("},{"line_number":466,"context_line":"            \u0027swift.shard_listing_history\u0027, [])"},{"line_number":467,"context_line":"        policy_key \u003d \u0027X-Backend-Storage-Policy-Index\u0027"},{"line_number":468,"context_line":"        if not (shard_listing_history or policy_key in req.headers):"},{"line_number":469,"context_line":"            # We\u0027re handling the original request to the root container: set"},{"line_number":470,"context_line":"            # the root policy index in the request, unless it is already set,"},{"line_number":471,"context_line":"            # so that shards will return listings for that policy index."}],"source_content_type":"text/x-python","patch_set":3,"id":"92013215_3ee5e0f3","side":"PARENT","line":468,"range":{"start_line":468,"start_character":41,"end_line":468,"end_character":66},"in_reply_to":"e5c341d6_44b67d2d","updated":"2026-03-06 11:44:39.000000000","message":"we\u0027d only overwrite policy when it was NOT already present in req.headers, and we continue to do that. The change is to always do that even when there is a pre-existing shard history.\n\n\u003eI\u0027m trying to convince myself that the fix shouldn\u0027t be\nreq.headers[policy_key] \u003d resp.headers[policy_key]\n\nWe explicitly do not want to do that. When the method recurses we get a different resp from a child shard and that may have a different policy index to the root container. The idea is to lock in the policy index of the root container so that all object listings returned by shards, sub-shards etc are for that policy, regardless of the policy of the shard, sub-shard containers.","commit_id":"51b068309a626d2228ab69b5158ee857e96d101d"},{"author":{"_account_id":15343,"name":"Tim Burke","email":"tburke@nvidia.com","username":"tburke"},"change_message_id":"6d417665c74c0a447de4fa175d41c6155fb2a40e","unresolved":true,"context_lines":[{"line_number":446,"context_line":"        namespaces."},{"line_number":447,"context_line":""},{"line_number":448,"context_line":"        :param req: an instance of :class:`~swift.common.swob.Request`."},{"line_number":449,"context_line":"        :param resp: an instance of :class:`~swift.common.swob.Response`."},{"line_number":450,"context_line":"        :param namespaces: a list of :class:`~swift.common.utils.Namespace`."},{"line_number":451,"context_line":"        :return: an instance of :class:`~swift.common.swob.Response`. If an"},{"line_number":452,"context_line":"            error is encountered while building the listing an instance of"}],"source_content_type":"text/x-python","patch_set":3,"id":"0f6eb447_6e8c2fa0","line":449,"updated":"2026-03-05 18:00:06.000000000","message":"Type-hints are nice, but I kinda wish we were more clear about what *kind* of response we expect this to be -- container-level, almost certainly; responding to a GET, most likely; but for objects, shards, namespaces...?","commit_id":"e9400d7d61906af956c366b0f307e97f726a4c3f"},{"author":{"_account_id":15343,"name":"Tim Burke","email":"tburke@nvidia.com","username":"tburke"},"change_message_id":"6d417665c74c0a447de4fa175d41c6155fb2a40e","unresolved":true,"context_lines":[{"line_number":474,"context_line":"        # Note: we only get here if the root responded with namespaces, or if"},{"line_number":475,"context_line":"        # the namespaces were cached and the cached root container info has"},{"line_number":476,"context_line":"        # sharding_state\u003d\u003dsharded; in both cases we can assume that the"},{"line_number":477,"context_line":"        # response is \"modern enough\" to include"},{"line_number":478,"context_line":"        # \u0027X-Backend-Storage-Policy-Index\u0027."},{"line_number":479,"context_line":"        req.headers.setdefault(policy_key, resp.headers[policy_key])"},{"line_number":480,"context_line":"        shard_listing_history.append((self.account_name, self.container_name))"}],"source_content_type":"text/x-python","patch_set":3,"id":"bde7832b_e2d6d977","line":477,"range":{"start_line":477,"start_character":23,"end_line":477,"end_character":36},"updated":"2026-03-05 18:00:06.000000000","message":"This is specifically referring to the ginned-up \"response\" that was really from cached data, yeah? All real shard/namespace responses have always included storage policy -- right?","commit_id":"e9400d7d61906af956c366b0f307e97f726a4c3f"},{"author":{"_account_id":7847,"name":"Alistair Coles","email":"alistairncoles@gmail.com","username":"acoles"},"change_message_id":"dce4d10cff2cac646ec4e9f897de861bd6d81421","unresolved":true,"context_lines":[{"line_number":474,"context_line":"        # Note: we only get here if the root responded with namespaces, or if"},{"line_number":475,"context_line":"        # the namespaces were cached and the cached root container info has"},{"line_number":476,"context_line":"        # sharding_state\u003d\u003dsharded; in both cases we can assume that the"},{"line_number":477,"context_line":"        # response is \"modern enough\" to include"},{"line_number":478,"context_line":"        # \u0027X-Backend-Storage-Policy-Index\u0027."},{"line_number":479,"context_line":"        req.headers.setdefault(policy_key, resp.headers[policy_key])"},{"line_number":480,"context_line":"        shard_listing_history.append((self.account_name, self.container_name))"}],"source_content_type":"text/x-python","patch_set":3,"id":"471c22fd_1533196f","line":477,"range":{"start_line":477,"start_character":23,"end_line":477,"end_character":36},"in_reply_to":"3212b982_cd24d8e2","updated":"2026-03-09 10:25:04.000000000","message":"Isn\u0027t that the point of the comment - you\u0027re not going to get a shard response from a container server that wouldn\u0027t already be returing the s-p-i header (so it\u0027s fine to assume the header is in resp.headers)\n\nNote: the comment is not added by this patch, just line-wrapped differently because it\u0027s de-dented w.r.t. master","commit_id":"e9400d7d61906af956c366b0f307e97f726a4c3f"},{"author":{"_account_id":7847,"name":"Alistair Coles","email":"alistairncoles@gmail.com","username":"acoles"},"change_message_id":"b1afffee76bd2b3d5e53e2a27e76cadb8d6c5086","unresolved":true,"context_lines":[{"line_number":474,"context_line":"        # Note: we only get here if the root responded with namespaces, or if"},{"line_number":475,"context_line":"        # the namespaces were cached and the cached root container info has"},{"line_number":476,"context_line":"        # sharding_state\u003d\u003dsharded; in both cases we can assume that the"},{"line_number":477,"context_line":"        # response is \"modern enough\" to include"},{"line_number":478,"context_line":"        # \u0027X-Backend-Storage-Policy-Index\u0027."},{"line_number":479,"context_line":"        req.headers.setdefault(policy_key, resp.headers[policy_key])"},{"line_number":480,"context_line":"        shard_listing_history.append((self.account_name, self.container_name))"}],"source_content_type":"text/x-python","patch_set":3,"id":"e867b7be_c4166fad","line":477,"range":{"start_line":477,"start_character":23,"end_line":477,"end_character":36},"in_reply_to":"bde7832b_e2d6d977","updated":"2026-03-06 11:44:39.000000000","message":"I read it as saying that whether the response is from a backend or from cache, we know it will have the s-p-i header, because that has always been returned with a namespaces response.","commit_id":"e9400d7d61906af956c366b0f307e97f726a4c3f"},{"author":{"_account_id":15343,"name":"Tim Burke","email":"tburke@nvidia.com","username":"tburke"},"change_message_id":"c7261cbef9a958ee331f12c8f0146d46c4646084","unresolved":true,"context_lines":[{"line_number":474,"context_line":"        # Note: we only get here if the root responded with namespaces, or if"},{"line_number":475,"context_line":"        # the namespaces were cached and the cached root container info has"},{"line_number":476,"context_line":"        # sharding_state\u003d\u003dsharded; in both cases we can assume that the"},{"line_number":477,"context_line":"        # response is \"modern enough\" to include"},{"line_number":478,"context_line":"        # \u0027X-Backend-Storage-Policy-Index\u0027."},{"line_number":479,"context_line":"        req.headers.setdefault(policy_key, resp.headers[policy_key])"},{"line_number":480,"context_line":"        shard_listing_history.append((self.account_name, self.container_name))"}],"source_content_type":"text/x-python","patch_set":3,"id":"3212b982_cd24d8e2","line":477,"range":{"start_line":477,"start_character":23,"end_line":477,"end_character":36},"in_reply_to":"e867b7be_c4166fad","updated":"2026-03-06 23:59:43.000000000","message":"I\u0027m just struggling to figure out what kind of a `resp` we could have that *wouldn\u0027t* have an s-p-i header -- \u0027cause I feel like we tend to be pretty good about including that. Are we talking some pre-2.0 container server response, or somewhere in between then and latest release? If I were to do some rolling-upgrade testing, what would be the versions of interest? Though I\u0027m not sure I **need** to -- if we\u0027re really talking about going back to 1.x, I don\u0027t think we can reasonably get to sharded containers before you\u0027ve upgraded the whole cluster.","commit_id":"e9400d7d61906af956c366b0f307e97f726a4c3f"},{"author":{"_account_id":7233,"name":"Matthew Oliver","email":"matt@oliver.net.au","username":"mattoliverau"},"change_message_id":"e285aa952acdd90ea6b0acba6f65c2894276b94f","unresolved":true,"context_lines":[{"line_number":476,"context_line":"        # sharding_state\u003d\u003dsharded; in both cases we can assume that the"},{"line_number":477,"context_line":"        # response is \"modern enough\" to include"},{"line_number":478,"context_line":"        # \u0027X-Backend-Storage-Policy-Index\u0027."},{"line_number":479,"context_line":"        req.headers.setdefault(policy_key, resp.headers[policy_key])"},{"line_number":480,"context_line":"        shard_listing_history.append((self.account_name, self.container_name))"},{"line_number":481,"context_line":"        self.logger.debug(\u0027GET listing from %s shards for: %s\u0027,"},{"line_number":482,"context_line":"                          len(namespaces), req.path_qs)"}],"source_content_type":"text/x-python","patch_set":3,"id":"a6a01b1a_891824cc","line":479,"updated":"2026-03-05 08:24:38.000000000","message":"Ahh I see what happened. When we found that bug that the root spi doesn\u0027t propergate over to shards when the roots spi changes, we decided make it so shards simply store and return whatever objects you throw at it for a given spi. And as such we added the set a policy index if there isn\u0027t one already. BUT we never took a versioned sharded container into account which happens to keep a copy of the shard_listing_history, so we can\u0027t assume it\u0027s empty and thus wont install the spi if it doesn\u0027t exist.\n\nWorse we basically now to expect it to exist as the code stands today!\n\nNo idea why we didn\u0027t just use `setdefault` in the originaly because that\u0027ll be setting it if it isn\u0027t already set. Nice one. Makes sense to me!","commit_id":"e9400d7d61906af956c366b0f307e97f726a4c3f"}]}