)]}'
{"/COMMIT_MSG":[{"author":{"_account_id":1179,"name":"Clay Gerrard","email":"clay.gerrard@gmail.com","username":"clay-gerrard"},"change_message_id":"48aff7ea454ad70e7549ecffcba774a504481590","unresolved":true,"context_lines":[{"line_number":11,"context_line":"shard was shrinking; shrinking is the only time when a shard normally"},{"line_number":12,"context_line":"*must* receive sub-shards from the root. However, the sharder *can*"},{"line_number":13,"context_line":"merge sub-shards at any time if they are known to be the children of"},{"line_number":14,"context_line":"the shard."},{"line_number":15,"context_line":""},{"line_number":16,"context_line":"Children sub-shards would previously only have been merged during"},{"line_number":17,"context_line":"replication with peers of the shard; merging sub-shards from the root"}],"source_content_type":"text/x-gerrit-commit-message","patch_set":5,"id":"11e8fdc2_ea5593db","line":14,"updated":"2022-09-21 18:25:29.000000000","message":"can you elaborate?  I think maybe a s-m-s-r command that creates sub-shards in FOUND always add rows to the shard ranges table - and peers replicate that?","commit_id":"f5024a3a70e2e50dcf114ed8f3048a9fb1a1afb3"},{"author":{"_account_id":7847,"name":"Alistair Coles","email":"alistairncoles@gmail.com","username":"acoles"},"change_message_id":"b6748f456a9497931ecfd48b399db753b85cfcf7","unresolved":true,"context_lines":[{"line_number":11,"context_line":"shard was shrinking; shrinking is the only time when a shard normally"},{"line_number":12,"context_line":"*must* receive sub-shards from the root. However, the sharder *can*"},{"line_number":13,"context_line":"merge sub-shards at any time if they are known to be the children of"},{"line_number":14,"context_line":"the shard."},{"line_number":15,"context_line":""},{"line_number":16,"context_line":"Children sub-shards would previously only have been merged during"},{"line_number":17,"context_line":"replication with peers of the shard; merging sub-shards from the root"}],"source_content_type":"text/x-gerrit-commit-message","patch_set":5,"id":"7dfb12ba_b9c745cb","line":14,"in_reply_to":"11e8fdc2_ea5593db","updated":"2022-09-23 10:22:58.000000000","message":"\u003e  a s-m-s-r command that creates sub-shards in FOUND always add rows to the shard ranges table - and peers replicate that\n\nyes. What I was trying to say here is that it would be OK for the sharder to also sync known *children* shard ranges fetched from the root in addition to the existing peer replication. So peer-\u003eroot-\u003epeer is OK as well as peer-\u003epeer.","commit_id":"f5024a3a70e2e50dcf114ed8f3048a9fb1a1afb3"},{"author":{"_account_id":1179,"name":"Clay Gerrard","email":"clay.gerrard@gmail.com","username":"clay-gerrard"},"change_message_id":"48aff7ea454ad70e7549ecffcba774a504481590","unresolved":true,"context_lines":[{"line_number":16,"context_line":"Children sub-shards would previously only have been merged during"},{"line_number":17,"context_line":"replication with peers of the shard; merging sub-shards from the root"},{"line_number":18,"context_line":"during audit potentially speeds their propagation to"},{"line_number":19,"context_line":"not-yet-replicated peers."},{"line_number":20,"context_line":""},{"line_number":21,"context_line":"Change-Id: I57aafc537ff94b081d0e1ea70e7fb7dd3598c61e"}],"source_content_type":"text/x-gerrit-commit-message","patch_set":5,"id":"80a2d533_2fa159ef","line":19,"updated":"2022-09-21 18:25:29.000000000","message":"is that the end goal, tho?  do we *want* to have two ways of getting the same information?","commit_id":"f5024a3a70e2e50dcf114ed8f3048a9fb1a1afb3"},{"author":{"_account_id":7233,"name":"Matthew Oliver","email":"matt@oliver.net.au","username":"mattoliverau"},"change_message_id":"69ef876f03776aeaea00c0587082e436c901c0ff","unresolved":true,"context_lines":[{"line_number":16,"context_line":"Children sub-shards would previously only have been merged during"},{"line_number":17,"context_line":"replication with peers of the shard; merging sub-shards from the root"},{"line_number":18,"context_line":"during audit potentially speeds their propagation to"},{"line_number":19,"context_line":"not-yet-replicated peers."},{"line_number":20,"context_line":""},{"line_number":21,"context_line":"Change-Id: I57aafc537ff94b081d0e1ea70e7fb7dd3598c61e"}],"source_content_type":"text/x-gerrit-commit-message","patch_set":5,"id":"a805b5bb_523513af","line":19,"in_reply_to":"4f4865dd_6965ab3e","updated":"2022-10-10 04:36:24.000000000","message":"Might also help with the quick sharding case after we land the patch that blocks smaller cleaves, as we won\u0027t have to wait for a replication cycle and a shard pass before we make progress. I like it.","commit_id":"f5024a3a70e2e50dcf114ed8f3048a9fb1a1afb3"},{"author":{"_account_id":7847,"name":"Alistair Coles","email":"alistairncoles@gmail.com","username":"acoles"},"change_message_id":"b6748f456a9497931ecfd48b399db753b85cfcf7","unresolved":true,"context_lines":[{"line_number":16,"context_line":"Children sub-shards would previously only have been merged during"},{"line_number":17,"context_line":"replication with peers of the shard; merging sub-shards from the root"},{"line_number":18,"context_line":"during audit potentially speeds their propagation to"},{"line_number":19,"context_line":"not-yet-replicated peers."},{"line_number":20,"context_line":""},{"line_number":21,"context_line":"Change-Id: I57aafc537ff94b081d0e1ea70e7fb7dd3598c61e"}],"source_content_type":"text/x-gerrit-commit-message","patch_set":5,"id":"4f4865dd_6965ab3e","line":19,"in_reply_to":"80a2d533_2fa159ef","updated":"2022-09-23 10:22:58.000000000","message":"The end goal is that other shard ranges will also be merged from root to avoid stuck DBs, but once I realised that \"other\" would include children I also realised that children did not need to be subject to the same checks and balances as \"other\" in general, so it started to feel like an incremental behavioural change that would be easier to digest. I also discovered that this change (merging children) specifically broke a probe test and i am glad to have debugged that without the added complexity of thinking about other shard ranges.\n\nThis patch is more of a step along the way than an end-goal, although now we have learnt how to do is_child_of(), it is arguably a case of why would we not do this i.e. replicate known useful state via whichever path possible?","commit_id":"f5024a3a70e2e50dcf114ed8f3048a9fb1a1afb3"}],"/PATCHSET_LEVEL":[{"author":{"_account_id":1179,"name":"Clay Gerrard","email":"clay.gerrard@gmail.com","username":"clay-gerrard"},"change_message_id":"48aff7ea454ad70e7549ecffcba774a504481590","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":5,"id":"e2987e55_7f792968","updated":"2022-09-21 18:25:29.000000000","message":"maybe this is fine - it doesn\u0027t seem to be an end onto itself","commit_id":"f5024a3a70e2e50dcf114ed8f3048a9fb1a1afb3"},{"author":{"_account_id":7233,"name":"Matthew Oliver","email":"matt@oliver.net.au","username":"mattoliverau"},"change_message_id":"698e50ae6522c129382fb8a3449df123e8f4a11d","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":9,"id":"00bac1ae_4451cccf","updated":"2022-10-10 21:53:41.000000000","message":"I\u0027m getting this in. Nice use of shard children to speed up inter replica shardrange sharing.","commit_id":"a46f2324ab926ae71d3a4d3214267ffd82fd1ee8"},{"author":{"_account_id":7233,"name":"Matthew Oliver","email":"matt@oliver.net.au","username":"mattoliverau"},"change_message_id":"b820d55c6fd02667b18524a77ad32ba23f61873f","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":9,"id":"7af0b273_bdb8f683","updated":"2022-10-10 05:45:33.000000000","message":"Yeah this looks really cool and no-brainer. It basically will speed up shard replica \u003c-\u003e shard replica shardrange replication, allowing to come from the root. If roots view is older, then merge_shards should stop it actaully merging.","commit_id":"a46f2324ab926ae71d3a4d3214267ffd82fd1ee8"},{"author":{"_account_id":7233,"name":"Matthew Oliver","email":"matt@oliver.net.au","username":"mattoliverau"},"change_message_id":"a78e1ca3e85c1ad8b1f8ae823ffe9d8c0d670ee8","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":9,"id":"a81160e3_8ef5bb73","updated":"2022-10-11 06:47:46.000000000","message":"recheck","commit_id":"a46f2324ab926ae71d3a4d3214267ffd82fd1ee8"},{"author":{"_account_id":7233,"name":"Matthew Oliver","email":"matt@oliver.net.au","username":"mattoliverau"},"change_message_id":"ad5541f69d6594acac38100391c1a33b1c479717","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":9,"id":"fbad00f0_7ea84537","updated":"2022-10-11 00:32:19.000000000","message":"recheck","commit_id":"a46f2324ab926ae71d3a4d3214267ffd82fd1ee8"},{"author":{"_account_id":7847,"name":"Alistair Coles","email":"alistairncoles@gmail.com","username":"acoles"},"change_message_id":"ec26377a8810347c87f17ad8d65d1d797c3e2d28","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":9,"id":"eb69ba7d_015d027d","updated":"2022-10-11 17:39:45.000000000","message":"recheck\n\nrolling upgrade test has hopefully been fixed on master https://review.opendev.org/c/openstack/swift/+/860937","commit_id":"a46f2324ab926ae71d3a4d3214267ffd82fd1ee8"}],"swift/common/utils.py":[{"author":{"_account_id":7847,"name":"Alistair Coles","email":"alistairncoles@gmail.com","username":"acoles"},"change_message_id":"02ac32541e32c84fe080e56af2ad4f258e1d04f5","unresolved":true,"context_lines":[{"line_number":5172,"context_line":"                                self.parent_container_hash,"},{"line_number":5173,"context_line":"                                self.timestamp.internal,"},{"line_number":5174,"context_line":"                                self.index)"},{"line_number":5175,"context_line":""},{"line_number":5176,"context_line":"    @classmethod"},{"line_number":5177,"context_line":"    def _validate(cls, arg):"},{"line_number":5178,"context_line":"        if arg is None:"}],"source_content_type":"text/x-python","patch_set":1,"id":"4b269604_881735af","line":5175,"updated":"2022-09-19 17:44:03.000000000","message":"this change is not actually be needed until the next patch in the chain. will fix.","commit_id":"4379fcea7da66f6f5a9f6f400e3ad1ccb4b10259"}],"swift/container/sharder.py":[{"author":{"_account_id":7847,"name":"Alistair Coles","email":"alistairncoles@gmail.com","username":"acoles"},"change_message_id":"02ac32541e32c84fe080e56af2ad4f258e1d04f5","unresolved":true,"context_lines":[{"line_number":1208,"context_line":"        children_shard_ranges \u003d [sr for sr in other_shard_ranges"},{"line_number":1209,"context_line":"                                 if sr.is_child_of(own_shard_range)]"},{"line_number":1210,"context_line":"        if children_shard_ranges:"},{"line_number":1211,"context_line":"            self.logger.debug(\u0027Updating children shard ranges from root\u0027)"},{"line_number":1212,"context_line":"            broker.merge_shard_ranges(children_shard_ranges)"},{"line_number":1213,"context_line":""},{"line_number":1214,"context_line":"        if (other_shard_ranges and"}],"source_content_type":"text/x-python","patch_set":1,"id":"bacced63_e3481f2f","line":1211,"updated":"2022-09-19 17:44:03.000000000","message":"nit: include number of shard ranges","commit_id":"4379fcea7da66f6f5a9f6f400e3ad1ccb4b10259"},{"author":{"_account_id":7847,"name":"Alistair Coles","email":"alistairncoles@gmail.com","username":"acoles"},"change_message_id":"5397464e70205213ed187764d6da004bab0462e1","unresolved":false,"context_lines":[{"line_number":1208,"context_line":"        children_shard_ranges \u003d [sr for sr in other_shard_ranges"},{"line_number":1209,"context_line":"                                 if sr.is_child_of(own_shard_range)]"},{"line_number":1210,"context_line":"        if children_shard_ranges:"},{"line_number":1211,"context_line":"            self.logger.debug(\u0027Updating children shard ranges from root\u0027)"},{"line_number":1212,"context_line":"            broker.merge_shard_ranges(children_shard_ranges)"},{"line_number":1213,"context_line":""},{"line_number":1214,"context_line":"        if (other_shard_ranges and"}],"source_content_type":"text/x-python","patch_set":1,"id":"333c685b_ff364c71","line":1211,"in_reply_to":"bacced63_e3481f2f","updated":"2022-09-19 17:45:10.000000000","message":"Done","commit_id":"4379fcea7da66f6f5a9f6f400e3ad1ccb4b10259"},{"author":{"_account_id":7847,"name":"Alistair Coles","email":"alistairncoles@gmail.com","username":"acoles"},"change_message_id":"5397464e70205213ed187764d6da004bab0462e1","unresolved":true,"context_lines":[{"line_number":1207,"context_line":""},{"line_number":1208,"context_line":"        children_shard_ranges \u003d [sr for sr in other_shard_ranges"},{"line_number":1209,"context_line":"                                 if sr.is_child_of(own_shard_range)]"},{"line_number":1210,"context_line":"        if children_shard_ranges:"},{"line_number":1211,"context_line":"            self.logger.debug(\u0027Updating %d children shard ranges from root\u0027,"},{"line_number":1212,"context_line":"                              len(children_shard_ranges))"},{"line_number":1213,"context_line":"            broker.merge_shard_ranges(children_shard_ranges)"}],"source_content_type":"text/x-python","patch_set":2,"id":"491b8392_febdc731","line":1210,"updated":"2022-09-19 17:45:10.000000000","message":"maybe this should not happen once the DB is sharded (so we freeze state of sub-shards)","commit_id":"1f699008eca29b096a4484b0a7e42dca538e4b43"},{"author":{"_account_id":1179,"name":"Clay Gerrard","email":"clay.gerrard@gmail.com","username":"clay-gerrard"},"change_message_id":"48aff7ea454ad70e7549ecffcba774a504481590","unresolved":true,"context_lines":[{"line_number":1207,"context_line":""},{"line_number":1208,"context_line":"        children_shard_ranges \u003d [sr for sr in other_shard_ranges"},{"line_number":1209,"context_line":"                                 if sr.is_child_of(own_shard_range)]"},{"line_number":1210,"context_line":"        if children_shard_ranges:"},{"line_number":1211,"context_line":"            self.logger.debug(\u0027Updating %d children shard ranges from root\u0027,"},{"line_number":1212,"context_line":"                              len(children_shard_ranges))"},{"line_number":1213,"context_line":"            broker.merge_shard_ranges(children_shard_ranges)"}],"source_content_type":"text/x-python","patch_set":2,"id":"860cf87e_fcf7de47","line":1210,"in_reply_to":"491b8392_febdc731","updated":"2022-09-21 18:25:29.000000000","message":"I don\u0027t have a clear picture of the data-flow for shard_range state, but I think I like where this is going (i.e. get more shard rnage state from root)\n\nI sort of think I could more easily get behind:\n\n\"always merge what the root has ... unless we can\u0027t because of these bug/race things we work around\"\n\nmore easily than\n\n\"mostly we just replicate between peers, but don\u0027t forget *sometimes* we may also merge what the root has ... but only when XYZ\"\n\n... i guess my assumption is the \"always merge what the root has\" wouldn\u0027t have any effect if what the root has is old/stale - maybe it\u0027s not always easy to detect that.","commit_id":"1f699008eca29b096a4484b0a7e42dca538e4b43"},{"author":{"_account_id":7847,"name":"Alistair Coles","email":"alistairncoles@gmail.com","username":"acoles"},"change_message_id":"b6748f456a9497931ecfd48b399db753b85cfcf7","unresolved":true,"context_lines":[{"line_number":1207,"context_line":""},{"line_number":1208,"context_line":"        children_shard_ranges \u003d [sr for sr in other_shard_ranges"},{"line_number":1209,"context_line":"                                 if sr.is_child_of(own_shard_range)]"},{"line_number":1210,"context_line":"        if children_shard_ranges:"},{"line_number":1211,"context_line":"            self.logger.debug(\u0027Updating %d children shard ranges from root\u0027,"},{"line_number":1212,"context_line":"                              len(children_shard_ranges))"},{"line_number":1213,"context_line":"            broker.merge_shard_ranges(children_shard_ranges)"}],"source_content_type":"text/x-python","patch_set":2,"id":"8bee55ff_408af89b","line":1210,"in_reply_to":"860cf87e_fcf7de47","updated":"2022-09-23 10:22:58.000000000","message":"maybe the next patchset addresses that. This one is meant to be the no-brainer: it\u0027s always safe to merge known-children, the more general case needs more care.","commit_id":"1f699008eca29b096a4484b0a7e42dca538e4b43"},{"author":{"_account_id":7233,"name":"Matthew Oliver","email":"matt@oliver.net.au","username":"mattoliverau"},"change_message_id":"69ef876f03776aeaea00c0587082e436c901c0ff","unresolved":true,"context_lines":[{"line_number":1207,"context_line":""},{"line_number":1208,"context_line":"        children_shard_ranges \u003d [sr for sr in other_shard_ranges"},{"line_number":1209,"context_line":"                                 if sr.is_child_of(own_shard_range)]"},{"line_number":1210,"context_line":"        if children_shard_ranges:"},{"line_number":1211,"context_line":"            self.logger.debug(\u0027Updating %d children shard ranges from root\u0027,"},{"line_number":1212,"context_line":"                              len(children_shard_ranges))"},{"line_number":1213,"context_line":"            broker.merge_shard_ranges(children_shard_ranges)"}],"source_content_type":"text/x-python","patch_set":2,"id":"415ed9b0_ced41d6c","line":1210,"in_reply_to":"8bee55ff_408af89b","updated":"2022-10-10 04:36:24.000000000","message":"+1. Faster syncing between peers of children seems like a no brainer to me. And moving toward the syncing of root is a good end goal (though I might need to marinate of on next patch in the chain a bit tho).","commit_id":"1f699008eca29b096a4484b0a7e42dca538e4b43"}],"test/probe/test_sharder.py":[{"author":{"_account_id":7847,"name":"Alistair Coles","email":"alistairncoles@gmail.com","username":"acoles"},"change_message_id":"02ac32541e32c84fe080e56af2ad4f258e1d04f5","unresolved":true,"context_lines":[{"line_number":1120,"context_line":"            number\u003dshard_1_nodes[2],"},{"line_number":1121,"context_line":"            additional_args\u003d\u0027--partitions\u003d%s\u0027 % shard_1_part)"},{"line_number":1122,"context_line":""},{"line_number":1123,"context_line":"        # third replica is sharding but has no sub-shard ranges yet..."},{"line_number":1124,"context_line":"        found_for_shard \u003d self.categorize_container_dir_content("},{"line_number":1125,"context_line":"            shard_1.account, shard_1.container)"},{"line_number":1126,"context_line":"        self.assertLengthEqual(found_for_shard[\u0027shard_dbs\u0027], 2)"}],"source_content_type":"text/x-python","patch_set":1,"id":"3edbf752_4544575e","side":"PARENT","line":1123,"range":{"start_line":1123,"start_character":36,"end_line":1123,"end_character":70},"updated":"2022-09-19 17:44:03.000000000","message":"this is no longer true because the *sharder* will fetch the sub-shards from root - we no longer rely on the replicator being able to sync the sub-shards between peers","commit_id":"f4849240731a2563e8a51ba6aa94bfc5369de8a1"},{"author":{"_account_id":7847,"name":"Alistair Coles","email":"alistairncoles@gmail.com","username":"acoles"},"change_message_id":"310d8a705a2dd5074b4618ca20107042dc568345","unresolved":false,"context_lines":[{"line_number":1120,"context_line":"            number\u003dshard_1_nodes[2],"},{"line_number":1121,"context_line":"            additional_args\u003d\u0027--partitions\u003d%s\u0027 % shard_1_part)"},{"line_number":1122,"context_line":""},{"line_number":1123,"context_line":"        # third replica is sharding but has no sub-shard ranges yet..."},{"line_number":1124,"context_line":"        found_for_shard \u003d self.categorize_container_dir_content("},{"line_number":1125,"context_line":"            shard_1.account, shard_1.container)"},{"line_number":1126,"context_line":"        self.assertLengthEqual(found_for_shard[\u0027shard_dbs\u0027], 2)"}],"source_content_type":"text/x-python","patch_set":1,"id":"5ac6ec0e_ee3cc053","side":"PARENT","line":1123,"range":{"start_line":1123,"start_character":36,"end_line":1123,"end_character":70},"in_reply_to":"3edbf752_4544575e","updated":"2022-09-28 15:39:20.000000000","message":"Done","commit_id":"f4849240731a2563e8a51ba6aa94bfc5369de8a1"}],"test/unit/container/test_sharder.py":[{"author":{"_account_id":1179,"name":"Clay Gerrard","email":"clay.gerrard@gmail.com","username":"clay-gerrard"},"change_message_id":"48aff7ea454ad70e7549ecffcba774a504481590","unresolved":true,"context_lines":[{"line_number":6029,"context_line":"    def test_audit_shard_container_children_merged_while_sharding(self):"},{"line_number":6030,"context_line":"        # Verify that sharding shard will always merge children shard ranges"},{"line_number":6031,"context_line":"        for child_deleted in (False, True):"},{"line_number":6032,"context_line":"            for child_state in ShardRange.STATES:"},{"line_number":6033,"context_line":"                root_sr \u003d ShardRange(\u0027a/root\u0027, next(self.ts_iter),"},{"line_number":6034,"context_line":"                                     state\u003dShardRange.SHARDED)"},{"line_number":6035,"context_line":"                parent_path \u003d ShardRange.make_path("}],"source_content_type":"text/x-python","patch_set":5,"id":"f8bedae7_2dca563f","line":6032,"updated":"2022-09-21 18:25:29.000000000","message":"this is great coverage, but not so easy to debug.  \"where am i in the loop\" when i hit an AssertionError could be quicker with something like a \"subtest\"\n\nhttps://pypi.org/project/pytest-subtests/\n^ I use pytest as my runner already, but I don\u0027t know if it\u0027s required\n\nhttps://docs.python.org/3/library/unittest.html#distinguishing-test-iterations-using-subtests\n^ new in 3.4, I don\u0027t see a py2.7 backport","commit_id":"f5024a3a70e2e50dcf114ed8f3048a9fb1a1afb3"},{"author":{"_account_id":7847,"name":"Alistair Coles","email":"alistairncoles@gmail.com","username":"acoles"},"change_message_id":"310d8a705a2dd5074b4618ca20107042dc568345","unresolved":true,"context_lines":[{"line_number":6029,"context_line":"    def test_audit_shard_container_children_merged_while_sharding(self):"},{"line_number":6030,"context_line":"        # Verify that sharding shard will always merge children shard ranges"},{"line_number":6031,"context_line":"        for child_deleted in (False, True):"},{"line_number":6032,"context_line":"            for child_state in ShardRange.STATES:"},{"line_number":6033,"context_line":"                root_sr \u003d ShardRange(\u0027a/root\u0027, next(self.ts_iter),"},{"line_number":6034,"context_line":"                                     state\u003dShardRange.SHARDED)"},{"line_number":6035,"context_line":"                parent_path \u003d ShardRange.make_path("}],"source_content_type":"text/x-python","patch_set":5,"id":"0702dbb8_3f989969","line":6032,"in_reply_to":"f8bedae7_2dca563f","updated":"2022-09-28 15:39:20.000000000","message":"we have our own annotate_failure() context manager which should help","commit_id":"f5024a3a70e2e50dcf114ed8f3048a9fb1a1afb3"}]}