)]}'
{"/COMMIT_MSG":[{"author":{"_account_id":15343,"name":"Tim Burke","email":"tburke@nvidia.com","username":"tburke"},"change_message_id":"bab144d617a9d72c1f72a0f10897eb2430c69cdd","unresolved":false,"context_lines":[{"line_number":10,"context_line":"containers.  The container server *should* only return shard ranges"},{"line_number":11,"context_line":"that container objects between marker and end_marker range, but if the"},{"line_number":12,"context_line":"proxy were to receive shard ranges outside of the marker to end_marker"},{"line_number":13,"context_line":"range then it will now ignore them and log a warning."},{"line_number":14,"context_line":""},{"line_number":15,"context_line":"Change-Id: Ie72bf9f218fe4390d078ef776ec2130f99274b64"}],"source_content_type":"text/x-gerrit-commit-message","patch_set":1,"id":"5f7c97a3_2a8607dd","line":13,"updated":"2018-05-22 23:55:56.000000000","message":"So, who is ultimately responsible for verifying that the proxy has all appropriate ranges? My gut is to say that should be the container-server... it has as close to a complete picture as we\u0027re going to get; let it do the filtering and let the proxy be dumb. This has the benefit of making shard range payloads back to the proxy smaller as we improve our filtering.\n\nSo we ought to go plumb prefix and path through getting shard ranges in the container-server, and we can hand-wave away upgrade issues with \"well, sure, it\u0027s going to continue making useless requests during the upgrade, but it doesn\u0027t change the correctness of the results\".\n\nExcept delimiter queries *still suck* -- if the client asks for ?delimiter\u003d/ and the proxy gets back ranges\n\n \u0027\u0027 - \u0027a/b/c\u0027\n \u0027a/b/c\u0027 - \u0027a/f/g\u0027\n \u0027a/f/g\u0027 - \u0027a/q/w\u0027\n \u0027a/q/w\u0027 - \u0027a/z/x\u0027\n \u0027a/z/x\u0027 - \u0027blah blah\u0027\n\nonly the *proxy* knows whether it got a \u0027a/\u0027 subdir, and from which shard :-/\n\n*Further*.... if we\u0027re ever going to try to cache shard ranges in the proxy... we probably need to put *the whole list* in memcache... so maybe we *always* send *everything* and let the proxy be smart? IDK...","commit_id":"d85dfccbbc62f9e5c89632c3f8f313b801b48816"},{"author":{"_account_id":7847,"name":"Alistair Coles","email":"alistairncoles@gmail.com","username":"acoles"},"change_message_id":"3ef195e49cfdc5cdf364cbf98e116fa0c9868e33","unresolved":false,"context_lines":[{"line_number":10,"context_line":"containers.  The container server *should* only return shard ranges"},{"line_number":11,"context_line":"that container objects between marker and end_marker range, but if the"},{"line_number":12,"context_line":"proxy were to receive shard ranges outside of the marker to end_marker"},{"line_number":13,"context_line":"range then it will now ignore them and log a warning."},{"line_number":14,"context_line":""},{"line_number":15,"context_line":"Change-Id: Ie72bf9f218fe4390d078ef776ec2130f99274b64"}],"source_content_type":"text/x-gerrit-commit-message","patch_set":1,"id":"5f7c97a3_2a62823f","line":13,"in_reply_to":"5f7c97a3_2a8607dd","updated":"2018-05-23 09:10:47.000000000","message":"The container server should be responsible for returning the correct ranges[1]. This patch was purely paranoid defensiveness on my part when I reviewed the proxy code and asked \u0027oh but what if?\u0027.\n\n[1] just like the ocntainer server is singularly responsible for not returning objects oustide marker-\u003eend_marker etc","commit_id":"d85dfccbbc62f9e5c89632c3f8f313b801b48816"}],"swift/proxy/controllers/container.py":[{"author":{"_account_id":7847,"name":"Alistair Coles","email":"alistairncoles@gmail.com","username":"acoles"},"change_message_id":"719bad208e7aff9f6abc72e3fdc5b77bfa4016ad","unresolved":false,"context_lines":[{"line_number":181,"context_line":"        for shard_range in shard_ranges:"},{"line_number":182,"context_line":"            if not self._check_shard_range(shard_range, marker, end_marker,"},{"line_number":183,"context_line":"                                           reverse):"},{"line_number":184,"context_line":"                self.app.logger.warning("},{"line_number":185,"context_line":"                    \u0027Unexpected shard range %s\u0027, shard_range)"},{"line_number":186,"context_line":"                continue"},{"line_number":187,"context_line":"            params[\u0027limit\u0027] \u003d limit"}],"source_content_type":"text/x-python","patch_set":1,"id":"5f7c97a3_7472b3af","line":184,"range":{"start_line":184,"start_character":32,"end_line":184,"end_character":39},"updated":"2018-05-22 08:59:40.000000000","message":"I debated \u0027warning\u0027 vs \u0027error\u0027 with myself and opted for \u0027warning\u0027 on basis that we deal with it gracefully, but I\u0027d welcome opinions.","commit_id":"d85dfccbbc62f9e5c89632c3f8f313b801b48816"},{"author":{"_account_id":15343,"name":"Tim Burke","email":"tburke@nvidia.com","username":"tburke"},"change_message_id":"bab144d617a9d72c1f72a0f10897eb2430c69cdd","unresolved":false,"context_lines":[{"line_number":181,"context_line":"        for shard_range in shard_ranges:"},{"line_number":182,"context_line":"            if not self._check_shard_range(shard_range, marker, end_marker,"},{"line_number":183,"context_line":"                                           reverse):"},{"line_number":184,"context_line":"                self.app.logger.warning("},{"line_number":185,"context_line":"                    \u0027Unexpected shard range %s\u0027, shard_range)"},{"line_number":186,"context_line":"                continue"},{"line_number":187,"context_line":"            params[\u0027limit\u0027] \u003d limit"}],"source_content_type":"text/x-python","patch_set":1,"id":"5f7c97a3_8a763b06","line":184,"range":{"start_line":184,"start_character":32,"end_line":184,"end_character":39},"in_reply_to":"5f7c97a3_5959e832","updated":"2018-05-22 23:55:56.000000000","message":"Hm, what? What about 4xx/5xx? Wouldn\u0027t those have tripped https://github.com/openstack/swift/blob/master/swift/proxy/controllers/base.py#L2029-L2033 so listing is None and we bail out at line 169 above? Or are you guys talking about something else?","commit_id":"d85dfccbbc62f9e5c89632c3f8f313b801b48816"},{"author":{"_account_id":1179,"name":"Clay Gerrard","email":"clay.gerrard@gmail.com","username":"clay-gerrard"},"change_message_id":"c8a171db2d5376fea128a74b968d72b417ff39a3","unresolved":false,"context_lines":[{"line_number":181,"context_line":"        for shard_range in shard_ranges:"},{"line_number":182,"context_line":"            if not self._check_shard_range(shard_range, marker, end_marker,"},{"line_number":183,"context_line":"                                           reverse):"},{"line_number":184,"context_line":"                self.app.logger.warning("},{"line_number":185,"context_line":"                    \u0027Unexpected shard range %s\u0027, shard_range)"},{"line_number":186,"context_line":"                continue"},{"line_number":187,"context_line":"            params[\u0027limit\u0027] \u003d limit"}],"source_content_type":"text/x-python","patch_set":1,"id":"5f7c97a3_5959e832","line":184,"range":{"start_line":184,"start_character":32,"end_line":184,"end_character":39},"in_reply_to":"5f7c97a3_7472b3af","updated":"2018-05-22 17:26:21.000000000","message":"\u003e for example backends may return 400\u0027s if proxy has a bug\n\nAlong these lines we might just decide to explode and return 500?  Without this check we just return wrong/invalid results?","commit_id":"d85dfccbbc62f9e5c89632c3f8f313b801b48816"},{"author":{"_account_id":7847,"name":"Alistair Coles","email":"alistairncoles@gmail.com","username":"acoles"},"change_message_id":"3ef195e49cfdc5cdf364cbf98e116fa0c9868e33","unresolved":false,"context_lines":[{"line_number":181,"context_line":"        for shard_range in shard_ranges:"},{"line_number":182,"context_line":"            if not self._check_shard_range(shard_range, marker, end_marker,"},{"line_number":183,"context_line":"                                           reverse):"},{"line_number":184,"context_line":"                self.app.logger.warning("},{"line_number":185,"context_line":"                    \u0027Unexpected shard range %s\u0027, shard_range)"},{"line_number":186,"context_line":"                continue"},{"line_number":187,"context_line":"            params[\u0027limit\u0027] \u003d limit"}],"source_content_type":"text/x-python","patch_set":1,"id":"5f7c97a3_8a1cf6c1","line":184,"range":{"start_line":184,"start_character":32,"end_line":184,"end_character":39},"in_reply_to":"5f7c97a3_8a763b06","updated":"2018-05-23 09:10:47.000000000","message":"I was referring to other interactions where a backend server does not trust the proxy to always send valid requests, as a possible justification for the proxy not always trusting the backend.\n\nOne day there may be an upgrade path where old proxies might temporarily get unexpected shard ranges, but I can\u0027t predict that, and it currently seems unlikely, so can\u0027t make a strong argument for warning log vs 500 vs not bothering.","commit_id":"d85dfccbbc62f9e5c89632c3f8f313b801b48816"}]}