)]}'
{"/COMMIT_MSG":[{"author":{"_account_id":15343,"name":"Tim Burke","email":"tburke@nvidia.com","username":"tburke"},"change_message_id":"b8a4b5f2107574b56c78106276e3c3922838bebd","unresolved":true,"context_lines":[{"line_number":4,"context_line":"Commit:     Clay Gerrard \u003cclay.gerrard@gmail.com\u003e"},{"line_number":5,"context_line":"CommitDate: 2021-12-20 12:36:44 -0600"},{"line_number":6,"context_line":""},{"line_number":7,"context_line":"WIP: use async delete to expire all SLO"},{"line_number":8,"context_line":""},{"line_number":9,"context_line":"Change-Id: Ib340590c0fdb9f4ce072daba4b621c19ecd6f593"}],"source_content_type":"text/x-gerrit-commit-message","patch_set":1,"id":"e75bc3e9_5addfc0a","line":7,"updated":"2021-12-21 23:03:18.000000000","message":"A few bugs we might want to link to:\n\n- https://bugs.launchpad.net/swift/+bug/1514606\n- https://bugs.launchpad.net/python-swiftclient/+bug/1159951\n- https://bugs.launchpad.net/python-swiftclient/+bug/1478830\n\nGives me this feeling like expiring SLO segments when the manifest expires would probably be desirous ... but maybe that\u0027s just because the users that want and expect the existing behavior are already satisfied.","commit_id":"03b13e56444213ec3135f70cbd0e9d2458ffd9fa"}],"/PATCHSET_LEVEL":[{"author":{"_account_id":7233,"name":"Matthew Oliver","email":"matt@oliver.net.au","username":"mattoliverau"},"change_message_id":"2ea8fdf3d4b10326ef8b1c3d0af2e064ddc867d0","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":1,"id":"eaef0aab_3b74b69e","updated":"2021-12-22 06:58:10.000000000","message":"I quite like this approach because of it\u0027s generalisation. It uses a known method of deleting segments.\n\nOnly problem is what if a person doesn\u0027t want the segments deleted. On an MTU, there isn\u0027t an option (which is great). Is there a way we can opt out if say someone wants to expire an SLO\u0027s manifest but not the segments.","commit_id":"03b13e56444213ec3135f70cbd0e9d2458ffd9fa"},{"author":{"_account_id":1179,"name":"Clay Gerrard","email":"clay.gerrard@gmail.com","username":"clay-gerrard"},"change_message_id":"03198dcc526a0adf50258417fbd2d9e960a73a2b","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":1,"id":"269ebd93_8b2b55d7","updated":"2022-01-07 21:55:54.000000000","message":"I\u0027m pretty sure we decided to abandon this one","commit_id":"03b13e56444213ec3135f70cbd0e9d2458ffd9fa"},{"author":{"_account_id":15343,"name":"Tim Burke","email":"tburke@nvidia.com","username":"tburke"},"change_message_id":"b8a4b5f2107574b56c78106276e3c3922838bebd","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":1,"id":"ff4daa0a_cd3bbc55","updated":"2021-12-21 23:03:18.000000000","message":"Manually verified that given a symlink that points to an SLO, the symlink expiring won\u0027t cause the SLO to expire.\n\nI\u0027m nervous about\n- SLO not looking at X-If-Delete-At/X-Delete-At\n- an SLO that pulls from multiple containers or nests another SLO will get stuck\n- this is definitely an interaction clients will/should be interested in knowing about, but there\u0027s no sensible way to expose it in /info -- it depends upon object-expirer configuration, not proxy-server.\n\n","commit_id":"03b13e56444213ec3135f70cbd0e9d2458ffd9fa"},{"author":{"_account_id":1179,"name":"Clay Gerrard","email":"clay.gerrard@gmail.com","username":"clay-gerrard"},"change_message_id":"f38cdfb2f8e228a5eca641266fc7510446b79232","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":1,"id":"73dd2e74_1eb41d66","updated":"2021-12-20 23:32:51.000000000","message":"needs a rebase to fix the s3_acl var thing","commit_id":"03b13e56444213ec3135f70cbd0e9d2458ffd9fa"}],"doc/saio/swift/object-expirer.conf":[{"author":{"_account_id":15343,"name":"Tim Burke","email":"tburke@nvidia.com","username":"tburke"},"change_message_id":"b8a4b5f2107574b56c78106276e3c3922838bebd","unresolved":true,"context_lines":[{"line_number":51,"context_line":""},{"line_number":52,"context_line":"[filter:slo]"},{"line_number":53,"context_line":"use \u003d egg:swift#slo"},{"line_number":54,"context_line":"allow_async_delete \u003d true"},{"line_number":55,"context_line":""},{"line_number":56,"context_line":"[filter:cache]"},{"line_number":57,"context_line":"use \u003d egg:swift#memcache"}],"source_content_type":"text/plain","patch_set":1,"id":"5ca4ce0e_9c1f9b52","line":54,"updated":"2021-12-21 23:03:18.000000000","message":"I\u0027m starting to rethink the default -- off-by-default made sense when this option was new, but we\u0027ve had more than a year of it working well for us now. Submitted https://review.opendev.org/c/openstack/swift/+/822565 to enable by default.\n\nIf this *isn\u0027t* enabled, we just process the deletes inline, yeah? So things go slower, but it\u0027s not terrible.","commit_id":"03b13e56444213ec3135f70cbd0e9d2458ffd9fa"}],"swift/common/internal_client.py":[{"author":{"_account_id":7847,"name":"Alistair Coles","email":"alistairncoles@gmail.com","username":"acoles"},"change_message_id":"28ff534720976ba548075cdf84324a83f64d2eb7","unresolved":true,"context_lines":[{"line_number":664,"context_line":""},{"line_number":665,"context_line":"        path \u003d self.make_path(account, container, obj)"},{"line_number":666,"context_line":"        self.handle_request(\u0027DELETE\u0027, path, (headers or {}),"},{"line_number":667,"context_line":"                            acceptable_statuses, params\u003dparams)"},{"line_number":668,"context_line":""},{"line_number":669,"context_line":"    def get_object_metadata("},{"line_number":670,"context_line":"            self, account, container, obj, metadata_prefix\u003d\u0027\u0027,"}],"source_content_type":"text/x-python","patch_set":1,"id":"e0713935_67a6344d","line":667,"updated":"2021-12-22 12:01:35.000000000","message":"+1 I\u0027d like to have these internal client improvements (here and in the other patch)","commit_id":"03b13e56444213ec3135f70cbd0e9d2458ffd9fa"}],"swift/common/middleware/slo.py":[{"author":{"_account_id":15343,"name":"Tim Burke","email":"tburke@nvidia.com","username":"tburke"},"change_message_id":"b8a4b5f2107574b56c78106276e3c3922838bebd","unresolved":true,"context_lines":[{"line_number":1526,"context_line":"            \u0027x-backend-replication\u0027, \u0027false\u0027)"},{"line_number":1527,"context_line":"        resp \u003d Request.blank(\u0027\u0027, new_env).get_response(self.app)"},{"line_number":1528,"context_line":""},{"line_number":1529,"context_line":"        if resp.is_success:"},{"line_number":1530,"context_line":"            if config_true_value(resp.headers.get(\u0027X-Static-Large-Object\u0027)):"},{"line_number":1531,"context_line":"                try:"},{"line_number":1532,"context_line":"                    return json.loads(resp.body)"}],"source_content_type":"text/x-python","patch_set":1,"id":"c1833502_cc519eaf","line":1529,"updated":"2021-12-21 23:03:18.000000000","message":"If the client included an X-If-Delete-At header, shouldn\u0027t this look for X-Delete-At and 412 if they don\u0027t match?","commit_id":"03b13e56444213ec3135f70cbd0e9d2458ffd9fa"}],"swift/obj/expirer.py":[{"author":{"_account_id":1179,"name":"Clay Gerrard","email":"clay.gerrard@gmail.com","username":"clay-gerrard"},"change_message_id":"f38cdfb2f8e228a5eca641266fc7510446b79232","unresolved":true,"context_lines":[{"line_number":493,"context_line":"                       \u0027X-If-Delete-At\u0027: timestamp.normal,"},{"line_number":494,"context_line":"                       \u0027X-Backend-Clean-Expiring-Object-Queue\u0027: \u0027no\u0027}"},{"line_number":495,"context_line":"            acceptable_statuses \u003d (2, HTTP_CONFLICT)"},{"line_number":496,"context_line":"        headers[\u0027x-backend-replication\u0027] \u003d \u0027true\u0027"},{"line_number":497,"context_line":"        params \u003d {"},{"line_number":498,"context_line":"            \u0027multipart-manifest\u0027: \u0027delete\u0027,"},{"line_number":499,"context_line":"            \u0027async\u0027: \u0027on\u0027,"}],"source_content_type":"text/x-python","patch_set":1,"id":"6dd2ee61_6a941ab2","line":496,"updated":"2021-12-20 23:32:51.000000000","message":"maybe need to think about other side effects of piggy backing on ssync subrequests:\n\nhttps://github.com/openstack/swift/blob/master/swift/obj/server.py#L460-L462\n\nbut it kind of looks like it ends up doing the same thing as x-backend-clean-expiring-object-queue","commit_id":"03b13e56444213ec3135f70cbd0e9d2458ffd9fa"},{"author":{"_account_id":7847,"name":"Alistair Coles","email":"alistairncoles@gmail.com","username":"acoles"},"change_message_id":"28ff534720976ba548075cdf84324a83f64d2eb7","unresolved":true,"context_lines":[{"line_number":493,"context_line":"                       \u0027X-If-Delete-At\u0027: timestamp.normal,"},{"line_number":494,"context_line":"                       \u0027X-Backend-Clean-Expiring-Object-Queue\u0027: \u0027no\u0027}"},{"line_number":495,"context_line":"            acceptable_statuses \u003d (2, HTTP_CONFLICT)"},{"line_number":496,"context_line":"        headers[\u0027x-backend-replication\u0027] \u003d \u0027true\u0027"},{"line_number":497,"context_line":"        params \u003d {"},{"line_number":498,"context_line":"            \u0027multipart-manifest\u0027: \u0027delete\u0027,"},{"line_number":499,"context_line":"            \u0027async\u0027: \u0027on\u0027,"}],"source_content_type":"text/x-python","patch_set":1,"id":"1447f03e_3e3b6edb","line":496,"in_reply_to":"0fa5f6c9_031712ad","updated":"2021-12-22 12:01:35.000000000","message":"is there any reason why we would ever want to do an async multipart delete but NOT have an expired manifest\u0027s segments deleted? in other words, could the *effect* of headers[\u0027x-backend-replication\u0027] \u003d \u0027true\u0027 actually become the default SLO behaviour? (the concerns about side-effects of this actual headers remain)","commit_id":"03b13e56444213ec3135f70cbd0e9d2458ffd9fa"},{"author":{"_account_id":15343,"name":"Tim Burke","email":"tburke@nvidia.com","username":"tburke"},"change_message_id":"b8a4b5f2107574b56c78106276e3c3922838bebd","unresolved":true,"context_lines":[{"line_number":493,"context_line":"                       \u0027X-If-Delete-At\u0027: timestamp.normal,"},{"line_number":494,"context_line":"                       \u0027X-Backend-Clean-Expiring-Object-Queue\u0027: \u0027no\u0027}"},{"line_number":495,"context_line":"            acceptable_statuses \u003d (2, HTTP_CONFLICT)"},{"line_number":496,"context_line":"        headers[\u0027x-backend-replication\u0027] \u003d \u0027true\u0027"},{"line_number":497,"context_line":"        params \u003d {"},{"line_number":498,"context_line":"            \u0027multipart-manifest\u0027: \u0027delete\u0027,"},{"line_number":499,"context_line":"            \u0027async\u0027: \u0027on\u0027,"}],"source_content_type":"text/x-python","patch_set":1,"id":"0fa5f6c9_031712ad","line":496,"in_reply_to":"6dd2ee61_6a941ab2","updated":"2021-12-21 23:03:18.000000000","message":"Hmm... https://github.com/openstack/swift/blob/master/swift/obj/server.py#L1353-L1355","commit_id":"03b13e56444213ec3135f70cbd0e9d2458ffd9fa"},{"author":{"_account_id":15343,"name":"Tim Burke","email":"tburke@nvidia.com","username":"tburke"},"change_message_id":"b8a4b5f2107574b56c78106276e3c3922838bebd","unresolved":true,"context_lines":[{"line_number":496,"context_line":"        headers[\u0027x-backend-replication\u0027] \u003d \u0027true\u0027"},{"line_number":497,"context_line":"        params \u003d {"},{"line_number":498,"context_line":"            \u0027multipart-manifest\u0027: \u0027delete\u0027,"},{"line_number":499,"context_line":"            \u0027async\u0027: \u0027on\u0027,"},{"line_number":500,"context_line":"        }"},{"line_number":501,"context_line":"        try:"},{"line_number":502,"context_line":"            self.swift.delete_object(*split_path(\u0027/\u0027 + actual_obj, 3, 3, True),"}],"source_content_type":"text/x-python","patch_set":1,"id":"8e005343_38d40c23","line":499,"updated":"2021-12-21 23:03:18.000000000","message":"How well are these query params going to interact with the X-If-Delete-At header?","commit_id":"03b13e56444213ec3135f70cbd0e9d2458ffd9fa"},{"author":{"_account_id":7847,"name":"Alistair Coles","email":"alistairncoles@gmail.com","username":"acoles"},"change_message_id":"56e89421e6f4fee9a626e17494c591f93a0fc600","unresolved":true,"context_lines":[{"line_number":499,"context_line":"            \u0027async\u0027: \u0027on\u0027,"},{"line_number":500,"context_line":"        }"},{"line_number":501,"context_line":"        try:"},{"line_number":502,"context_line":"            self.swift.delete_object(*split_path(\u0027/\u0027 + actual_obj, 3, 3, True),"},{"line_number":503,"context_line":"                                     headers\u003dheaders, params\u003dparams,"},{"line_number":504,"context_line":"                                     acceptable_statuses\u003dacceptable_statuses)"},{"line_number":505,"context_line":"        except UnexpectedResponse as e:"}],"source_content_type":"text/x-python","patch_set":1,"id":"f4780e8e_a014a3b1","line":502,"updated":"2021-12-22 12:06:36.000000000","message":"if we did a HEAD here (with headers[\u0027x-backend-replication\u0027] \u003d \u0027true\u0027) then we\u0027d learn (a) if the object is a manifest and (b) if it\u0027s an s3 MPU, which could make the subsequent DELETE more targeted i.e. only include the params if deleting a known s3 MPU\n\ntrade-off is always 2 requests vs how many expirer tasks are manifests?","commit_id":"03b13e56444213ec3135f70cbd0e9d2458ffd9fa"},{"author":{"_account_id":15343,"name":"Tim Burke","email":"tburke@nvidia.com","username":"tburke"},"change_message_id":"b8a4b5f2107574b56c78106276e3c3922838bebd","unresolved":true,"context_lines":[{"line_number":504,"context_line":"                                     acceptable_statuses\u003dacceptable_statuses)"},{"line_number":505,"context_line":"        except UnexpectedResponse as e:"},{"line_number":506,"context_line":"            if not (e.resp.status_int \u003d\u003d HTTP_BAD_REQUEST and"},{"line_number":507,"context_line":"                    e.resp.body \u003d\u003d \u0027Not an SLO manifest\u0027):"},{"line_number":508,"context_line":"                raise"},{"line_number":509,"context_line":"            # just do a regular DELETE"},{"line_number":510,"context_line":"            params \u003d None"}],"source_content_type":"text/x-python","patch_set":1,"id":"ce3d4eed_785b5bce","line":507,"updated":"2021-12-21 23:03:18.000000000","message":"We want to compare the body to a byte string. On py3,\n\n \u0027Not an SLO manifest\u0027 !\u003d b\u0027Not an SLO manifest\u0027\n\nI still kind of want a first-class way to have the proxy take on this complexity, like was proposed in https://review.opendev.org/c/openstack/swift/+/463849 -- maybe as some new query-param?","commit_id":"03b13e56444213ec3135f70cbd0e9d2458ffd9fa"},{"author":{"_account_id":7233,"name":"Matthew Oliver","email":"matt@oliver.net.au","username":"mattoliverau"},"change_message_id":"2ea8fdf3d4b10326ef8b1c3d0af2e064ddc867d0","unresolved":true,"context_lines":[{"line_number":504,"context_line":"                                     acceptable_statuses\u003dacceptable_statuses)"},{"line_number":505,"context_line":"        except UnexpectedResponse as e:"},{"line_number":506,"context_line":"            if not (e.resp.status_int \u003d\u003d HTTP_BAD_REQUEST and"},{"line_number":507,"context_line":"                    e.resp.body \u003d\u003d \u0027Not an SLO manifest\u0027):"},{"line_number":508,"context_line":"                raise"},{"line_number":509,"context_line":"            # just do a regular DELETE"},{"line_number":510,"context_line":"            params \u003d None"}],"source_content_type":"text/x-python","patch_set":1,"id":"9cb12374_ee592581","line":507,"in_reply_to":"ce3d4eed_785b5bce","updated":"2021-12-22 06:58:10.000000000","message":"Another reason to migrate of py2 :P\n\nIf always first going to attempt to fire a delete_object assuming for an SLO manifest. Then fallback to firing off a delete_object without a `multipart-manifest: delete`. I do wonder if we should get the above 463849 landed. I mean I can go fire something like:\n\n curl -i -H \"X-Auth-Token: $OS_AUTH_TOKEN\" \"$OS_STORAGE_URL/ec/o?reverse\u003dtrue\" -X DELETE\n\nAnd it happily deletes the object. Because the param doesn\u0027t effect the object path, we just ignore it. So likewise with 463849.","commit_id":"03b13e56444213ec3135f70cbd0e9d2458ffd9fa"}]}