)]}'
{"/PATCHSET_LEVEL":[{"author":{"_account_id":9816,"name":"Takashi Kajinami","email":"kajinamit@oss.nttdata.com","username":"kajinamit"},"change_message_id":"3c0653040a3227294ac6a6782706bd335d10a9a3","unresolved":true,"context_lines":[],"source_content_type":"","patch_set":3,"id":"1d22b2eb_967c06f5","updated":"2022-08-28 23:37:58.000000000","message":"Also, there a few more places where get_container is called. Maybe we should update these as well ?","commit_id":"7803706a61ab5841d7034c7a2f91eaa42444fe5d"},{"author":{"_account_id":9816,"name":"Takashi Kajinami","email":"kajinamit@oss.nttdata.com","username":"kajinamit"},"change_message_id":"46ea9da3b5de54d4b26c5d2d44454fdbf3a25e7a","unresolved":true,"context_lines":[],"source_content_type":"","patch_set":6,"id":"9d279eda_0fce2c68","updated":"2022-08-29 06:49:53.000000000","message":"There are two more get_container call used to obtain list of objects. Can we also fix these ?\n\nhttps://github.com/openstack/tripleo-common/blob/eed18aa5a6756b04592573a3c1edd3688c6e9c24/tripleo_common/actions/templates.py#L200\nhttps://github.com/openstack/tripleo-common/blob/eed18aa5a6756b04592573a3c1edd3688c6e9c24/tripleo_common/actions/heat_capabilities.py#L58\n\nThe usage in validations.py limit scope by the specific folder so I don\u0027t expect it can really hit the listing limit.\nhttps://github.com/openstack/tripleo-common/blob/eed18aa5a6756b04592573a3c1edd3688c6e9c24/tripleo_common/utils/validations.py#L88-L101\n\nThe other usage in plan.py only reads the header field thus we don\u0027t need full listing.\nhttps://github.com/openstack/tripleo-common/blob/eed18aa5a6756b04592573a3c1edd3688c6e9c24/tripleo_common/actions/plan.py#L87\n\n\n","commit_id":"41ddbbdcb9545897e3330a77305ffbabf1a0d949"},{"author":{"_account_id":28223,"name":"Cedric Jeanneret","display_name":"cjeanner (Tengu)","email":"cjeanner@redhat.com","username":"cjeanner"},"change_message_id":"fa336dc59a8ef50e4ec3cd276e960b82ab49e027","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":8,"id":"cd8bae20_7481d8db","updated":"2022-08-29 10:59:53.000000000","message":"I\u0027m wondering about the perf impact. Is it better to get an infinite list loaded in memory, or to iterate on the pagination at this point?","commit_id":"1388f073b6a66293cd87cf16ffc5710a7b2f91f0"},{"author":{"_account_id":9816,"name":"Takashi Kajinami","email":"kajinamit@oss.nttdata.com","username":"kajinamit"},"change_message_id":"d0b1b041dfbcb6901508be016eae3073f89cf75a","unresolved":true,"context_lines":[],"source_content_type":"","patch_set":8,"id":"c0490ea2_52272764","in_reply_to":"47cf3577_84768fc4","updated":"2022-09-08 01:57:08.000000000","message":"I\u0027ve updated that wip patch which would allows us to understand how many changes we need to implement paging requests within tripleo-common. https://review.opendev.org/c/openstack/tripleo-common/+/855118 .","commit_id":"1388f073b6a66293cd87cf16ffc5710a7b2f91f0"},{"author":{"_account_id":9816,"name":"Takashi Kajinami","email":"kajinamit@oss.nttdata.com","username":"kajinamit"},"change_message_id":"6a5fcaee69b3f4b8676c692b0e5fc14a94e0e861","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":8,"id":"47cf3577_84768fc4","in_reply_to":"c29dd70b_8c64fe69","updated":"2022-08-30 05:40:00.000000000","message":"We haven\u0027t notice this problem until it was reported quite recently and this indicates hitting the limit is not really common. Enabling full listing does not affect the existing deployment but affects only a few large scale ones.\n\nWe might be able to split list process like https://review.opendev.org/c/openstack/tripleo-common/+/855118/ but that would introduce complicated logic. I tend to prefer this simple approach with full_string rather than making complex change in old stable/train at this moment.","commit_id":"1388f073b6a66293cd87cf16ffc5710a7b2f91f0"},{"author":{"_account_id":30073,"name":"Brendan Shephard","email":"bshephar@bne-home.net","username":"bshephar"},"change_message_id":"0dcb4be83274a9eae87f410bf1968434f6d1b0ec","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":8,"id":"c29dd70b_8c64fe69","in_reply_to":"cd8bae20_7481d8db","updated":"2022-08-29 22:32:19.000000000","message":"I\u0027m not too sure there will ever be enough objects in that git repo to have a significant impact on memory? I would say repeated API calls are probably the biggest enemy here. \n\nI marked this as WIP though, as I would like to figure out why we\u0027re looping over the objects and deleting them one by one before deleting the container. It seems to me that we should be able to just delete the container?\n\nI\u0027ll test this theory anyway. Leaving it WIP in the meantime.","commit_id":"1388f073b6a66293cd87cf16ffc5710a7b2f91f0"},{"author":{"_account_id":30073,"name":"Brendan Shephard","email":"bshephar@bne-home.net","username":"bshephar"},"change_message_id":"7d8e2f19458996b901e7b3b2585e92a24be1a970","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":10,"id":"a156754b_3200d559","updated":"2022-09-02 03:21:00.000000000","message":"Looks like this is causing a failure during the update:\n2022-08-31 13:44:29.697 7 ERROR mistral.executors.default_executor   File \"/usr/lib/python3.6/site-packages/swiftclient/client.py\", line 1170, in delete_container\n17890\t2022-08-31 13:44:29.697 7 ERROR mistral.executors.default_executor     resp, \u0027Container DELETE failed\u0027, body)\n17891\t2022-08-31 13:44:29.697 7 ERROR mistral.executors.default_executor swiftclient.exceptions.ClientException: Container DELETE failed: http://192.168.24.1:8080/v1/AUTH_cb74c8faf96a40929a521708ef804490/overcloud-config 409 Conflict  [first 60 chars of response] b\u0027\u003chtml\u003e\u003ch1\u003eConflict\u003c/h1\u003e\u003cp\u003eThere was a conflict when trying t\u0027\n17892\t2022-08-31 13:44:29.697 7 ERROR mistral.executors.default_executor \n\n\nSo we\u0027ll need to go back to emptying the container first, with the full_listing.","commit_id":"083a49faa87b20917b9c29fd044403719611a753"},{"author":{"_account_id":30073,"name":"Brendan Shephard","email":"bshephar@bne-home.net","username":"bshephar"},"change_message_id":"7f7e01966ff081298a712e1cf15f11cf6c17064c","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":10,"id":"d26a6795_ba0ce29c","updated":"2022-08-31 10:09:12.000000000","message":"recheck","commit_id":"083a49faa87b20917b9c29fd044403719611a753"},{"author":{"_account_id":30073,"name":"Brendan Shephard","email":"bshephar@bne-home.net","username":"bshephar"},"change_message_id":"5519583f2973a0441b7198a215b9db16bdd587c5","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":11,"id":"49dd78ca_310be39c","updated":"2022-09-05 03:30:05.000000000","message":"recheck","commit_id":"d4a29b5788bfcc6699caf66344a48c045f3cbafe"},{"author":{"_account_id":30073,"name":"Brendan Shephard","email":"bshephar@bne-home.net","username":"bshephar"},"change_message_id":"afdd6f33cd6083c5d029b05250068870a816cf52","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":11,"id":"4da8b2af_bf7a8d00","updated":"2022-09-05 12:15:03.000000000","message":"recheck","commit_id":"d4a29b5788bfcc6699caf66344a48c045f3cbafe"}],"tripleo_common/actions/base.py":[{"author":{"_account_id":9816,"name":"Takashi Kajinami","email":"kajinamit@oss.nttdata.com","username":"kajinamit"},"change_message_id":"0df3441c51e839d5c710fcf92db97f88b2f8356e","unresolved":true,"context_lines":[{"line_number":64,"context_line":"            \u0027retries\u0027: 10,"},{"line_number":65,"context_line":"            \u0027starting_backoff\u0027: 3,"},{"line_number":66,"context_line":"            \u0027max_backoff\u0027: 120,"},{"line_number":67,"context_line":"            \u0027full_listing\u0027: True"},{"line_number":68,"context_line":"        }"},{"line_number":69,"context_line":"        return swift_client.Connection(**kwargs)"},{"line_number":70,"context_line":""}],"source_content_type":"text/x-python","patch_set":8,"id":"63016f7d_6bfabcd6","line":67,"range":{"start_line":67,"start_character":12,"end_line":67,"end_character":32},"updated":"2022-08-30 05:43:46.000000000","message":"The Connection class does not accept full_listing IIUC.\n\nhttps://github.com/openstack/python-swiftclient/blob/34ffb6a83deb1692d88e1ab502d52a2ccb5027ea/swiftclient/client.py#L1585-L1591","commit_id":"1388f073b6a66293cd87cf16ffc5710a7b2f91f0"}],"tripleo_common/utils/swift.py":[{"author":{"_account_id":9816,"name":"Takashi Kajinami","email":"kajinamit@oss.nttdata.com","username":"kajinamit"},"change_message_id":"cf4cf66c830c4b3442a67683d478460f0c0cef8d","unresolved":true,"context_lines":[{"line_number":33,"context_line":"                       in swiftclient.get_account()[1]]"},{"line_number":34,"context_line":""},{"line_number":35,"context_line":"    if name in container_names:"},{"line_number":36,"context_line":"        headers, objects \u003d swiftclient.get_container(name)"},{"line_number":37,"context_line":"        # FIXME(rbrady): remove delete_object loop when"},{"line_number":38,"context_line":"        # LP#1615830 is fixed.  See LP#1615825 for more info."},{"line_number":39,"context_line":"        # delete files from plan"}],"source_content_type":"text/x-python","patch_set":3,"id":"5f0a1a67_6af1c7b8","line":36,"range":{"start_line":36,"start_character":27,"end_line":36,"end_character":58},"updated":"2022-08-28 23:34:53.000000000","message":"You also need to modify this one (I think this is the logic causing the original issue)","commit_id":"7803706a61ab5841d7034c7a2f91eaa42444fe5d"},{"author":{"_account_id":30073,"name":"Brendan Shephard","email":"bshephar@bne-home.net","username":"bshephar"},"change_message_id":"c8c3549779fb745cff2f4a92b857128ec6e28daa","unresolved":false,"context_lines":[{"line_number":33,"context_line":"                       in swiftclient.get_account()[1]]"},{"line_number":34,"context_line":""},{"line_number":35,"context_line":"    if name in container_names:"},{"line_number":36,"context_line":"        headers, objects \u003d swiftclient.get_container(name)"},{"line_number":37,"context_line":"        # FIXME(rbrady): remove delete_object loop when"},{"line_number":38,"context_line":"        # LP#1615830 is fixed.  See LP#1615825 for more info."},{"line_number":39,"context_line":"        # delete files from plan"}],"source_content_type":"text/x-python","patch_set":3,"id":"289f2b39_143e36ae","line":36,"range":{"start_line":36,"start_character":27,"end_line":36,"end_character":58},"in_reply_to":"5f0a1a67_6af1c7b8","updated":"2022-08-29 06:40:43.000000000","message":"Done","commit_id":"7803706a61ab5841d7034c7a2f91eaa42444fe5d"},{"author":{"_account_id":9816,"name":"Takashi Kajinami","email":"kajinamit@oss.nttdata.com","username":"kajinamit"},"change_message_id":"cf4cf66c830c4b3442a67683d478460f0c0cef8d","unresolved":true,"context_lines":[{"line_number":61,"context_line":"    # In large environments. It\u0027s possible that we have greater than 10,000"},{"line_number":62,"context_line":"    # objects in the container. We need to call get_container() with"},{"line_number":63,"context_line":"    # full_listing\u003dTrue"},{"line_number":64,"context_line":"    # https://bugzilla.redhat.com/show_bug.cgi?id\u003d2119145"},{"line_number":65,"context_line":"    objects \u003d swiftclient.get_container(container, full_listing\u003dTrue)[1]"},{"line_number":66,"context_line":""},{"line_number":67,"context_line":"    for obj in objects:"}],"source_content_type":"text/x-python","patch_set":3,"id":"d88469ca_66df7523","line":64,"range":{"start_line":64,"start_character":6,"end_line":64,"end_character":57},"updated":"2022-08-28 23:34:53.000000000","message":"description of bug is private and this does not provide much information for people outside of Red Hat. I tend to remove this.","commit_id":"7803706a61ab5841d7034c7a2f91eaa42444fe5d"},{"author":{"_account_id":30073,"name":"Brendan Shephard","email":"bshephar@bne-home.net","username":"bshephar"},"change_message_id":"c8c3549779fb745cff2f4a92b857128ec6e28daa","unresolved":false,"context_lines":[{"line_number":61,"context_line":"    # In large environments. It\u0027s possible that we have greater than 10,000"},{"line_number":62,"context_line":"    # objects in the container. We need to call get_container() with"},{"line_number":63,"context_line":"    # full_listing\u003dTrue"},{"line_number":64,"context_line":"    # https://bugzilla.redhat.com/show_bug.cgi?id\u003d2119145"},{"line_number":65,"context_line":"    objects \u003d swiftclient.get_container(container, full_listing\u003dTrue)[1]"},{"line_number":66,"context_line":""},{"line_number":67,"context_line":"    for obj in objects:"}],"source_content_type":"text/x-python","patch_set":3,"id":"88d95d29_966b1224","line":64,"range":{"start_line":64,"start_character":6,"end_line":64,"end_character":57},"in_reply_to":"d88469ca_66df7523","updated":"2022-08-29 06:40:43.000000000","message":"Done","commit_id":"7803706a61ab5841d7034c7a2f91eaa42444fe5d"},{"author":{"_account_id":8833,"name":"Rabi Mishra","email":"ramishra@redhat.com","username":"rabi"},"change_message_id":"66a2d0d13606052193f2f8cc6cc1a0b5464fedbb","unresolved":true,"context_lines":[{"line_number":33,"context_line":"                       in swiftclient.get_account()[1]]"},{"line_number":34,"context_line":""},{"line_number":35,"context_line":"    if name in container_names:"},{"line_number":36,"context_line":"        headers, objects \u003d swiftclient.get_container(name, full_listing\u003dTrue)"},{"line_number":37,"context_line":"        # FIXME(rbrady): remove delete_object loop when"},{"line_number":38,"context_line":"        # LP#1615830 is fixed.  See LP#1615825 for more info."},{"line_number":39,"context_line":"        # delete files from plan"}],"source_content_type":"text/x-python","patch_set":11,"id":"e7b49b3d_ad9256c0","line":36,"range":{"start_line":36,"start_character":59,"end_line":36,"end_character":76},"updated":"2022-09-12 02:32:25.000000000","message":"I agree with the performance concerns of looping through a list of too many items. Do we really need to store git history in swift? I don\u0027t think we make the config-downloaded dir a git repo in master/wallaby anymore. Anyway, config-download takes ages atm, so probably this won\u0027t make it too worse.","commit_id":"d4a29b5788bfcc6699caf66344a48c045f3cbafe"},{"author":{"_account_id":30073,"name":"Brendan Shephard","email":"bshephar@bne-home.net","username":"bshephar"},"change_message_id":"89b4ac0cbfafbbf6979ebc7cd18975ea35c391c3","unresolved":true,"context_lines":[{"line_number":33,"context_line":"                       in swiftclient.get_account()[1]]"},{"line_number":34,"context_line":""},{"line_number":35,"context_line":"    if name in container_names:"},{"line_number":36,"context_line":"        headers, objects \u003d swiftclient.get_container(name, full_listing\u003dTrue)"},{"line_number":37,"context_line":"        # FIXME(rbrady): remove delete_object loop when"},{"line_number":38,"context_line":"        # LP#1615830 is fixed.  See LP#1615825 for more info."},{"line_number":39,"context_line":"        # delete files from plan"}],"source_content_type":"text/x-python","patch_set":11,"id":"74b4d7ff_374297f6","line":36,"range":{"start_line":36,"start_character":59,"end_line":36,"end_character":76},"in_reply_to":"e7b49b3d_ad9256c0","updated":"2022-09-12 03:47:29.000000000","message":"Do we want to rebase on Takashi\u0027s patch to paginate these then?\nhttps://review.opendev.org/c/openstack/tripleo-common/+/855118","commit_id":"d4a29b5788bfcc6699caf66344a48c045f3cbafe"},{"author":{"_account_id":8833,"name":"Rabi Mishra","email":"ramishra@redhat.com","username":"rabi"},"change_message_id":"66a2d0d13606052193f2f8cc6cc1a0b5464fedbb","unresolved":true,"context_lines":[{"line_number":58,"context_line":"                       overwrite_only_newer\u003dFalse):"},{"line_number":59,"context_line":"    \"\"\"Download the contents of a Swift container to a directory\"\"\""},{"line_number":60,"context_line":""},{"line_number":61,"context_line":"    # In large environments. It\u0027s possible that we have greater than 10,000"},{"line_number":62,"context_line":"    # objects in the container. We need to call get_container() with"},{"line_number":63,"context_line":"    # full_listing\u003dTrue"},{"line_number":64,"context_line":"    objects \u003d swiftclient.get_container(container, full_listing\u003dTrue)[1]"}],"source_content_type":"text/x-python","patch_set":11,"id":"05ea3bfb_7adc5360","line":61,"range":{"start_line":61,"start_character":5,"end_line":61,"end_character":27},"updated":"2022-09-12 02:32:25.000000000","message":"I don\u0027t think it\u0027s large environments. Just that .git has too many objects.","commit_id":"d4a29b5788bfcc6699caf66344a48c045f3cbafe"},{"author":{"_account_id":30073,"name":"Brendan Shephard","email":"bshephar@bne-home.net","username":"bshephar"},"change_message_id":"89b4ac0cbfafbbf6979ebc7cd18975ea35c391c3","unresolved":true,"context_lines":[{"line_number":58,"context_line":"                       overwrite_only_newer\u003dFalse):"},{"line_number":59,"context_line":"    \"\"\"Download the contents of a Swift container to a directory\"\"\""},{"line_number":60,"context_line":""},{"line_number":61,"context_line":"    # In large environments. It\u0027s possible that we have greater than 10,000"},{"line_number":62,"context_line":"    # objects in the container. We need to call get_container() with"},{"line_number":63,"context_line":"    # full_listing\u003dTrue"},{"line_number":64,"context_line":"    objects \u003d swiftclient.get_container(container, full_listing\u003dTrue)[1]"}],"source_content_type":"text/x-python","patch_set":11,"id":"49e70455_3f40fd48","line":61,"range":{"start_line":61,"start_character":5,"end_line":61,"end_character":27},"in_reply_to":"05ea3bfb_7adc5360","updated":"2022-09-12 03:47:29.000000000","message":"Probably still true though. The more roles you have, the more objects in this directory. Each role with it\u0027s own directory:\nhttps://c9f81a0d7d88c54137a0-52415b2e407074455a79e58120e157b6.ssl.cf1.rackcdn.com/854722/11/check/tripleo-ci-centos-8-containers-multinode/6c441d0/logs/undercloud/var/lib/mistral/overcloud/Controller/\n\nThen each overcloud deploy would create a new revision of files. I don\u0027t think I will ever hit this issue in my small 2 node homelab for example:\n```\n[stack@tripleo-director overcloud]$ git count-objects\n309 objects, 1728 kilobytes\n```\n\nI can reword it if we don\u0027t think it\u0027s entirely accurate though.","commit_id":"d4a29b5788bfcc6699caf66344a48c045f3cbafe"},{"author":{"_account_id":8833,"name":"Rabi Mishra","email":"ramishra@redhat.com","username":"rabi"},"change_message_id":"dbac1a6d7485be9a8a99943ee897bd201ae6890a","unresolved":true,"context_lines":[{"line_number":58,"context_line":"                       overwrite_only_newer\u003dFalse):"},{"line_number":59,"context_line":"    \"\"\"Download the contents of a Swift container to a directory\"\"\""},{"line_number":60,"context_line":""},{"line_number":61,"context_line":"    # In large environments. It\u0027s possible that we have greater than 10,000"},{"line_number":62,"context_line":"    # objects in the container. We need to call get_container() with"},{"line_number":63,"context_line":"    # full_listing\u003dTrue"},{"line_number":64,"context_line":"    objects \u003d swiftclient.get_container(container, full_listing\u003dTrue)[1]"}],"source_content_type":"text/x-python","patch_set":11,"id":"6960455a_8fc19778","line":61,"range":{"start_line":61,"start_character":5,"end_line":61,"end_character":27},"in_reply_to":"49e70455_3f40fd48","updated":"2022-09-13 06:49:28.000000000","message":"What I meant is with time the .git folder increases over time irrespective of addition of nodes/roles etc. It contains more than just files/directory object blobs (that would increase with environment size).","commit_id":"d4a29b5788bfcc6699caf66344a48c045f3cbafe"}]}