)]}'
{"/PATCHSET_LEVEL":[{"author":{"_account_id":9656,"name":"Ihar Hrachyshka","email":"ihrachys@redhat.com","username":"ihrachys","status":"Red Hat Networking Systems Engineer"},"change_message_id":"308bfc53d16975bb402ce3d072a8e6c7ab6354cc","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":1,"id":"54920cee_8144d723","updated":"2025-01-16 13:35:08.000000000","message":"Do you have logs / bug reports that reviewers could check? I\u0027m not sure bumping timeouts without a clear explanation of why it would help / what changed since before is desirable. There were plenty of ovs/ovn timeouts / events lost recently, wonder if we don\u0027t just try to work real issues around while ignoring the causes.","commit_id":"ccfb39e543c65a50b00ed3583c934acfd62cf480"},{"author":{"_account_id":9656,"name":"Ihar Hrachyshka","email":"ihrachys@redhat.com","username":"ihrachys","status":"Red Hat Networking Systems Engineer"},"change_message_id":"ad9d062557371b5c33547ff1c8bddf9cd2ad0aac","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":1,"id":"15263cc9_a6927403","in_reply_to":"0e6164d7_3c7e311b","updated":"2025-01-17 18:57:30.000000000","message":"I reported that probably covers your issue: https://bugs.launchpad.net/neutron/+bug/2095196\n\nNote that it looks like maybe we have a problem with ovs itself?","commit_id":"ccfb39e543c65a50b00ed3583c934acfd62cf480"},{"author":{"_account_id":14525,"name":"Vasyl Saienko","email":"vsaienko@mirantis.com","username":"vsaienko"},"change_message_id":"28c220e38e24fb5b18d14028110611e93d3bb1c8","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":1,"id":"9b1481d2_c29e49c3","in_reply_to":"15263cc9_a6927403","updated":"2025-01-18 15:58:07.000000000","message":"I\u0027ve opened a bug https://launchpad.net/bugs/2095224. And prepare installation of atop https://review.opendev.org/c/openstack/devstack/+/939578. By briefly checking atop logs we can see that environment may be easily running out of RAM. https://zuul.opendev.org/t/openstack/build/14b24c156d814735a54d725fb490e2ed/logs. We need to enable it on all environments to be ready for a case when issue is reproduced.","commit_id":"ccfb39e543c65a50b00ed3583c934acfd62cf480"},{"author":{"_account_id":14525,"name":"Vasyl Saienko","email":"vsaienko@mirantis.com","username":"vsaienko"},"change_message_id":"8d11aac03f311447b1e9f0221510cd5c5bed39dd","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":1,"id":"ed387cbb_5129c0e2","in_reply_to":"54920cee_8144d723","updated":"2025-01-17 06:23:49.000000000","message":"I\u0027ve seen many functional tests failures due to timeouts here https://review.opendev.org/c/openstack/neutron/+/938657 for example \n[1] https://zuul.opendev.org/t/openstack/build/37637fb42acd4cbaa455291fdd9ff529\nor here [2] https://zuul.opendev.org/t/openstack/build/2f7484d74ba548e5976a3e79b009a1bd\nAlso I saw issues which looks like related to load (like in [2]). It looks like node is overloaded during tests, we need to install atop to have more info on what is happening during tests (mysql or keystone randomly timeouts as well).","commit_id":"ccfb39e543c65a50b00ed3583c934acfd62cf480"},{"author":{"_account_id":14525,"name":"Vasyl Saienko","email":"vsaienko@mirantis.com","username":"vsaienko"},"change_message_id":"d36a0faeba6dfff86a613599ef53030d9af306bb","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":1,"id":"0e6164d7_3c7e311b","in_reply_to":"a2a73e1d_9e5282e3","updated":"2025-01-17 15:26:22.000000000","message":"ack","commit_id":"ccfb39e543c65a50b00ed3583c934acfd62cf480"},{"author":{"_account_id":16688,"name":"Rodolfo Alonso","email":"ralonsoh@redhat.com","username":"rodolfo-alonso-hernandez"},"change_message_id":"bb6582b66df12f978faf389e16dee938f58dbbad","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":1,"id":"a2a73e1d_9e5282e3","in_reply_to":"ed387cbb_5129c0e2","updated":"2025-01-17 14:52:39.000000000","message":"Please, report a LP bug","commit_id":"ccfb39e543c65a50b00ed3583c934acfd62cf480"},{"author":{"_account_id":9656,"name":"Ihar Hrachyshka","email":"ihrachys@redhat.com","username":"ihrachys","status":"Red Hat Networking Systems Engineer"},"change_message_id":"0b696233a5e52b715f014f0ed0529261f8d5b4c9","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":4,"id":"ca075bc6_55f76687","updated":"2025-01-17 19:07:49.000000000","message":"As seen in the bug I linked to, it\u0027s something about infra / low memory / leakage that triggers these timeouts (because ovsdb-server can\u0027t process a simple transaction in 20s+). I don\u0027t think bumping timeout is a good approach, we need to figure out what is happening on the test node.","commit_id":"cfdc2ff557838c517c3ea1a4a595cd056cd20322"},{"author":{"_account_id":14525,"name":"Vasyl Saienko","email":"vsaienko@mirantis.com","username":"vsaienko"},"change_message_id":"990511b9a5f9fb4ba8105d789904ca83a3aa1c41","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":4,"id":"e4ea8029_d0382a32","in_reply_to":"ca075bc6_55f76687","updated":"2025-01-17 20:20:26.000000000","message":"existing tools like dstat will not allow to do this, we should install atop. All try to handle this on next week.","commit_id":"cfdc2ff557838c517c3ea1a4a595cd056cd20322"},{"author":{"_account_id":14525,"name":"Vasyl Saienko","email":"vsaienko@mirantis.com","username":"vsaienko"},"change_message_id":"fc59669a291a172d3afe82f6947b7becee8eb032","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":9,"id":"2abce67c_cebbbf4d","updated":"2025-01-18 13:10:56.000000000","message":"recheck","commit_id":"c1fab1b3cd43c278e4c63c7354a56e990db8dbb4"}],"tools/configure_for_func_testing.sh":[{"author":{"_account_id":1131,"name":"Brian Haley","email":"haleyb.dev@gmail.com","username":"brian-haley"},"change_message_id":"3c022688773bd61a81012159efab807fcd209893","unresolved":true,"context_lines":[{"line_number":286,"context_line":"    source $DEVSTACK_PATH/lib/atop"},{"line_number":287,"context_line":"    configure_atop"},{"line_number":288,"context_line":"    install_atop"},{"line_number":289,"context_line":"    start_atop"},{"line_number":290,"context_line":"    if [[ \"$IS_GATE\" !\u003d \"True\" ]]; then"},{"line_number":291,"context_line":"        # Ensure home directory for the ``stack`` user has executable"},{"line_number":292,"context_line":"        # permissions for all. Only for local (non-gate) installations."}],"source_content_type":"text/x-sh","patch_set":10,"id":"a4e64b2b_c367a40b","line":289,"updated":"2025-03-20 04:20:43.000000000","message":"It just seems like to me this should be under some if() statement like the above code and not just enabled automatically","commit_id":"ebf72cb773a2f217803e1151478e5fafe0f1f845"}]}