)]}'
{"/PATCHSET_LEVEL":[{"author":{"_account_id":30073,"name":"Brendan Shephard","email":"bshephar@bne-home.net","username":"bshephar"},"change_message_id":"96ba7362d100b86fadd0ab5734f56c2de7d566b3","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":2,"id":"51c9e549_f1b95878","updated":"2022-08-02 07:06:15.000000000","message":"I can see that these firewall rules have been added:\n\nhttps://zuul.opendev.org/t/openstack/build/791ad75a3d9642028fbc5ebe79b9d95a/log/logs/subnode-1/etc/sysconfig/iptables#51-54\n\n```\n-A INPUT -s 192.168.24.0/24 -p tcp -m tcp --dport 6644 -m conntrack --ctstate NEW -m comment --comment \"121 OVN DB server and cluster ports ipv4\" -j ACCEPT\n52\t-A INPUT -s 192.168.24.0/24 -p tcp -m tcp --dport 6643 -m conntrack --ctstate NEW -m comment --comment \"121 OVN DB server and cluster ports ipv4\" -j ACCEPT\n53\t-A INPUT -s 192.168.24.0/24 -p tcp -m tcp --dport 6642 -m conntrack --ctstate NEW -m comment --comment \"121 OVN DB server and cluster ports ipv4\" -j ACCEPT\n54\t-A INPUT -s 192.168.24.0/24 -p tcp -m tcp --dport 6641 -m conntrack --ctstate NEW -m comment --comment \"121 OVN DB server and cluster ports ipv4\" -j ACCEPT\n```\n\nBut I do wonder how well we have this covered without a multi-Controller scenario to test this?\n\nFor example, I can see some connection errors being reported by OVN:\nhttps://zuul.opendev.org/t/openstack/build/791ad75a3d9642028fbc5ebe79b9d95a/log/logs/subnode-1/var/log/extra/podman/containers/ovn_cluster_north_db_server/log/ovn/ovsdb-server-nb.log\n```\n2022-07-29T03:30:09.542Z|00131|socket_util|ERR|Dropped 192 log messages in last 60 seconds (most recently, 1 seconds ago) due to excessive rate\n132\t2022-07-29T03:30:09.542Z|00132|socket_util|ERR|6641: bind: Address already in use\n133\t2022-07-29T03:30:09.542Z|00133|ovsdb_jsonrpc_server|ERR|Dropped 192 log messages in last 60 seconds (most recently, 1 seconds ago) due to excessive rate\n134\t2022-07-29T03:30:09.542Z|00134|ovsdb_jsonrpc_server|ERR|ptcp:6641: listen failed: Address already in use\n135\t2022-07-29T03:30:22.779Z|00135|jsonrpc|WARN|tcp:192.168.24.3:48718: receive error: Connection reset by peer\n136\t2022-07-29T03:30:22.779Z|00136|reconnect|WARN|tcp:192.168.24.3:48718: connection dropped (Connection reset by peer)\n137\t2022-07-29T03:31:09.567Z|00137|socket_util|ERR|Dropped 197 log messages in last 60 seconds (most recently, 1 seconds ago) due to excessive rate\n138\t2022-07-29T03:31:09.567Z|00138|socket_util|ERR|6641: bind: Address already in use\n139\t2022-07-29T03:31:09.567Z|00139|ovsdb_jsonrpc_server|ERR|Dropped 197 log messages in last 60 seconds (most recently, 1 seconds ago) due to excessive rate\n140\t2022-07-29T03:31:09.567Z|00140|ovsdb_jsonrpc_server|ERR|ptcp:6641: listen failed: Address already in use\n```\n\nI can\u0027t confidently vote on this unfortunately, can we double check these \"Address already in use\" errors?\n\n","commit_id":"e2c0b839131c576d7dbba8155c46b237a828c2ef"},{"author":{"_account_id":30073,"name":"Brendan Shephard","email":"bshephar@bne-home.net","username":"bshephar"},"change_message_id":"dad8deefca11a98d9fb130b8d0f1ec51404bb358","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":2,"id":"34e20aa4_50a36f29","updated":"2022-08-02 13:28:38.000000000","message":"Looks good to me with clarification of the errors seen from the NB and SB containers. Code here looks fine and I can confirm the iptables rules were indeed added as described.","commit_id":"e2c0b839131c576d7dbba8155c46b237a828c2ef"},{"author":{"_account_id":30073,"name":"Brendan Shephard","email":"bshephar@bne-home.net","username":"bshephar"},"change_message_id":"0ff9de86f8b4d6dee21fbd8c4a018f59de514f2a","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":2,"id":"226fd121_0e85b820","updated":"2022-08-02 07:11:10.000000000","message":"My -1 is just here while we validate the \"Address already in use\" errors for the NB and SB processes:\nhttps://zuul.opendev.org/t/openstack/build/791ad75a3d9642028fbc5ebe79b9d95a/log/logs/subnode-1/var/log/containers/openvswitch/ovsdb-server-nb.log\nhttps://zuul.opendev.org/t/openstack/build/791ad75a3d9642028fbc5ebe79b9d95a/log/logs/subnode-1/var/log/containers/openvswitch/ovsdb-server-sb.log\n\nHappy to change my vote with clarification that the issue is unrelated to the change. ","commit_id":"e2c0b839131c576d7dbba8155c46b237a828c2ef"},{"author":{"_account_id":5756,"name":"Terry Wilson","email":"twilson@redhat.com","username":"otherwiseguy"},"change_message_id":"77857e7c562daf13947fcb162f4df55c45c100b5","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":2,"id":"2e9a278e_955fae0a","updated":"2022-07-29 00:18:48.000000000","message":"Rebased onto the default to raft clustering patch so that tests would run using clustering.","commit_id":"e2c0b839131c576d7dbba8155c46b237a828c2ef"},{"author":{"_account_id":5756,"name":"Terry Wilson","email":"twilson@redhat.com","username":"otherwiseguy"},"change_message_id":"eff1032c6607c8793546ae864b347cd614b884c2","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":2,"id":"87fd5cc8_ba088d58","in_reply_to":"226fd121_0e85b820","updated":"2022-08-02 12:38:42.000000000","message":"Thanks! the tripleo-ansible patch that goes with this is what fixes those address already in use errors.","commit_id":"e2c0b839131c576d7dbba8155c46b237a828c2ef"},{"author":{"_account_id":30073,"name":"Brendan Shephard","email":"bshephar@bne-home.net","username":"bshephar"},"change_message_id":"dad8deefca11a98d9fb130b8d0f1ec51404bb358","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":2,"id":"36aead31_2a1570d4","in_reply_to":"2c617728_e17d8678","updated":"2022-08-02 13:28:38.000000000","message":"Ahh ok. Makes sense. Thanks for the clarification.","commit_id":"e2c0b839131c576d7dbba8155c46b237a828c2ef"},{"author":{"_account_id":5756,"name":"Terry Wilson","email":"twilson@redhat.com","username":"otherwiseguy"},"change_message_id":"b1b2b0b32275dac9314932edc4a4d0776ce67841","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":2,"id":"2c617728_e17d8678","in_reply_to":"87fd5cc8_ba088d58","updated":"2022-08-02 13:05:27.000000000","message":"https://review.opendev.org/c/openstack/tripleo-ansible/+/851452 for reference.","commit_id":"e2c0b839131c576d7dbba8155c46b237a828c2ef"}],"deployment/ovn/ovn-dbs-cluster-ansible.yaml":[{"author":{"_account_id":5756,"name":"Terry Wilson","email":"twilson@redhat.com","username":"otherwiseguy"},"change_message_id":"96bccbac7c8b15dc29a8249f81a1cf42228d85ca","unresolved":false,"context_lines":[{"line_number":311,"context_line":"            - name: Set connection # FIXME workaround until RHBZ #1952038 is fixed"},{"line_number":312,"context_line":"              become: true"},{"line_number":313,"context_line":"              shell: |"},{"line_number":314,"context_line":"                podman exec ovn_cluster_north_db_server bash -c \"ovn-nbctl -p /etc/pki/tls/private/ovn_dbs.key -c /etc/pki/tls/certs/ovn_dbs.crt -C /etc/ipa/ca.crt set-connection pssl:{{ tripleo_ovn_cluster_nb_db_port }}\""},{"line_number":315,"context_line":"                podman exec ovn_cluster_south_db_server bash -c \"ovn-sbctl -p /etc/pki/tls/private/ovn_dbs.key -c /etc/pki/tls/certs/ovn_dbs.crt -C /etc/ipa/ca.crt set-connection pssl:{{ tripleo_ovn_cluster_sb_db_port }}\""},{"line_number":316,"context_line":"              when:"},{"line_number":317,"context_line":"                - enable_internal_tls | bool"}],"source_content_type":"text/x-yaml","patch_set":1,"id":"e019b389_6526102b","side":"PARENT","line":314,"range":{"start_line":314,"start_character":75,"end_line":314,"end_character":163},"updated":"2022-07-28 22:45:29.000000000","message":"This should not be necessary since ovn-nbctl is connection to the db over the unix socket.","commit_id":"7401aae83c9ca691329c0b81cbcdb714feeb3949"},{"author":{"_account_id":5756,"name":"Terry Wilson","email":"twilson@redhat.com","username":"otherwiseguy"},"change_message_id":"96bccbac7c8b15dc29a8249f81a1cf42228d85ca","unresolved":false,"context_lines":[{"line_number":325,"context_line":"            - name: Set connection"},{"line_number":326,"context_line":"              become: true"},{"line_number":327,"context_line":"              shell: |"},{"line_number":328,"context_line":"                podman exec ovn_cluster_north_db_server bash -c \"ovn-nbctl --no-leader-only --inactivity-probe\u003d{{ tripleo_ovn_cluster_probe_interval }} set-connection p{{ tripleo_ovn_cluster_dbs_protocol }}:{{ tripleo_ovn_cluster_nb_db_port }}\""},{"line_number":329,"context_line":"                podman exec ovn_cluster_south_db_server bash -c \"ovn-sbctl --no-leader-only --inactivity-probe\u003d{{ tripleo_ovn_cluster_probe_interval }} set-connection p{{ tripleo_ovn_cluster_dbs_protocol }}:{{ tripleo_ovn_cluster_sb_db_port }}\""},{"line_number":330,"context_line":"              when:"},{"line_number":331,"context_line":"                - is_ovn_dbs_bootstrap_node | bool"}],"source_content_type":"text/x-yaml","patch_set":1,"id":"a5da0beb_22b53d8b","line":328,"range":{"start_line":328,"start_character":75,"end_line":328,"end_character":91},"updated":"2022-07-28 22:45:29.000000000","message":"This is to ensure that the command is run even if there is a leadership change from the bootstrap server.","commit_id":"b996aae7b43ce7784becf0a8675d368c4b58b23b"}]}