)]}'
{"/COMMIT_MSG":[{"author":{"_account_id":4395,"name":"Aaron Rosen","email":"aaronorosen@gmail.com","username":"arosen"},"change_message_id":"727a4f3da9673a1eac6a6fba6a4fca42b3a55c5a","unresolved":false,"context_lines":[{"line_number":6,"context_line":""},{"line_number":7,"context_line":"Add IPset cleanup script"},{"line_number":8,"context_line":""},{"line_number":9,"context_line":"This will aid in removing stale IPsets when we change the prefix"},{"line_number":10,"context_line":"used in creating IPset names."},{"line_number":11,"context_line":""},{"line_number":12,"context_line":"Change-Id: Ia9ff79c34bd4c9124ec8663a8f616ded4f389f62"}],"source_content_type":"text/x-gerrit-commit-message","patch_set":4,"id":"3acd31a7_2ed418bb","line":9,"updated":"2015-05-05 02:24:15.000000000","message":"out of curiosity could we embed this in the l2-agent automatically? Just wondering why we need to make this a script and another thing for someone to run?","commit_id":"99edbb35de923ee6b3cdff63b23ac58f27e4ee0d"},{"author":{"_account_id":1131,"name":"Brian Haley","email":"haleyb.dev@gmail.com","username":"brian-haley"},"change_message_id":"c69bd424b45ad44ba3f8484c031582cf1d41b026","unresolved":false,"context_lines":[{"line_number":6,"context_line":""},{"line_number":7,"context_line":"Add IPset cleanup script"},{"line_number":8,"context_line":""},{"line_number":9,"context_line":"This will aid in removing stale IPsets when we change the prefix"},{"line_number":10,"context_line":"used in creating IPset names."},{"line_number":11,"context_line":""},{"line_number":12,"context_line":"Change-Id: Ia9ff79c34bd4c9124ec8663a8f616ded4f389f62"}],"source_content_type":"text/x-gerrit-commit-message","patch_set":4,"id":"3acd31a7_ec5056d6","line":9,"in_reply_to":"3acd31a7_2ed418bb","updated":"2015-05-05 14:48:41.000000000","message":"This script, like the network namespace cleanup script, is mainly meant to be run on two occasions:\n\n1) upgrade from Juno to Kilo\n2) after a devstack run (although it\u0027s good about cleaning-up), for example, when there are no agents running\n\nBecause we changed the prefix on all ipsets from \"IPv4-uuid\" to \"NETIPv4-uuid\" and this is a hard-coded value in the code, the agent doesn\u0027t know of the old prefix, so doesn\u0027t know to remove the old ones.  That is why I added a --prefix\u003d so that we could give the old \"IPv4\" prefix manually to remove things.\n\nThis is different from the iptables_manager that basically re-writes the entire table on restart, removing any stale entries that might be there.\n\nIt is on my to-do list (my name is in the code :) to have the ipset_manager be better at managing things on it\u0027s own - load known ipsets at startup, modify if necessary, remove stale ones.  Until then this script is the simple way to clean-up.\n\nHopefully that answers your question.  I would wait for Kevin\u0027s and Miguel\u0027s approval before letting this in anyway.","commit_id":"99edbb35de923ee6b3cdff63b23ac58f27e4ee0d"},{"author":{"_account_id":1131,"name":"Brian Haley","email":"haleyb.dev@gmail.com","username":"brian-haley"},"change_message_id":"052bcb18c342cb01aba48b34ab6009212f48f18d","unresolved":false,"context_lines":[{"line_number":6,"context_line":""},{"line_number":7,"context_line":"Add IPset cleanup script"},{"line_number":8,"context_line":""},{"line_number":9,"context_line":"This will aid in removing stale IPsets when we change the prefix"},{"line_number":10,"context_line":"used in creating IPset names."},{"line_number":11,"context_line":""},{"line_number":12,"context_line":"Change-Id: Ia9ff79c34bd4c9124ec8663a8f616ded4f389f62"}],"source_content_type":"text/x-gerrit-commit-message","patch_set":4,"id":"3acd31a7_d70f5a37","line":9,"in_reply_to":"3acd31a7_31f7be5a","updated":"2015-05-05 19:18:33.000000000","message":"Aaron - right, if the agent/manager knows it was added by itself, for example, by the prefix being \"NTRN\" then I\u0027m happy to let it delete it.  We are limited by the name length so the less prefix the better.  Then this script would purely be for cleaning things up if I stopped all agents, instead of forcing a reboot to remove all the sets.","commit_id":"99edbb35de923ee6b3cdff63b23ac58f27e4ee0d"},{"author":{"_account_id":4395,"name":"Aaron Rosen","email":"aaronorosen@gmail.com","username":"arosen"},"change_message_id":"889eb107ae64ca8b34d48bb0b311671df156064c","unresolved":false,"context_lines":[{"line_number":6,"context_line":""},{"line_number":7,"context_line":"Add IPset cleanup script"},{"line_number":8,"context_line":""},{"line_number":9,"context_line":"This will aid in removing stale IPsets when we change the prefix"},{"line_number":10,"context_line":"used in creating IPset names."},{"line_number":11,"context_line":""},{"line_number":12,"context_line":"Change-Id: Ia9ff79c34bd4c9124ec8663a8f616ded4f389f62"}],"source_content_type":"text/x-gerrit-commit-message","patch_set":4,"id":"5afe65bd_2c75a1ca","line":9,"in_reply_to":"3acd31a7_d70f5a37","updated":"2015-06-02 18:13:22.000000000","message":"sounds good. LGTM after rebase.","commit_id":"99edbb35de923ee6b3cdff63b23ac58f27e4ee0d"},{"author":{"_account_id":4395,"name":"Aaron Rosen","email":"aaronorosen@gmail.com","username":"arosen"},"change_message_id":"5097d647ddda490b3aefb0bddb3933005d731fc7","unresolved":false,"context_lines":[{"line_number":6,"context_line":""},{"line_number":7,"context_line":"Add IPset cleanup script"},{"line_number":8,"context_line":""},{"line_number":9,"context_line":"This will aid in removing stale IPsets when we change the prefix"},{"line_number":10,"context_line":"used in creating IPset names."},{"line_number":11,"context_line":""},{"line_number":12,"context_line":"Change-Id: Ia9ff79c34bd4c9124ec8663a8f616ded4f389f62"}],"source_content_type":"text/x-gerrit-commit-message","patch_set":4,"id":"3acd31a7_31f7be5a","line":9,"in_reply_to":"3acd31a7_ec5056d6","updated":"2015-05-05 18:12:55.000000000","message":"Thanks for the reply, It seems like both of these cases could be handled internally in the agent per say. I.e: the new agent cloud delete the old ipset names and do cleanup for stuff that shouldn\u0027t be there. That said, this is probably a little tricky to do right since we don\u0027t want the agent to take over all the network of the box (for example if i create a ns I wouldn\u0027t want the agent to automatically delete it).  \n\nI\u0027m okay with the script approach for now though.","commit_id":"99edbb35de923ee6b3cdff63b23ac58f27e4ee0d"}],"neutron/cmd/ipset_cleanup.py":[{"author":{"_account_id":8788,"name":"Miguel Angel Ajo","email":"mangelajo@redhat.com","username":"mangelajo"},"change_message_id":"943cb5bd6ec14ffd059d62d9d86c21e820ccb093","unresolved":false,"context_lines":[{"line_number":63,"context_line":"    cmd \u003d [\u0027ipset\u0027, \u0027-L\u0027, \u0027-n\u0027]"},{"line_number":64,"context_line":"    ipsets \u003d utils.execute(cmd, run_as_root\u003dTrue)"},{"line_number":65,"context_line":"    for ipset in ipsets.split(\u0027\\n\u0027):"},{"line_number":66,"context_line":"        if conf.allsets or ipset.startswith(conf.prefix):"},{"line_number":67,"context_line":"            destroy_ipset(conf, ipset)"},{"line_number":68,"context_line":""},{"line_number":69,"context_line":"    LOG.info(_LI(\"IPset cleanup completed successfully\"))"}],"source_content_type":"text/x-python","patch_set":1,"id":"3ab451fb_5504f913","line":66,"updated":"2015-04-17 11:19:51.000000000","message":"May be it\u0027s a better option if we inspect iptables and then delete any set that is not referenced, what do you think?","commit_id":"c7a0b3080fcd15b0d0cfe1928eb01d8f9db7d378"},{"author":{"_account_id":1131,"name":"Brian Haley","email":"haleyb.dev@gmail.com","username":"brian-haley"},"change_message_id":"d70aa1eca7a9cf70cd526773ba2db373d1cff241","unresolved":false,"context_lines":[{"line_number":63,"context_line":"    cmd \u003d [\u0027ipset\u0027, \u0027-L\u0027, \u0027-n\u0027]"},{"line_number":64,"context_line":"    ipsets \u003d utils.execute(cmd, run_as_root\u003dTrue)"},{"line_number":65,"context_line":"    for ipset in ipsets.split(\u0027\\n\u0027):"},{"line_number":66,"context_line":"        if conf.allsets or ipset.startswith(conf.prefix):"},{"line_number":67,"context_line":"            destroy_ipset(conf, ipset)"},{"line_number":68,"context_line":""},{"line_number":69,"context_line":"    LOG.info(_LI(\"IPset cleanup completed successfully\"))"}],"source_content_type":"text/x-python","patch_set":1,"id":"3ab451fb_00b05f42","line":66,"in_reply_to":"3ab451fb_5504f913","updated":"2015-04-17 18:43:09.000000000","message":"Yes, for covering the upgrade case we could also check the set isn\u0027t referenced from iptables, but I\u0027d still filter by name by default.","commit_id":"c7a0b3080fcd15b0d0cfe1928eb01d8f9db7d378"},{"author":{"_account_id":7787,"name":"Kevin Benton","email":"kevin@benton.pub","username":"blak111"},"change_message_id":"36918f097de750117925e3fcca0d684d5724bd0a","unresolved":false,"context_lines":[{"line_number":38,"context_line":"                    help\u003d_(\u0027Destroy all IPsets.\u0027)),"},{"line_number":39,"context_line":"        cfg.BoolOpt(\u0027force\u0027,"},{"line_number":40,"context_line":"                    default\u003dFalse,"},{"line_number":41,"context_line":"                    help\u003d_(\u0027Destroy IPsets even if there is an iptables \u0027"},{"line_number":42,"context_line":"                           \u0027reference.\u0027)),"},{"line_number":43,"context_line":"        cfg.StrOpt(\u0027prefix\u0027,"},{"line_number":44,"context_line":"                   default\u003dipset_manager.NET_PREFIX,"}],"source_content_type":"text/x-python","patch_set":3,"id":"1ab14d0d_9bebebb0","line":41,"updated":"2015-04-17 23:26:10.000000000","message":"This doesn\u0027t work. You have to actually go in and delete the iptables rules that reference the chain. That\u0027s why my other patch was so invasive.","commit_id":"6c6ca191e7546f253b5cb1e67d8148e01c23e3e6"},{"author":{"_account_id":1131,"name":"Brian Haley","email":"haleyb.dev@gmail.com","username":"brian-haley"},"change_message_id":"247eb6c7f5aad821515597f544cce9027982fbf5","unresolved":false,"context_lines":[{"line_number":38,"context_line":"                    help\u003d_(\u0027Destroy all IPsets.\u0027)),"},{"line_number":39,"context_line":"        cfg.BoolOpt(\u0027force\u0027,"},{"line_number":40,"context_line":"                    default\u003dFalse,"},{"line_number":41,"context_line":"                    help\u003d_(\u0027Destroy IPsets even if there is an iptables \u0027"},{"line_number":42,"context_line":"                           \u0027reference.\u0027)),"},{"line_number":43,"context_line":"        cfg.StrOpt(\u0027prefix\u0027,"},{"line_number":44,"context_line":"                   default\u003dipset_manager.NET_PREFIX,"}],"source_content_type":"text/x-python","patch_set":3,"id":"dac4157f_da7706d3","line":41,"in_reply_to":"1ab14d0d_9bebebb0","updated":"2015-04-20 14:03:18.000000000","message":"Ok, well I can make the script remove that reference, I was only testing with a matching chain name so hadn\u0027t tripped over this.","commit_id":"6c6ca191e7546f253b5cb1e67d8148e01c23e3e6"},{"author":{"_account_id":1131,"name":"Brian Haley","email":"haleyb.dev@gmail.com","username":"brian-haley"},"change_message_id":"68dad8be55ec29139f6cd62ea077e618d3ee1292","unresolved":false,"context_lines":[{"line_number":38,"context_line":"                    help\u003d_(\u0027Destroy all IPsets.\u0027)),"},{"line_number":39,"context_line":"        cfg.BoolOpt(\u0027force\u0027,"},{"line_number":40,"context_line":"                    default\u003dFalse,"},{"line_number":41,"context_line":"                    help\u003d_(\u0027Destroy IPsets even if there is an iptables \u0027"},{"line_number":42,"context_line":"                           \u0027reference.\u0027)),"},{"line_number":43,"context_line":"        cfg.StrOpt(\u0027prefix\u0027,"},{"line_number":44,"context_line":"                   default\u003dipset_manager.NET_PREFIX,"}],"source_content_type":"text/x-python","patch_set":3,"id":"3acd31a7_cd949144","line":41,"in_reply_to":"dac4157f_da7706d3","updated":"2015-05-04 21:30:18.000000000","message":"Sending out a new version that will iptables -D the referencing rule if it exists.","commit_id":"6c6ca191e7546f253b5cb1e67d8148e01c23e3e6"},{"author":{"_account_id":7787,"name":"Kevin Benton","email":"kevin@benton.pub","username":"blak111"},"change_message_id":"36918f097de750117925e3fcca0d684d5724bd0a","unresolved":false,"context_lines":[{"line_number":66,"context_line":""},{"line_number":67,"context_line":"    # Need to make sure there is not an iptables reference"},{"line_number":68,"context_line":"    cmd \u003d [\u0027iptables-save\u0027]"},{"line_number":69,"context_line":"    iptables_save \u003d utils.execute(cmd, run_as_root\u003dTrue)"},{"line_number":70,"context_line":""},{"line_number":71,"context_line":"    cmd \u003d [\u0027ipset\u0027, \u0027-L\u0027, \u0027-n\u0027]"},{"line_number":72,"context_line":"    ipsets \u003d utils.execute(cmd, run_as_root\u003dTrue)"}],"source_content_type":"text/x-python","patch_set":3,"id":"1ab14d0d_9b39ab44","line":69,"updated":"2015-04-17 23:26:10.000000000","message":"Is this safe if the iptables rules are in a namespace but the ipset isn\u0027t, Or are the sets forced to be scoped the same namespace?\n\nA safer approach would probably be to just try to destroy the namespace and catch the error that results when it has references. That way it doesn\u0027t depend on checking iptables output.","commit_id":"6c6ca191e7546f253b5cb1e67d8148e01c23e3e6"},{"author":{"_account_id":1131,"name":"Brian Haley","email":"haleyb.dev@gmail.com","username":"brian-haley"},"change_message_id":"247eb6c7f5aad821515597f544cce9027982fbf5","unresolved":false,"context_lines":[{"line_number":66,"context_line":""},{"line_number":67,"context_line":"    # Need to make sure there is not an iptables reference"},{"line_number":68,"context_line":"    cmd \u003d [\u0027iptables-save\u0027]"},{"line_number":69,"context_line":"    iptables_save \u003d utils.execute(cmd, run_as_root\u003dTrue)"},{"line_number":70,"context_line":""},{"line_number":71,"context_line":"    cmd \u003d [\u0027ipset\u0027, \u0027-L\u0027, \u0027-n\u0027]"},{"line_number":72,"context_line":"    ipsets \u003d utils.execute(cmd, run_as_root\u003dTrue)"}],"source_content_type":"text/x-python","patch_set":3,"id":"dac4157f_15dd5142","line":69,"in_reply_to":"1ab14d0d_9b39ab44","updated":"2015-04-20 14:03:18.000000000","message":"Since this is on the compute node where the SG lives, there is no namespace involvement, everything lives in the \"root\" namespace.  This is why Miguel had issues with name collision in the set name.  It is curious that the ipset_manager takes a namespace on init since I don\u0027t think it ever gets one except in a test.\n\nThe target goal of this script is to catch those sets that were orphaned from an agent restart, or after a devstack run, so I don\u0027t want to go much further than that on what we remove.","commit_id":"6c6ca191e7546f253b5cb1e67d8148e01c23e3e6"},{"author":{"_account_id":1131,"name":"Brian Haley","email":"haleyb.dev@gmail.com","username":"brian-haley"},"change_message_id":"c42ada2fe1e5fdef27791f99108c97147229ef5a","unresolved":false,"context_lines":[{"line_number":66,"context_line":""},{"line_number":67,"context_line":"    # Need to make sure there is not an iptables reference"},{"line_number":68,"context_line":"    cmd \u003d [\u0027iptables-save\u0027]"},{"line_number":69,"context_line":"    iptables_save \u003d utils.execute(cmd, run_as_root\u003dTrue)"},{"line_number":70,"context_line":""},{"line_number":71,"context_line":"    cmd \u003d [\u0027ipset\u0027, \u0027-L\u0027, \u0027-n\u0027]"},{"line_number":72,"context_line":"    ipsets \u003d utils.execute(cmd, run_as_root\u003dTrue)"}],"source_content_type":"text/x-python","patch_set":3,"id":"3acd31a7_3b3c6dc7","line":69,"in_reply_to":"3acd31a7_16318d05","updated":"2015-05-04 13:42:43.000000000","message":"Right, if I \"Tried to delete the ipset and caught the error\", that\u0027s what it\u0027s doing now, so wouldn\u0027t need another change :)\n\nOtherwise it\u0027s a s/-A/-D/ for the iptables rule from iptables-save and try and delete the ipset.","commit_id":"6c6ca191e7546f253b5cb1e67d8148e01c23e3e6"},{"author":{"_account_id":1131,"name":"Brian Haley","email":"haleyb.dev@gmail.com","username":"brian-haley"},"change_message_id":"68dad8be55ec29139f6cd62ea077e618d3ee1292","unresolved":false,"context_lines":[{"line_number":66,"context_line":""},{"line_number":67,"context_line":"    # Need to make sure there is not an iptables reference"},{"line_number":68,"context_line":"    cmd \u003d [\u0027iptables-save\u0027]"},{"line_number":69,"context_line":"    iptables_save \u003d utils.execute(cmd, run_as_root\u003dTrue)"},{"line_number":70,"context_line":""},{"line_number":71,"context_line":"    cmd \u003d [\u0027ipset\u0027, \u0027-L\u0027, \u0027-n\u0027]"},{"line_number":72,"context_line":"    ipsets \u003d utils.execute(cmd, run_as_root\u003dTrue)"}],"source_content_type":"text/x-python","patch_set":3,"id":"3acd31a7_2d330564","line":69,"in_reply_to":"3acd31a7_3b3c6dc7","updated":"2015-05-04 21:30:18.000000000","message":"I\u0027ll also send a follow-up that removes the namespace argument from ipset_manager as it\u0027s not required","commit_id":"6c6ca191e7546f253b5cb1e67d8148e01c23e3e6"},{"author":{"_account_id":8788,"name":"Miguel Angel Ajo","email":"mangelajo@redhat.com","username":"mangelajo"},"change_message_id":"38a3c1b201178a0131efa50d84ae29b228269aa7","unresolved":false,"context_lines":[{"line_number":66,"context_line":""},{"line_number":67,"context_line":"    # Need to make sure there is not an iptables reference"},{"line_number":68,"context_line":"    cmd \u003d [\u0027iptables-save\u0027]"},{"line_number":69,"context_line":"    iptables_save \u003d utils.execute(cmd, run_as_root\u003dTrue)"},{"line_number":70,"context_line":""},{"line_number":71,"context_line":"    cmd \u003d [\u0027ipset\u0027, \u0027-L\u0027, \u0027-n\u0027]"},{"line_number":72,"context_line":"    ipsets \u003d utils.execute(cmd, run_as_root\u003dTrue)"}],"source_content_type":"text/x-python","patch_set":3,"id":"3acd31a7_16318d05","line":69,"in_reply_to":"3acd31a7_67025571","updated":"2015-05-04 09:59:54.000000000","message":"I you changed ipset for namespace in your phrase :) in that case yes, with ipset that makes sense to me.\n\nWe clean the unused ones, and we fail/ignore the referenced ones.","commit_id":"6c6ca191e7546f253b5cb1e67d8148e01c23e3e6"},{"author":{"_account_id":7787,"name":"Kevin Benton","email":"kevin@benton.pub","username":"blak111"},"change_message_id":"39eeda992a5068293277ed7aef6b43fc0e1ae1e4","unresolved":false,"context_lines":[{"line_number":66,"context_line":""},{"line_number":67,"context_line":"    # Need to make sure there is not an iptables reference"},{"line_number":68,"context_line":"    cmd \u003d [\u0027iptables-save\u0027]"},{"line_number":69,"context_line":"    iptables_save \u003d utils.execute(cmd, run_as_root\u003dTrue)"},{"line_number":70,"context_line":""},{"line_number":71,"context_line":"    cmd \u003d [\u0027ipset\u0027, \u0027-L\u0027, \u0027-n\u0027]"},{"line_number":72,"context_line":"    ipsets \u003d utils.execute(cmd, run_as_root\u003dTrue)"}],"source_content_type":"text/x-python","patch_set":3,"id":"3acd31a7_67025571","line":69,"in_reply_to":"dac4157f_07b182e1","updated":"2015-05-04 03:35:26.000000000","message":"Makes sense. But from a simplicity perspective, why not just try to delete the namespace and catch the error?","commit_id":"6c6ca191e7546f253b5cb1e67d8148e01c23e3e6"},{"author":{"_account_id":8788,"name":"Miguel Angel Ajo","email":"mangelajo@redhat.com","username":"mangelajo"},"change_message_id":"c0054145d28bfa2a5842f09767482e0dc25338a4","unresolved":false,"context_lines":[{"line_number":66,"context_line":""},{"line_number":67,"context_line":"    # Need to make sure there is not an iptables reference"},{"line_number":68,"context_line":"    cmd \u003d [\u0027iptables-save\u0027]"},{"line_number":69,"context_line":"    iptables_save \u003d utils.execute(cmd, run_as_root\u003dTrue)"},{"line_number":70,"context_line":""},{"line_number":71,"context_line":"    cmd \u003d [\u0027ipset\u0027, \u0027-L\u0027, \u0027-n\u0027]"},{"line_number":72,"context_line":"    ipsets \u003d utils.execute(cmd, run_as_root\u003dTrue)"}],"source_content_type":"text/x-python","patch_set":3,"id":"dac4157f_07b182e1","line":69,"in_reply_to":"dac4157f_15dd5142","updated":"2015-04-21 08:29:25.000000000","message":"I\u0027m afraid, the namespace is only a YAGNI, well, I put it in place to make testing more robust, but I see now that sets are not scoped (at least now) by namespaces:\n\n[root@compute01 ~]# ip netns exec test ipset create test-set hash:net\n\n[root@compute01 ~]# ipset save\ncreate test-set hash:net family inet hashsize 1024 maxelem 65536","commit_id":"6c6ca191e7546f253b5cb1e67d8148e01c23e3e6"}]}