)]}'
{"/COMMIT_MSG":[{"author":{"_account_id":748,"name":"Armando Migliaccio","email":"armamig@gmail.com","username":"armando-migliaccio"},"change_message_id":"70450b1e5dd7f2f6033965db0c506cb9464db703","unresolved":false,"context_lines":[{"line_number":6,"context_line":""},{"line_number":7,"context_line":"Make sync_routers chunking configurable"},{"line_number":8,"context_line":""},{"line_number":9,"context_line":"This patch adds two extra configuration parameters to the l3_agent.ini"},{"line_number":10,"context_line":"DEFAULT section, it is intended to allow an operator to configue"},{"line_number":11,"context_line":"SYNC_ROUTERS_MAX_CHUNK_SIZE and SYNC_ROUTERS_MIN_CHUNK_SIZE.i"},{"line_number":12,"context_line":""},{"line_number":13,"context_line":"Normally when the sync_routers call is made and a MessagingTimeout"},{"line_number":14,"context_line":"exception occures the agent will start decreasing the chunk size.i"}],"source_content_type":"text/x-gerrit-commit-message","patch_set":2,"id":"ff0f0b1f_4a99511d","line":11,"range":{"start_line":9,"start_character":0,"end_line":11,"end_character":59},"updated":"2017-05-27 00:22:53.000000000","message":"As a team we have pondered a lot about this area of the code and went down the path of keeping these hidden to the operator. Typically there are other tunables that can be tweaked before going down the path of exposing yet another set of obscure config options. Perhaps raising the RPC timeout or lowering the router/agent ratio is a more effective way of dealing with the problem you\u0027re seeing.\n\nReducing the amount of data you fetch to the point that you\u0027re trickling data from the server back to the agent may lead to other side effects like longer startup times and potentially longer dataplane downtime.\n\nExposing these config option is easy, and we kept them hard coded for a reason.","commit_id":"d5a77f0edcee2ae7c74b23de7750b2120ac54bce"},{"author":{"_account_id":12021,"name":"Thomas Morin","email":"tmmorin.orange@gmail.com","username":"tmorin"},"change_message_id":"94a78b4a264f442ce1358b5456393a49da0368a6","unresolved":false,"context_lines":[{"line_number":6,"context_line":""},{"line_number":7,"context_line":"Make sync_routers chunking configurable"},{"line_number":8,"context_line":""},{"line_number":9,"context_line":"This patch adds two extra configuration parameters to the l3_agent.ini"},{"line_number":10,"context_line":"DEFAULT section, it is intended to allow an operator to configue"},{"line_number":11,"context_line":"SYNC_ROUTERS_MAX_CHUNK_SIZE and SYNC_ROUTERS_MIN_CHUNK_SIZE.i"},{"line_number":12,"context_line":""},{"line_number":13,"context_line":"Normally when the sync_routers call is made and a MessagingTimeout"},{"line_number":14,"context_line":"exception occures the agent will start decreasing the chunk size.i"}],"source_content_type":"text/x-gerrit-commit-message","patch_set":2,"id":"df140735_f62030fa","line":11,"range":{"start_line":9,"start_character":0,"end_line":11,"end_character":59},"in_reply_to":"df140735_c717bded","updated":"2017-05-29 10:58:58.000000000","message":"I understand the rationale brought forward by Armando, but I would observe that increasing the number of agents seems to me as a wrong solution in such a case (solving neutron server perf issue by adding network nodes before they are a bottleneck by themselves?).  So, if increasing the rcp timeout has been attempted and is not enough, then reducing chunk size looks like a logical step, isn\u0027t it ?","commit_id":"d5a77f0edcee2ae7c74b23de7750b2120ac54bce"},{"author":{"_account_id":26086,"name":"Cristian Calin","email":"cristian.calin@orange.com","username":"cristicalin"},"change_message_id":"4cf5e5a257574ffcc37353150b0801c305fd0a88","unresolved":false,"context_lines":[{"line_number":6,"context_line":""},{"line_number":7,"context_line":"Make sync_routers chunking configurable"},{"line_number":8,"context_line":""},{"line_number":9,"context_line":"This patch adds two extra configuration parameters to the l3_agent.ini"},{"line_number":10,"context_line":"DEFAULT section, it is intended to allow an operator to configue"},{"line_number":11,"context_line":"SYNC_ROUTERS_MAX_CHUNK_SIZE and SYNC_ROUTERS_MIN_CHUNK_SIZE.i"},{"line_number":12,"context_line":""},{"line_number":13,"context_line":"Normally when the sync_routers call is made and a MessagingTimeout"},{"line_number":14,"context_line":"exception occures the agent will start decreasing the chunk size.i"}],"source_content_type":"text/x-gerrit-commit-message","patch_set":2,"id":"df140735_3676a8da","line":11,"range":{"start_line":9,"start_character":0,"end_line":11,"end_character":59},"in_reply_to":"df140735_f62030fa","updated":"2017-05-29 11:09:05.000000000","message":"Just to be clear, increasing the RPC timeout is no a matter of going from 60 seconds to 120 seconds or similar, in our case even with rpc_timeout\u003d600 we did not observe the neutron server managing to respond. The only solution in such a case is when the agent eventually scales down the chunk size.\nAdding multiple agents (in our case) is due to the number of routers needing to be managed and overall memory consumption by the helper processes of each router such as neutron-keepalived-state-change. Adding multiple agents is something an operator ultimately has to do to scale up and this ends up flooding the server. As the sync_router call is unusually heavy, regardless of how many instances of the neutron-server you spin up you will still wait for a single server to process the sync call.\nA proper solution is to rework the queries behind the sync_routers call and lighten the load but that requires a lot more performance tuning work.","commit_id":"d5a77f0edcee2ae7c74b23de7750b2120ac54bce"},{"author":{"_account_id":26086,"name":"Cristian Calin","email":"cristian.calin@orange.com","username":"cristicalin"},"change_message_id":"27c3e2e4493327af67a7557d64ce57da0c6e2540","unresolved":false,"context_lines":[{"line_number":6,"context_line":""},{"line_number":7,"context_line":"Make sync_routers chunking configurable"},{"line_number":8,"context_line":""},{"line_number":9,"context_line":"This patch adds two extra configuration parameters to the l3_agent.ini"},{"line_number":10,"context_line":"DEFAULT section, it is intended to allow an operator to configue"},{"line_number":11,"context_line":"SYNC_ROUTERS_MAX_CHUNK_SIZE and SYNC_ROUTERS_MIN_CHUNK_SIZE.i"},{"line_number":12,"context_line":""},{"line_number":13,"context_line":"Normally when the sync_routers call is made and a MessagingTimeout"},{"line_number":14,"context_line":"exception occures the agent will start decreasing the chunk size.i"}],"source_content_type":"text/x-gerrit-commit-message","patch_set":2,"id":"df140735_c717bded","line":11,"range":{"start_line":9,"start_character":0,"end_line":11,"end_character":59},"in_reply_to":"ff0f0b1f_4a99511d","updated":"2017-05-29 09:44:19.000000000","message":"Thanks for the feedback.\nWe tried playing around with the way the agent recommends, like raising the RPC timeout which has a different side-effect of delaying the scaling down (please see the related bug (https://bugs.launchpad.net/neutron/+bug/1692971) for details of the test.\nReducing the amount of data fetched does indeed increase the complete startup time due to multiple back and forth between the server and agent(s) and that is something the operator should be able to consciously decide for themselves. The issue we trigger here is more related to the fact we run this in an HA deployment with lots of L3 agents (at least 6 in our case and planing to go to 10 because of an increase in the number of routers). Because of this, in a total outage scenario when the agents have to start up they flood the neutron-server and it never manages to respond to the query until the chunk size is decreased leading do a much longer startup time.\nA better solution is definitely to improve the time of sync_routers call but this is a nob to allow advanced operators to relieve some pressure in large environments.\n\nAs oslo.config provides the capability to mark certain config options as advanced (https://docs.openstack.org/developer/oslo.config/cfg.html#advanced-option) we can leverage that to explicitly tell operators not to change the default (which we keep from the hard coded values) unless they know exactly what they are doing and fully understand the side effects.","commit_id":"d5a77f0edcee2ae7c74b23de7750b2120ac54bce"},{"author":{"_account_id":1653,"name":"garyk","email":"gkotton@vmware.com","username":"garyk"},"change_message_id":"a9ddf8a1bd9faa4a07be8cb649c9d0c5978382ec","unresolved":false,"context_lines":[{"line_number":19,"context_line":""},{"line_number":20,"context_line":"On large environments this patch gives an operator extra tunables"},{"line_number":21,"context_line":"to set while keeping the old defaults in place."},{"line_number":22,"context_line":""},{"line_number":23,"context_line":"Closes-Bug: 1692971"},{"line_number":24,"context_line":""},{"line_number":25,"context_line":"Change-Id: I70e5e1db709009a1c004a2ebe44b1ba595eba7b3"}],"source_content_type":"text/x-gerrit-commit-message","patch_set":2,"id":"df140735_3c0d673e","line":22,"updated":"2017-05-29 12:44:42.000000000","message":"I think that this requires a DocImpact\nIn addition to this this at least requires a release note and unit tests","commit_id":"d5a77f0edcee2ae7c74b23de7750b2120ac54bce"},{"author":{"_account_id":748,"name":"Armando Migliaccio","email":"armamig@gmail.com","username":"armando-migliaccio"},"change_message_id":"70450b1e5dd7f2f6033965db0c506cb9464db703","unresolved":false,"context_lines":[{"line_number":20,"context_line":"On large environments this patch gives an operator extra tunables"},{"line_number":21,"context_line":"to set while keeping the old defaults in place."},{"line_number":22,"context_line":""},{"line_number":23,"context_line":"Closes-Bug: 1692971"},{"line_number":24,"context_line":""},{"line_number":25,"context_line":"Change-Id: I70e5e1db709009a1c004a2ebe44b1ba595eba7b3"}],"source_content_type":"text/x-gerrit-commit-message","patch_set":2,"id":"ff0f0b1f_ca8441b4","line":23,"updated":"2017-05-27 00:22:53.000000000","message":"I am personally opposed to this patch, but if other team members feel positively this needs:\n\nDocImpact: add two more configs to the L3 agent\n\nAnd a release note.","commit_id":"d5a77f0edcee2ae7c74b23de7750b2120ac54bce"},{"author":{"_account_id":26086,"name":"Cristian Calin","email":"cristian.calin@orange.com","username":"cristicalin"},"change_message_id":"27c3e2e4493327af67a7557d64ce57da0c6e2540","unresolved":false,"context_lines":[{"line_number":20,"context_line":"On large environments this patch gives an operator extra tunables"},{"line_number":21,"context_line":"to set while keeping the old defaults in place."},{"line_number":22,"context_line":""},{"line_number":23,"context_line":"Closes-Bug: 1692971"},{"line_number":24,"context_line":""},{"line_number":25,"context_line":"Change-Id: I70e5e1db709009a1c004a2ebe44b1ba595eba7b3"}],"source_content_type":"text/x-gerrit-commit-message","patch_set":2,"id":"df140735_27943944","line":23,"in_reply_to":"ff0f0b1f_ca8441b4","updated":"2017-05-29 09:44:19.000000000","message":"I understand your point especially in the context of reducing the number of configurable options. This however does have impact in large scale deployments and operators like us will have to carry private patches which further complicates our life and ability to engage vendors.\nI will add relevant release notes and documentation updates with guidance on when and how to tune these parameters so operators can understand what they are getting their selves into when modifying them.","commit_id":"d5a77f0edcee2ae7c74b23de7750b2120ac54bce"},{"author":{"_account_id":748,"name":"Armando Migliaccio","email":"armamig@gmail.com","username":"armando-migliaccio"},"change_message_id":"001ecea090bd68b77819bfdfe6d79138a9ac94c9","unresolved":false,"context_lines":[{"line_number":22,"context_line":""},{"line_number":23,"context_line":"Closes-Bug: 1692971"},{"line_number":24,"context_line":""},{"line_number":25,"context_line":"Change-Id: I70e5e1db709009a1c004a2ebe44b1ba595eba7b3"}],"source_content_type":"text/x-gerrit-commit-message","patch_set":2,"id":"df140735_24c76448","line":25,"updated":"2017-05-31 00:29:58.000000000","message":"The point of keeping these options hidden is that one can\u0027t determine a good set without trial and error. Besides, the state of the system changes and thus that may invalidate whatever change was made at a given point in time. The only fix to this problem is to accelerate the convergence of the algorithm, because right now there\u0027s no way to either \u0027abort\u0027 a request nor making the agent sit patiently and wait for the response.\n\nUltimately exposing these tunable is a stop-gap that might as well backfire and what I don\u0027t like about that is that it feels like handing the operators the rope to hang themselves.","commit_id":"d5a77f0edcee2ae7c74b23de7750b2120ac54bce"},{"author":{"_account_id":12021,"name":"Thomas Morin","email":"tmmorin.orange@gmail.com","username":"tmorin"},"change_message_id":"dddea222430e63dcbd43ffa50a6530fb121f1cb8","unresolved":false,"context_lines":[{"line_number":11,"context_line":"SYNC_ROUTERS_MAX_CHUNK_SIZE and SYNC_ROUTERS_MIN_CHUNK_SIZE."},{"line_number":12,"context_line":""},{"line_number":13,"context_line":"When the sync_routers call is made and a MessagingTimeout exception"},{"line_number":14,"context_line":"occures the agent will start decreasing the chunk size. The time it"},{"line_number":15,"context_line":"takes to trigger MessagingTimeout depends on the ``rpc_response_timeout``"},{"line_number":16,"context_line":"configuration parameter."},{"line_number":17,"context_line":""}],"source_content_type":"text/x-gerrit-commit-message","patch_set":5,"id":"df140735_85e63344","line":14,"range":{"start_line":14,"start_character":5,"end_line":14,"end_character":6},"updated":"2017-05-30 19:16:01.000000000","message":"(spurious \u0027e\u0027)","commit_id":"8e8918c8ac1d5ff34c849bc0d1cc896ff24bade2"},{"author":{"_account_id":12021,"name":"Thomas Morin","email":"tmmorin.orange@gmail.com","username":"tmorin"},"change_message_id":"dddea222430e63dcbd43ffa50a6530fb121f1cb8","unresolved":false,"context_lines":[{"line_number":16,"context_line":"configuration parameter."},{"line_number":17,"context_line":""},{"line_number":18,"context_line":"The current back-off logic implemented in ``neutron/common/rpc.py``"},{"line_number":19,"context_line":"is to increase the rpc_response_timeout up to 10 times before resoting"},{"line_number":20,"context_line":"to chunking. In effect this tries to scale up the RPC response timeout"},{"line_number":21,"context_line":"window but still floods the server with calls that take several minutes"},{"line_number":22,"context_line":"to process."}],"source_content_type":"text/x-gerrit-commit-message","patch_set":5,"id":"df140735_45d25b5e","line":19,"range":{"start_line":19,"start_character":62,"end_line":19,"end_character":70},"updated":"2017-05-30 19:16:01.000000000","message":"(resoRting)","commit_id":"8e8918c8ac1d5ff34c849bc0d1cc896ff24bade2"},{"author":{"_account_id":12021,"name":"Thomas Morin","email":"tmmorin.orange@gmail.com","username":"tmorin"},"change_message_id":"dddea222430e63dcbd43ffa50a6530fb121f1cb8","unresolved":false,"context_lines":[{"line_number":22,"context_line":"to process."},{"line_number":23,"context_line":""},{"line_number":24,"context_line":"We observed that in environments with multiple L3 agents (more than 6)"},{"line_number":25,"context_line":"and high number of routers pe agent (more than 240) the RCP call to"},{"line_number":26,"context_line":"sync_routes behaves exponentially instead of linearly."},{"line_number":27,"context_line":""},{"line_number":28,"context_line":"The solution that the neutron-server advertises in the logs (to increase"}],"source_content_type":"text/x-gerrit-commit-message","patch_set":5,"id":"df140735_e55e0fe5","line":25,"range":{"start_line":25,"start_character":56,"end_line":25,"end_character":59},"updated":"2017-05-30 19:16:01.000000000","message":"(RPC)","commit_id":"8e8918c8ac1d5ff34c849bc0d1cc896ff24bade2"},{"author":{"_account_id":12021,"name":"Thomas Morin","email":"tmmorin.orange@gmail.com","username":"tmorin"},"change_message_id":"dddea222430e63dcbd43ffa50a6530fb121f1cb8","unresolved":false,"context_lines":[{"line_number":22,"context_line":"to process."},{"line_number":23,"context_line":""},{"line_number":24,"context_line":"We observed that in environments with multiple L3 agents (more than 6)"},{"line_number":25,"context_line":"and high number of routers pe agent (more than 240) the RCP call to"},{"line_number":26,"context_line":"sync_routes behaves exponentially instead of linearly."},{"line_number":27,"context_line":""},{"line_number":28,"context_line":"The solution that the neutron-server advertises in the logs (to increase"}],"source_content_type":"text/x-gerrit-commit-message","patch_set":5,"id":"df140735_256527b6","line":25,"updated":"2017-05-30 19:16:01.000000000","message":"(a high)","commit_id":"8e8918c8ac1d5ff34c849bc0d1cc896ff24bade2"},{"author":{"_account_id":12021,"name":"Thomas Morin","email":"tmmorin.orange@gmail.com","username":"tmorin"},"change_message_id":"dddea222430e63dcbd43ffa50a6530fb121f1cb8","unresolved":false,"context_lines":[{"line_number":22,"context_line":"to process."},{"line_number":23,"context_line":""},{"line_number":24,"context_line":"We observed that in environments with multiple L3 agents (more than 6)"},{"line_number":25,"context_line":"and high number of routers pe agent (more than 240) the RCP call to"},{"line_number":26,"context_line":"sync_routes behaves exponentially instead of linearly."},{"line_number":27,"context_line":""},{"line_number":28,"context_line":"The solution that the neutron-server advertises in the logs (to increase"}],"source_content_type":"text/x-gerrit-commit-message","patch_set":5,"id":"df140735_055a23f2","line":25,"range":{"start_line":25,"start_character":27,"end_line":25,"end_character":29},"updated":"2017-05-30 19:16:01.000000000","message":"(per)","commit_id":"8e8918c8ac1d5ff34c849bc0d1cc896ff24bade2"},{"author":{"_account_id":12021,"name":"Thomas Morin","email":"tmmorin.orange@gmail.com","username":"tmorin"},"change_message_id":"dddea222430e63dcbd43ffa50a6530fb121f1cb8","unresolved":false,"context_lines":[{"line_number":31,"context_line":""},{"line_number":32,"context_line":"During our experiments we oserved that lower values for the"},{"line_number":33,"context_line":"SYNC_ROUTERS_MAX_CHUNK_SIZE (tested with 64 and 32) allows the server"},{"line_number":34,"context_line":"to respond within the timeout window and leads to a total recoveryi"},{"line_number":35,"context_line":"within 3-5 minutes. While in theory this might increase the time needed"},{"line_number":36,"context_line":"for the agent to fully recover due to trickling the list of routers"},{"line_number":37,"context_line":"from the server, the observed behavior in practice is that it avoids"}],"source_content_type":"text/x-gerrit-commit-message","patch_set":5,"id":"df140735_856d139a","line":34,"range":{"start_line":34,"start_character":58,"end_line":34,"end_character":67},"updated":"2017-05-30 19:16:01.000000000","message":"(recovery)","commit_id":"8e8918c8ac1d5ff34c849bc0d1cc896ff24bade2"}],"neutron/agent/l3/agent.py":[{"author":{"_account_id":1653,"name":"garyk","email":"gkotton@vmware.com","username":"garyk"},"change_message_id":"a9ddf8a1bd9faa4a07be8cb649c9d0c5978382ec","unresolved":false,"context_lines":[{"line_number":251,"context_line":"        The actual values are not verified for correctness."},{"line_number":252,"context_line":"        \"\"\""},{"line_number":253,"context_line":"        if (self.conf.sync_routers_max_chunk_size \u003c"},{"line_number":254,"context_line":"            self.conf.sync_routers_min_chunk_size):"},{"line_number":255,"context_line":"            msg \u003d _LW(\"sync_routers_min_chunk_size\u003d%d is greater than \""},{"line_number":256,"context_line":"                      \"sync_routers_max_chunk_size\u003d%d. It is recommended \""},{"line_number":257,"context_line":"                      \"that sync_routers_min_chunk_size be at most half the \""}],"source_content_type":"text/x-python","patch_set":2,"id":"df140735_503d9fb9","line":254,"updated":"2017-05-29 12:44:42.000000000","message":"i think that you should fail here like in line 265","commit_id":"d5a77f0edcee2ae7c74b23de7750b2120ac54bce"},{"author":{"_account_id":26086,"name":"Cristian Calin","email":"cristian.calin@orange.com","username":"cristicalin"},"change_message_id":"27069cd01032cf211ef7fd8a06326ca99bc786f5","unresolved":false,"context_lines":[{"line_number":251,"context_line":"        The actual values are not verified for correctness."},{"line_number":252,"context_line":"        \"\"\""},{"line_number":253,"context_line":"        if (self.conf.sync_routers_max_chunk_size \u003c"},{"line_number":254,"context_line":"            self.conf.sync_routers_min_chunk_size):"},{"line_number":255,"context_line":"            msg \u003d _LW(\"sync_routers_min_chunk_size\u003d%d is greater than \""},{"line_number":256,"context_line":"                      \"sync_routers_max_chunk_size\u003d%d. It is recommended \""},{"line_number":257,"context_line":"                      \"that sync_routers_min_chunk_size be at most half the \""}],"source_content_type":"text/x-python","patch_set":2,"id":"df140735_13b5f131","line":254,"in_reply_to":"df140735_503d9fb9","updated":"2017-05-29 14:12:04.000000000","message":"It is not really a configuration error which would trigger a crash of the agent but it will never go through the scaling down process which is the point of this parameter.\n\nI guess it makes more sense to fail early instead of having an undesired behavior.","commit_id":"d5a77f0edcee2ae7c74b23de7750b2120ac54bce"},{"author":{"_account_id":1653,"name":"garyk","email":"gkotton@vmware.com","username":"garyk"},"change_message_id":"a9ddf8a1bd9faa4a07be8cb649c9d0c5978382ec","unresolved":false,"context_lines":[{"line_number":252,"context_line":"        \"\"\""},{"line_number":253,"context_line":"        if (self.conf.sync_routers_max_chunk_size \u003c"},{"line_number":254,"context_line":"            self.conf.sync_routers_min_chunk_size):"},{"line_number":255,"context_line":"            msg \u003d _LW(\"sync_routers_min_chunk_size\u003d%d is greater than \""},{"line_number":256,"context_line":"                      \"sync_routers_max_chunk_size\u003d%d. It is recommended \""},{"line_number":257,"context_line":"                      \"that sync_routers_min_chunk_size be at most half the \""},{"line_number":258,"context_line":"                      \"size of sync_routers_max_chunk_size in order to \""}],"source_content_type":"text/x-python","patch_set":2,"id":"df140735_10196778","line":255,"range":{"start_line":255,"start_character":51,"end_line":255,"end_character":53},"updated":"2017-05-29 12:44:42.000000000","message":"%(min_chunk_size)d","commit_id":"d5a77f0edcee2ae7c74b23de7750b2120ac54bce"},{"author":{"_account_id":1653,"name":"garyk","email":"gkotton@vmware.com","username":"garyk"},"change_message_id":"a9ddf8a1bd9faa4a07be8cb649c9d0c5978382ec","unresolved":false,"context_lines":[{"line_number":252,"context_line":"        \"\"\""},{"line_number":253,"context_line":"        if (self.conf.sync_routers_max_chunk_size \u003c"},{"line_number":254,"context_line":"            self.conf.sync_routers_min_chunk_size):"},{"line_number":255,"context_line":"            msg \u003d _LW(\"sync_routers_min_chunk_size\u003d%d is greater than \""},{"line_number":256,"context_line":"                      \"sync_routers_max_chunk_size\u003d%d. It is recommended \""},{"line_number":257,"context_line":"                      \"that sync_routers_min_chunk_size be at most half the \""},{"line_number":258,"context_line":"                      \"size of sync_routers_max_chunk_size in order to \""}],"source_content_type":"text/x-python","patch_set":2,"id":"df140735_103287f2","line":255,"range":{"start_line":255,"start_character":12,"end_line":255,"end_character":15},"updated":"2017-05-29 12:44:42.000000000","message":"why do we need this?","commit_id":"d5a77f0edcee2ae7c74b23de7750b2120ac54bce"},{"author":{"_account_id":26086,"name":"Cristian Calin","email":"cristian.calin@orange.com","username":"cristicalin"},"change_message_id":"27069cd01032cf211ef7fd8a06326ca99bc786f5","unresolved":false,"context_lines":[{"line_number":252,"context_line":"        \"\"\""},{"line_number":253,"context_line":"        if (self.conf.sync_routers_max_chunk_size \u003c"},{"line_number":254,"context_line":"            self.conf.sync_routers_min_chunk_size):"},{"line_number":255,"context_line":"            msg \u003d _LW(\"sync_routers_min_chunk_size\u003d%d is greater than \""},{"line_number":256,"context_line":"                      \"sync_routers_max_chunk_size\u003d%d. It is recommended \""},{"line_number":257,"context_line":"                      \"that sync_routers_min_chunk_size be at most half the \""},{"line_number":258,"context_line":"                      \"size of sync_routers_max_chunk_size in order to \""}],"source_content_type":"text/x-python","patch_set":2,"id":"df140735_5394898b","line":255,"range":{"start_line":255,"start_character":12,"end_line":255,"end_character":15},"in_reply_to":"df140735_103287f2","updated":"2017-05-29 14:12:04.000000000","message":"The warning is in place to explain how to properly set the parameters.\nGiven the comment above I will change it so that the agent errors out when the configuration is invalid.","commit_id":"d5a77f0edcee2ae7c74b23de7750b2120ac54bce"},{"author":{"_account_id":1653,"name":"garyk","email":"gkotton@vmware.com","username":"garyk"},"change_message_id":"a9ddf8a1bd9faa4a07be8cb649c9d0c5978382ec","unresolved":false,"context_lines":[{"line_number":253,"context_line":"        if (self.conf.sync_routers_max_chunk_size \u003c"},{"line_number":254,"context_line":"            self.conf.sync_routers_min_chunk_size):"},{"line_number":255,"context_line":"            msg \u003d _LW(\"sync_routers_min_chunk_size\u003d%d is greater than \""},{"line_number":256,"context_line":"                      \"sync_routers_max_chunk_size\u003d%d. It is recommended \""},{"line_number":257,"context_line":"                      \"that sync_routers_min_chunk_size be at most half the \""},{"line_number":258,"context_line":"                      \"size of sync_routers_max_chunk_size in order to \""},{"line_number":259,"context_line":"                      \"allow the agent to scale down the chunk size and \""}],"source_content_type":"text/x-python","patch_set":2,"id":"df140735_f02713b6","line":256,"range":{"start_line":256,"start_character":51,"end_line":256,"end_character":53},"updated":"2017-05-29 12:44:42.000000000","message":"%(max_chunk_size)d","commit_id":"d5a77f0edcee2ae7c74b23de7750b2120ac54bce"},{"author":{"_account_id":1653,"name":"garyk","email":"gkotton@vmware.com","username":"garyk"},"change_message_id":"a9ddf8a1bd9faa4a07be8cb649c9d0c5978382ec","unresolved":false,"context_lines":[{"line_number":258,"context_line":"                      \"size of sync_routers_max_chunk_size in order to \""},{"line_number":259,"context_line":"                      \"allow the agent to scale down the chunk size and \""},{"line_number":260,"context_line":"                      \"decrease the load on the neutron server.\")"},{"line_number":261,"context_line":"            LOG.warning(msg, self.conf.sync_routers_min_chunk_size,"},{"line_number":262,"context_line":"                        self.conf.sync_routers_max_chunk_size)"},{"line_number":263,"context_line":"        if not self.conf.interface_driver:"},{"line_number":264,"context_line":"            msg \u003d _LE(\u0027An interface driver must be specified\u0027)"},{"line_number":265,"context_line":"            LOG.error(msg)"}],"source_content_type":"text/x-python","patch_set":2,"id":"df140735_902697b7","line":262,"range":{"start_line":261,"start_character":29,"end_line":262,"end_character":61},"updated":"2017-05-29 12:44:42.000000000","message":"dictionary","commit_id":"d5a77f0edcee2ae7c74b23de7750b2120ac54bce"},{"author":{"_account_id":1131,"name":"Brian Haley","email":"haleyb.dev@gmail.com","username":"brian-haley"},"change_message_id":"ffec5373c7b51e82a915641985fe3d9a0769c7ee","unresolved":false,"context_lines":[{"line_number":61,"context_line":"# Number of routers to fetch from server at a time on resync."},{"line_number":62,"context_line":"# Needed to reduce load on server side and to speed up resync on agent side."},{"line_number":63,"context_line":"SYNC_ROUTERS_MAX_CHUNK_SIZE \u003d 256"},{"line_number":64,"context_line":"SYNC_ROUTERS_MIN_CHUNK_SIZE \u003d 32"},{"line_number":65,"context_line":""},{"line_number":66,"context_line":""},{"line_number":67,"context_line":"def log_verbose_exc(message, router_payload):"}],"source_content_type":"text/x-python","patch_set":6,"id":"df140735_ebac39ee","side":"PARENT","line":64,"updated":"2017-06-02 14:42:12.000000000","message":"I\u0027m thinking we should move this block to neutron/common/constants.py and use them in the config change.","commit_id":"e137e63db3d27451fce17227bcc12194ddd7d0c6"},{"author":{"_account_id":12021,"name":"Thomas Morin","email":"tmmorin.orange@gmail.com","username":"tmorin"},"change_message_id":"0bd2cff3c7e9370b3a5b6e1bcbed470fffcaf9c4","unresolved":false,"context_lines":[{"line_number":61,"context_line":"# Number of routers to fetch from server at a time on resync."},{"line_number":62,"context_line":"# Needed to reduce load on server side and to speed up resync on agent side."},{"line_number":63,"context_line":"SYNC_ROUTERS_MAX_CHUNK_SIZE \u003d 256"},{"line_number":64,"context_line":"SYNC_ROUTERS_MIN_CHUNK_SIZE \u003d 32"},{"line_number":65,"context_line":""},{"line_number":66,"context_line":""},{"line_number":67,"context_line":"def log_verbose_exc(message, router_payload):"}],"source_content_type":"text/x-python","patch_set":6,"id":"df140735_41c4b492","side":"PARENT","line":64,"in_reply_to":"df140735_d672cc49","updated":"2017-06-07 07:36:11.000000000","message":"I think this would be totally overkill for a value that is only used in the option definition and hence does not need to be coordinated/reused anywhere else.","commit_id":"e137e63db3d27451fce17227bcc12194ddd7d0c6"},{"author":{"_account_id":26086,"name":"Cristian Calin","email":"cristian.calin@orange.com","username":"cristicalin"},"change_message_id":"b26d76e57a03f460a130e64c02ba7ca2c131569e","unresolved":false,"context_lines":[{"line_number":61,"context_line":"# Number of routers to fetch from server at a time on resync."},{"line_number":62,"context_line":"# Needed to reduce load on server side and to speed up resync on agent side."},{"line_number":63,"context_line":"SYNC_ROUTERS_MAX_CHUNK_SIZE \u003d 256"},{"line_number":64,"context_line":"SYNC_ROUTERS_MIN_CHUNK_SIZE \u003d 32"},{"line_number":65,"context_line":""},{"line_number":66,"context_line":""},{"line_number":67,"context_line":"def log_verbose_exc(message, router_payload):"}],"source_content_type":"text/x-python","patch_set":6,"id":"df140735_d672cc49","side":"PARENT","line":64,"in_reply_to":"df140735_ebac39ee","updated":"2017-06-07 07:26:46.000000000","message":"This means a change to the neutron-lib project (separate change I guess), but the point is indeed valid. It was my first impulse to do so but being a cross-project change I wanted to get feedback on the logic of the change first.","commit_id":"e137e63db3d27451fce17227bcc12194ddd7d0c6"}],"neutron/conf/agent/l3/config.py":[{"author":{"_account_id":12021,"name":"Thomas Morin","email":"tmmorin.orange@gmail.com","username":"tmorin"},"change_message_id":"a88499675ee25f4f39c3be2fd9f8e2fd7691308b","unresolved":false,"context_lines":[{"line_number":104,"context_line":"    cfg.IntOpt(\u0027sync_routers_min_chunk_size\u0027,"},{"line_number":105,"context_line":"               default\u003d32,"},{"line_number":106,"context_line":"               min\u003d1,"},{"line_number":107,"context_line":"               max\u003d\u0027$sync_routers_max_chunk_size\u0027,"},{"line_number":108,"context_line":"               help\u003d_(\u0027Minimum chunk size for the number of routers to fetch \u0027"},{"line_number":109,"context_line":"                      \u0027in a single resync call to the server. Make sure this is \u0027"},{"line_number":110,"context_line":"                      \u0027at most half of sync_routers_max_chunk_size to allow the \u0027"}],"source_content_type":"text/x-python","patch_set":2,"id":"df140735_b661983c","line":107,"range":{"start_line":107,"start_character":19,"end_line":107,"end_character":50},"updated":"2017-05-29 11:14:40.000000000","message":"this seems to make the py35 check fail, apparently because this produces a string which py35 does not want to compare with an int\n\nis oslo.config interpolation available for these values ?\n(or only for default) ?","commit_id":"d5a77f0edcee2ae7c74b23de7750b2120ac54bce"},{"author":{"_account_id":26086,"name":"Cristian Calin","email":"cristian.calin@orange.com","username":"cristicalin"},"change_message_id":"b6ad774ba590d0b16e942f388efd57165fae7c30","unresolved":false,"context_lines":[{"line_number":104,"context_line":"    cfg.IntOpt(\u0027sync_routers_min_chunk_size\u0027,"},{"line_number":105,"context_line":"               default\u003d32,"},{"line_number":106,"context_line":"               min\u003d1,"},{"line_number":107,"context_line":"               max\u003d\u0027$sync_routers_max_chunk_size\u0027,"},{"line_number":108,"context_line":"               help\u003d_(\u0027Minimum chunk size for the number of routers to fetch \u0027"},{"line_number":109,"context_line":"                      \u0027in a single resync call to the server. Make sure this is \u0027"},{"line_number":110,"context_line":"                      \u0027at most half of sync_routers_max_chunk_size to allow the \u0027"}],"source_content_type":"text/x-python","patch_set":2,"id":"df140735_9c7a3386","line":107,"range":{"start_line":107,"start_character":19,"end_line":107,"end_character":50},"in_reply_to":"df140735_b661983c","updated":"2017-05-29 11:56:13.000000000","message":"According to https://github.com/openstack/oslo.config/blob/master/oslo_config/cfg.py#L909-L914 only to default. I will remove the max parameter as it doesn\u0027t make much sense to force it, minimum should suffice.","commit_id":"d5a77f0edcee2ae7c74b23de7750b2120ac54bce"},{"author":{"_account_id":1131,"name":"Brian Haley","email":"haleyb.dev@gmail.com","username":"brian-haley"},"change_message_id":"ffec5373c7b51e82a915641985fe3d9a0769c7ee","unresolved":false,"context_lines":[{"line_number":98,"context_line":"               min\u003d1,"},{"line_number":99,"context_line":"               advanced\u003dTrue,"},{"line_number":100,"context_line":"               help\u003d_(\u0027Initial chunk size for the number of routers to fetch \u0027"},{"line_number":101,"context_line":"                      \u0027in a single resync call to the server. Decrease this \u0027"},{"line_number":102,"context_line":"                      \u0027to reduce the load on overloaded servers by sending \u0027"},{"line_number":103,"context_line":"                      \u0027multiple smaller requests and avoid a cascading \u0027"},{"line_number":104,"context_line":"                      \u0027failure effect.\u0027)),"},{"line_number":105,"context_line":"    cfg.IntOpt(\u0027sync_routers_min_chunk_size\u0027,"},{"line_number":106,"context_line":"               default\u003d32,"},{"line_number":107,"context_line":"               min\u003d1,"}],"source_content_type":"text/x-python","patch_set":6,"id":"df140735_2b0b5124","line":104,"range":{"start_line":101,"start_character":62,"end_line":104,"end_character":38},"updated":"2017-06-02 14:42:12.000000000","message":"I don\u0027t think we need all the info about a cascading failure effect, something shorter like:\n\ns/Decrease this value if the server is unable to handle large requests due to load.","commit_id":"27137b7322de01879e13a5b13d3d3e8225d39ae7"},{"author":{"_account_id":26086,"name":"Cristian Calin","email":"cristian.calin@orange.com","username":"cristicalin"},"change_message_id":"b26d76e57a03f460a130e64c02ba7ca2c131569e","unresolved":false,"context_lines":[{"line_number":98,"context_line":"               min\u003d1,"},{"line_number":99,"context_line":"               advanced\u003dTrue,"},{"line_number":100,"context_line":"               help\u003d_(\u0027Initial chunk size for the number of routers to fetch \u0027"},{"line_number":101,"context_line":"                      \u0027in a single resync call to the server. Decrease this \u0027"},{"line_number":102,"context_line":"                      \u0027to reduce the load on overloaded servers by sending \u0027"},{"line_number":103,"context_line":"                      \u0027multiple smaller requests and avoid a cascading \u0027"},{"line_number":104,"context_line":"                      \u0027failure effect.\u0027)),"},{"line_number":105,"context_line":"    cfg.IntOpt(\u0027sync_routers_min_chunk_size\u0027,"},{"line_number":106,"context_line":"               default\u003d32,"},{"line_number":107,"context_line":"               min\u003d1,"}],"source_content_type":"text/x-python","patch_set":6,"id":"df140735_76b0a0de","line":104,"range":{"start_line":101,"start_character":62,"end_line":104,"end_character":38},"in_reply_to":"df140735_2b0b5124","updated":"2017-06-07 07:26:46.000000000","message":"Noted. I\u0027ll move this more detailed description to the release notes.","commit_id":"27137b7322de01879e13a5b13d3d3e8225d39ae7"},{"author":{"_account_id":1131,"name":"Brian Haley","email":"haleyb.dev@gmail.com","username":"brian-haley"},"change_message_id":"ffec5373c7b51e82a915641985fe3d9a0769c7ee","unresolved":false,"context_lines":[{"line_number":107,"context_line":"               min\u003d1,"},{"line_number":108,"context_line":"               advanced\u003dTrue,"},{"line_number":109,"context_line":"               help\u003d_(\u0027Minimum chunk size for the number of routers to fetch \u0027"},{"line_number":110,"context_line":"                      \u0027in a single resync call to the server. Make sure this \u0027"},{"line_number":111,"context_line":"                      \u0027is at most half of sync_routers_max_chunk_size to \u0027"},{"line_number":112,"context_line":"                      \u0027allow the agent to scale down the chunk size.\u0027)),"},{"line_number":113,"context_line":"]"},{"line_number":114,"context_line":""},{"line_number":115,"context_line":"OPTS +\u003d config.EXT_NET_BRIDGE_OPTS"}],"source_content_type":"text/x-python","patch_set":6,"id":"df140735_a8a52749","line":112,"range":{"start_line":110,"start_character":62,"end_line":112,"end_character":68},"updated":"2017-06-02 14:42:12.000000000","message":"s/It is recommended to not set this higher than sync_routers_max_chunk_size/2 to allow the agent to scale down the chunk size","commit_id":"27137b7322de01879e13a5b13d3d3e8225d39ae7"},{"author":{"_account_id":26086,"name":"Cristian Calin","email":"cristian.calin@orange.com","username":"cristicalin"},"change_message_id":"b26d76e57a03f460a130e64c02ba7ca2c131569e","unresolved":false,"context_lines":[{"line_number":107,"context_line":"               min\u003d1,"},{"line_number":108,"context_line":"               advanced\u003dTrue,"},{"line_number":109,"context_line":"               help\u003d_(\u0027Minimum chunk size for the number of routers to fetch \u0027"},{"line_number":110,"context_line":"                      \u0027in a single resync call to the server. Make sure this \u0027"},{"line_number":111,"context_line":"                      \u0027is at most half of sync_routers_max_chunk_size to \u0027"},{"line_number":112,"context_line":"                      \u0027allow the agent to scale down the chunk size.\u0027)),"},{"line_number":113,"context_line":"]"},{"line_number":114,"context_line":""},{"line_number":115,"context_line":"OPTS +\u003d config.EXT_NET_BRIDGE_OPTS"}],"source_content_type":"text/x-python","patch_set":6,"id":"df140735_d6c08c2d","line":112,"range":{"start_line":110,"start_character":62,"end_line":112,"end_character":68},"in_reply_to":"df140735_a8a52749","updated":"2017-06-07 07:26:46.000000000","message":"OK","commit_id":"27137b7322de01879e13a5b13d3d3e8225d39ae7"}],"releasenotes/notes/l3-agent-sync_routers_chunk-size-f359f5a3db248f6c.yaml":[{"author":{"_account_id":1131,"name":"Brian Haley","email":"haleyb.dev@gmail.com","username":"brian-haley"},"change_message_id":"ffec5373c7b51e82a915641985fe3d9a0769c7ee","unresolved":false,"context_lines":[{"line_number":11,"context_line":"    by issuing a RPC sync_routers call with at most this"},{"line_number":12,"context_line":"    number of routers."},{"line_number":13,"context_line":"  - Add parameter ``sync_routers_min_chunk_size`` which"},{"line_number":14,"context_line":"    instructs the angent not to scale down the chunk"},{"line_number":15,"context_line":"    size to below this size."},{"line_number":16,"context_line":"critical:"},{"line_number":17,"context_line":"  - The sync_routers_{min|max}_chunk_size parameters"}],"source_content_type":"text/x-yaml","patch_set":6,"id":"df140735_4b1fa55a","line":14,"range":{"start_line":14,"start_character":18,"end_line":14,"end_character":24},"updated":"2017-06-02 14:42:12.000000000","message":"s/agent","commit_id":"27137b7322de01879e13a5b13d3d3e8225d39ae7"},{"author":{"_account_id":1131,"name":"Brian Haley","email":"haleyb.dev@gmail.com","username":"brian-haley"},"change_message_id":"ffec5373c7b51e82a915641985fe3d9a0769c7ee","unresolved":false,"context_lines":[{"line_number":19,"context_line":"    options to improve the response time for the"},{"line_number":20,"context_line":"    sync_routers call. A side effect of setting these"},{"line_number":21,"context_line":"    values is that the L3 agent start may be delayed"},{"line_number":22,"context_line":"    due to trickling down router metadata as opposed"},{"line_number":23,"context_line":"    to fetching all data at once."},{"line_number":24,"context_line":"fixes:"},{"line_number":25,"context_line":"  - Fixes bug/1692971. Neutron server may not be able to"}],"source_content_type":"text/x-yaml","patch_set":6,"id":"df140735_8b77fda0","line":22,"range":{"start_line":22,"start_character":11,"end_line":22,"end_character":25},"updated":"2017-06-02 14:42:12.000000000","message":"s/multiple calls being required to fetch","commit_id":"27137b7322de01879e13a5b13d3d3e8225d39ae7"}]}