)]}'
{"specs/train/l3-agent-capacity.rst":[{"author":{"_account_id":1131,"name":"Brian Haley","email":"haleyb.dev@gmail.com","username":"brian-haley"},"change_message_id":"78aaa03749bc6657c7e6ce72f99e50a1ac5509da","unresolved":false,"context_lines":[{"line_number":19,"context_line":"risk, especially for agent restart."},{"line_number":20,"context_line":""},{"line_number":21,"context_line":"This spec describes a scheduling mechanism for l3 router based on node"},{"line_number":22,"context_line":"bandwidth usage collection and reporting."},{"line_number":23,"context_line":""},{"line_number":24,"context_line":""},{"line_number":25,"context_line":"Problem Description"}],"source_content_type":"text/x-rst","patch_set":2,"id":"dfbec78f_e23d67dd","line":22,"updated":"2019-05-10 21:38:59.000000000","message":"Alternately the cloud admin could charge for routers, or set a small quota so when you need a lot you have to pay to increase it.  This just doesn\u0027t sound like a \"friendly\" operator though if I have to pay for basic things.","commit_id":"c353ed4de8324a62148cb6b7ceb63b648f45ba8d"},{"author":{"_account_id":9531,"name":"liuyulong","display_name":"LIU Yulong","email":"i@liuyulong.me","username":"LIU-Yulong"},"change_message_id":"6006d5bfbf1f4998fc8839f00efb7a23c202a3e3","unresolved":false,"context_lines":[{"line_number":19,"context_line":"risk, especially for agent restart."},{"line_number":20,"context_line":""},{"line_number":21,"context_line":"This spec describes a scheduling mechanism for l3 router based on node"},{"line_number":22,"context_line":"bandwidth usage collection and reporting."},{"line_number":23,"context_line":""},{"line_number":24,"context_line":""},{"line_number":25,"context_line":"Problem Description"}],"source_content_type":"text/x-rst","patch_set":2,"id":"dfbec78f_05c0c5e0","line":22,"in_reply_to":"dfbec78f_e23d67dd","updated":"2019-05-10 22:46:11.000000000","message":"Small quota does not actually solve the problem, since The number of users can grow indefinitely. So if no charging, problem is still there. And indeed, manual adjustment of quota is unavailable to operate.","commit_id":"c353ed4de8324a62148cb6b7ceb63b648f45ba8d"},{"author":{"_account_id":1131,"name":"Brian Haley","email":"haleyb.dev@gmail.com","username":"brian-haley"},"change_message_id":"78aaa03749bc6657c7e6ce72f99e50a1ac5509da","unresolved":false,"context_lines":[{"line_number":50,"context_line":""},{"line_number":51,"context_line":"* Add a ``bandwidth`` bandwidth for router which indicates the total"},{"line_number":52,"context_line":"  bandwidth of a router can use. This will be a new extension and OVO"},{"line_number":53,"context_line":"  resouce ``RouterBandwidth``."},{"line_number":54,"context_line":""},{"line_number":55,"context_line":"* Relationship table between ``Router`` and ``RouterBandwidth``."},{"line_number":56,"context_line":""}],"source_content_type":"text/x-rst","patch_set":2,"id":"dfbec78f_8514b545","line":53,"updated":"2019-05-10 21:38:59.000000000","message":"Bandwidth to me sounds like something we\u0027d do with QoS, and to me would imply we are limiting routers to not consume a traffic-based quantity.\n\nDo you just want to be able to say an l3-agent can only host a certain amount of routers?  Today I would think an operator plans for some amount (routers, instances, whatever) and increases capacity if necessary (more controllers, etc), so I\u0027m not sure I agree with this proposal - it\u0027s something we should talk about at the drivers meeting.","commit_id":"c353ed4de8324a62148cb6b7ceb63b648f45ba8d"},{"author":{"_account_id":9531,"name":"liuyulong","display_name":"LIU Yulong","email":"i@liuyulong.me","username":"LIU-Yulong"},"change_message_id":"6006d5bfbf1f4998fc8839f00efb7a23c202a3e3","unresolved":false,"context_lines":[{"line_number":50,"context_line":""},{"line_number":51,"context_line":"* Add a ``bandwidth`` bandwidth for router which indicates the total"},{"line_number":52,"context_line":"  bandwidth of a router can use. This will be a new extension and OVO"},{"line_number":53,"context_line":"  resouce ``RouterBandwidth``."},{"line_number":54,"context_line":""},{"line_number":55,"context_line":"* Relationship table between ``Router`` and ``RouterBandwidth``."},{"line_number":56,"context_line":""}],"source_content_type":"text/x-rst","patch_set":2,"id":"dfbec78f_25ce292e","line":53,"in_reply_to":"dfbec78f_8514b545","updated":"2019-05-10 22:46:11.000000000","message":"For the virtual machine (nova instance), we basically have CPU and memory to estimate capacity. But for the centralized SNAT nodes, we have no such calculation. As I mentioned in \"Problem Description\" b), we even have no method to limit the number of routes that grow indefinitely as tenants grow.","commit_id":"c353ed4de8324a62148cb6b7ceb63b648f45ba8d"},{"author":{"_account_id":1131,"name":"Brian Haley","email":"haleyb.dev@gmail.com","username":"brian-haley"},"change_message_id":"78aaa03749bc6657c7e6ce72f99e50a1ac5509da","unresolved":false,"context_lines":[{"line_number":77,"context_line":"* 1. User create a router with specified or default bandwidth."},{"line_number":78,"context_line":"* 2. Scheduler router according to the bandwidth."},{"line_number":79,"context_line":"* 3. Available agent selected return 200 to user"},{"line_number":80,"context_line":"* 4. Failed to schedule return 400 to user"},{"line_number":81,"context_line":""},{"line_number":82,"context_line":""},{"line_number":83,"context_line":"REST API Impact"}],"source_content_type":"text/x-rst","patch_set":2,"id":"dfbec78f_e2ba476c","line":80,"updated":"2019-05-10 21:38:59.000000000","message":"How would this work?  Creating a router doesn\u0027t schedule it, adding an interface would, so we can\u0027t stop the creation and return an error like this.","commit_id":"c353ed4de8324a62148cb6b7ceb63b648f45ba8d"},{"author":{"_account_id":9531,"name":"liuyulong","display_name":"LIU Yulong","email":"i@liuyulong.me","username":"LIU-Yulong"},"change_message_id":"6006d5bfbf1f4998fc8839f00efb7a23c202a3e3","unresolved":false,"context_lines":[{"line_number":77,"context_line":"* 1. User create a router with specified or default bandwidth."},{"line_number":78,"context_line":"* 2. Scheduler router according to the bandwidth."},{"line_number":79,"context_line":"* 3. Available agent selected return 200 to user"},{"line_number":80,"context_line":"* 4. Failed to schedule return 400 to user"},{"line_number":81,"context_line":""},{"line_number":82,"context_line":""},{"line_number":83,"context_line":"REST API Impact"}],"source_content_type":"text/x-rst","patch_set":2,"id":"dfbec78f_e5ec9190","line":80,"in_reply_to":"dfbec78f_e2ba476c","updated":"2019-05-10 22:46:11.000000000","message":"For legacy and dvr Router, when it finished the creationm, we will force to schedule it. For HA routers, it is already in such way.","commit_id":"c353ed4de8324a62148cb6b7ceb63b648f45ba8d"},{"author":{"_account_id":7016,"name":"Swaminathan Vasudevan","email":"swvasude@cisco.com","username":"souminathan"},"change_message_id":"9946e9a718bcd561b1f5c49211d8831e1d9ebbed","unresolved":false,"context_lines":[{"line_number":10,"context_line":""},{"line_number":11,"context_line":"RFE: https://bugs.launchpad.net/neutron/+bug/1828494"},{"line_number":12,"context_line":""},{"line_number":13,"context_line":"For a large sacle public cloud it meets scale issue about L3-agent. Most"},{"line_number":14,"context_line":"cloud service provider does not charge for the neutron virtual router."},{"line_number":15,"context_line":"This can become a headach for the operators. Every tenant may create"},{"line_number":16,"context_line":"free routers for doing nothing. But neutron will create many resource"}],"source_content_type":"text/x-rst","patch_set":3,"id":"dfbec78f_1867ccbf","line":13,"range":{"start_line":13,"start_character":12,"end_line":13,"end_character":18},"updated":"2019-05-10 23:27:10.000000000","message":"scale","commit_id":"c008f1c056c88d059e039638627412a53709df2c"},{"author":{"_account_id":9531,"name":"liuyulong","display_name":"LIU Yulong","email":"i@liuyulong.me","username":"LIU-Yulong"},"change_message_id":"65e95cf4361ff2e42d3ba9ee0dcd2ecab1dc42f0","unresolved":false,"context_lines":[{"line_number":10,"context_line":""},{"line_number":11,"context_line":"RFE: https://bugs.launchpad.net/neutron/+bug/1828494"},{"line_number":12,"context_line":""},{"line_number":13,"context_line":"For a large sacle public cloud it meets scale issue about L3-agent. Most"},{"line_number":14,"context_line":"cloud service provider does not charge for the neutron virtual router."},{"line_number":15,"context_line":"This can become a headach for the operators. Every tenant may create"},{"line_number":16,"context_line":"free routers for doing nothing. But neutron will create many resource"}],"source_content_type":"text/x-rst","patch_set":3,"id":"dfbec78f_b859e08e","line":13,"range":{"start_line":13,"start_character":12,"end_line":13,"end_character":18},"in_reply_to":"dfbec78f_1867ccbf","updated":"2019-05-11 00:27:35.000000000","message":"Done","commit_id":"c008f1c056c88d059e039638627412a53709df2c"},{"author":{"_account_id":7016,"name":"Swaminathan Vasudevan","email":"swvasude@cisco.com","username":"souminathan"},"change_message_id":"9946e9a718bcd561b1f5c49211d8831e1d9ebbed","unresolved":false,"context_lines":[{"line_number":12,"context_line":""},{"line_number":13,"context_line":"For a large sacle public cloud it meets scale issue about L3-agent. Most"},{"line_number":14,"context_line":"cloud service provider does not charge for the neutron virtual router."},{"line_number":15,"context_line":"This can become a headach for the operators. Every tenant may create"},{"line_number":16,"context_line":"free routers for doing nothing. But neutron will create many resource"},{"line_number":17,"context_line":"for it, especially the HA scenario, there will be namespaces, keepalived"},{"line_number":18,"context_line":"processes, and monitor processes. It will absolutely increase the failure"}],"source_content_type":"text/x-rst","patch_set":3,"id":"dfbec78f_b857a0ca","line":15,"range":{"start_line":15,"start_character":17,"end_line":15,"end_character":25},"updated":"2019-05-10 23:27:10.000000000","message":"headache","commit_id":"c008f1c056c88d059e039638627412a53709df2c"},{"author":{"_account_id":9531,"name":"liuyulong","display_name":"LIU Yulong","email":"i@liuyulong.me","username":"LIU-Yulong"},"change_message_id":"65e95cf4361ff2e42d3ba9ee0dcd2ecab1dc42f0","unresolved":false,"context_lines":[{"line_number":12,"context_line":""},{"line_number":13,"context_line":"For a large sacle public cloud it meets scale issue about L3-agent. Most"},{"line_number":14,"context_line":"cloud service provider does not charge for the neutron virtual router."},{"line_number":15,"context_line":"This can become a headach for the operators. Every tenant may create"},{"line_number":16,"context_line":"free routers for doing nothing. But neutron will create many resource"},{"line_number":17,"context_line":"for it, especially the HA scenario, there will be namespaces, keepalived"},{"line_number":18,"context_line":"processes, and monitor processes. It will absolutely increase the failure"}],"source_content_type":"text/x-rst","patch_set":3,"id":"dfbec78f_18690c20","line":15,"range":{"start_line":15,"start_character":17,"end_line":15,"end_character":25},"in_reply_to":"dfbec78f_b857a0ca","updated":"2019-05-11 00:27:35.000000000","message":"Done","commit_id":"c008f1c056c88d059e039638627412a53709df2c"},{"author":{"_account_id":7016,"name":"Swaminathan Vasudevan","email":"swvasude@cisco.com","username":"souminathan"},"change_message_id":"9946e9a718bcd561b1f5c49211d8831e1d9ebbed","unresolved":false,"context_lines":[{"line_number":42,"context_line":"  to estimate capacity. But for the centralized SNAT nodes, we have no such"},{"line_number":43,"context_line":"  calculation."},{"line_number":44,"context_line":""},{"line_number":45,"context_line":"* d) large sacle l3 resources increase OP difficulties."},{"line_number":46,"context_line":""},{"line_number":47,"context_line":"* e) L3 agent host a router for no compute resources, especially HA router,"},{"line_number":48,"context_line":"  this means router no real resources use router functionality. And somtimes"}],"source_content_type":"text/x-rst","patch_set":3,"id":"dfbec78f_386bb085","line":45,"range":{"start_line":45,"start_character":39,"end_line":45,"end_character":41},"updated":"2019-05-10 23:27:10.000000000","message":"expand it","commit_id":"c008f1c056c88d059e039638627412a53709df2c"},{"author":{"_account_id":9531,"name":"liuyulong","display_name":"LIU Yulong","email":"i@liuyulong.me","username":"LIU-Yulong"},"change_message_id":"65e95cf4361ff2e42d3ba9ee0dcd2ecab1dc42f0","unresolved":false,"context_lines":[{"line_number":42,"context_line":"  to estimate capacity. But for the centralized SNAT nodes, we have no such"},{"line_number":43,"context_line":"  calculation."},{"line_number":44,"context_line":""},{"line_number":45,"context_line":"* d) large sacle l3 resources increase OP difficulties."},{"line_number":46,"context_line":""},{"line_number":47,"context_line":"* e) L3 agent host a router for no compute resources, especially HA router,"},{"line_number":48,"context_line":"  this means router no real resources use router functionality. And somtimes"}],"source_content_type":"text/x-rst","patch_set":3,"id":"dfbec78f_f863d840","line":45,"range":{"start_line":45,"start_character":39,"end_line":45,"end_character":41},"in_reply_to":"dfbec78f_386bb085","updated":"2019-05-11 00:27:35.000000000","message":"Done","commit_id":"c008f1c056c88d059e039638627412a53709df2c"},{"author":{"_account_id":7016,"name":"Swaminathan Vasudevan","email":"swvasude@cisco.com","username":"souminathan"},"change_message_id":"9946e9a718bcd561b1f5c49211d8831e1d9ebbed","unresolved":false,"context_lines":[{"line_number":45,"context_line":"* d) large sacle l3 resources increase OP difficulties."},{"line_number":46,"context_line":""},{"line_number":47,"context_line":"* e) L3 agent host a router for no compute resources, especially HA router,"},{"line_number":48,"context_line":"  this means router no real resources use router functionality. And somtimes"},{"line_number":49,"context_line":"  causes waste of IP resources (This can be overcomed by [1]_), but still be"},{"line_number":50,"context_line":"  a problem without such settings."},{"line_number":51,"context_line":""}],"source_content_type":"text/x-rst","patch_set":3,"id":"dfbec78f_5868e485","line":48,"range":{"start_line":48,"start_character":68,"end_line":48,"end_character":76},"updated":"2019-05-10 23:27:10.000000000","message":"sometimes","commit_id":"c008f1c056c88d059e039638627412a53709df2c"},{"author":{"_account_id":9531,"name":"liuyulong","display_name":"LIU Yulong","email":"i@liuyulong.me","username":"LIU-Yulong"},"change_message_id":"65e95cf4361ff2e42d3ba9ee0dcd2ecab1dc42f0","unresolved":false,"context_lines":[{"line_number":45,"context_line":"* d) large sacle l3 resources increase OP difficulties."},{"line_number":46,"context_line":""},{"line_number":47,"context_line":"* e) L3 agent host a router for no compute resources, especially HA router,"},{"line_number":48,"context_line":"  this means router no real resources use router functionality. And somtimes"},{"line_number":49,"context_line":"  causes waste of IP resources (This can be overcomed by [1]_), but still be"},{"line_number":50,"context_line":"  a problem without such settings."},{"line_number":51,"context_line":""}],"source_content_type":"text/x-rst","patch_set":3,"id":"dfbec78f_5871c40b","line":48,"range":{"start_line":48,"start_character":68,"end_line":48,"end_character":76},"in_reply_to":"dfbec78f_5868e485","updated":"2019-05-11 00:27:35.000000000","message":"Done","commit_id":"c008f1c056c88d059e039638627412a53709df2c"},{"author":{"_account_id":11975,"name":"Slawek Kaplonski","email":"skaplons@redhat.com","username":"slaweq"},"change_message_id":"5f3084346433084bcca7c2937ac80c972c80cc23","unresolved":false,"context_lines":[{"line_number":12,"context_line":""},{"line_number":13,"context_line":"For a large scale public cloud it meets scale issue about L3-agent. Most"},{"line_number":14,"context_line":"cloud service provider does not charge for the neutron virtual router."},{"line_number":15,"context_line":"This can become a headache for the operators. Every tenant may create"},{"line_number":16,"context_line":"free routers for doing nothing. But neutron will create many resource"},{"line_number":17,"context_line":"for it, especially the HA scenario, there will be namespaces, keepalived"},{"line_number":18,"context_line":"processes, and monitor processes. It will absolutely increase the failure"},{"line_number":19,"context_line":"risk, especially for agent restart."},{"line_number":20,"context_line":""},{"line_number":21,"context_line":"This spec describes a scheduling mechanism for l3 router based on node"},{"line_number":22,"context_line":"bandwidth usage collection and reporting."}],"source_content_type":"text/x-rst","patch_set":5,"id":"bfb3d3c7_bb800348","line":19,"range":{"start_line":15,"start_character":46,"end_line":19,"end_character":35},"updated":"2019-05-24 07:24:48.000000000","message":"IIUC proposed in this spec solution is to limit number of routers hosted on L3 agent but that will not solve problem described here. Still any user can create routers and use resources. The only difference would be that operator will need to add extra network nodes to host new routers.","commit_id":"8669ab4ff25513f6ba2f25bcddd8e1f07655f2d9"},{"author":{"_account_id":9531,"name":"liuyulong","display_name":"LIU Yulong","email":"i@liuyulong.me","username":"LIU-Yulong"},"change_message_id":"268172d1e3cff769201242bdc328687ecaa78afc","unresolved":false,"context_lines":[{"line_number":12,"context_line":""},{"line_number":13,"context_line":"For a large scale public cloud it meets scale issue about L3-agent. Most"},{"line_number":14,"context_line":"cloud service provider does not charge for the neutron virtual router."},{"line_number":15,"context_line":"This can become a headache for the operators. Every tenant may create"},{"line_number":16,"context_line":"free routers for doing nothing. But neutron will create many resource"},{"line_number":17,"context_line":"for it, especially the HA scenario, there will be namespaces, keepalived"},{"line_number":18,"context_line":"processes, and monitor processes. It will absolutely increase the failure"},{"line_number":19,"context_line":"risk, especially for agent restart."},{"line_number":20,"context_line":""},{"line_number":21,"context_line":"This spec describes a scheduling mechanism for l3 router based on node"},{"line_number":22,"context_line":"bandwidth usage collection and reporting."}],"source_content_type":"text/x-rst","patch_set":5,"id":"bfb3d3c7_ec2fa33c","line":19,"range":{"start_line":15,"start_character":46,"end_line":19,"end_character":35},"in_reply_to":"bfb3d3c7_bb800348","updated":"2019-05-24 10:15:53.000000000","message":"Your concern can be another RFE I mentioned before \"cluster total resource usage limitaion\". https://bugs.launchpad.net/neutron/+bug/1817886\n\n\nFor this one, force scheduling will be needed for every router during the creating. If no availbale host to hold this router, return 400 to user. Then when the cluster meets the limit of scheduling, no more  new router.","commit_id":"8669ab4ff25513f6ba2f25bcddd8e1f07655f2d9"},{"author":{"_account_id":4694,"name":"Miguel Lavalle","email":"miguel@mlavalle.com","username":"minsel"},"change_message_id":"ce31dbcc8524a5e5336a66dd125b557d0711a87c","unresolved":false,"context_lines":[{"line_number":28,"context_line":"For centralized snat node, we have router scheduling mechanism for l3 router"},{"line_number":29,"context_line":"but we are facing the following problems:"},{"line_number":30,"context_line":""},{"line_number":31,"context_line":"* a) For a public cloud, the user (tenant) may be free to register. Then"},{"line_number":32,"context_line":"  they will have some free quota. The unrestricted growth of tenant"},{"line_number":33,"context_line":"  number will let the router number or any other free resource reach"},{"line_number":34,"context_line":"  a unbearable number of cluster."}],"source_content_type":"text/x-rst","patch_set":5,"id":"bfb3d3c7_fe08b0d9","line":31,"range":{"start_line":31,"start_character":2,"end_line":31,"end_character":4},"updated":"2019-05-23 21:56:11.000000000","message":"Since this is a bulleted list, you don\u0027t need this","commit_id":"8669ab4ff25513f6ba2f25bcddd8e1f07655f2d9"},{"author":{"_account_id":9531,"name":"liuyulong","display_name":"LIU Yulong","email":"i@liuyulong.me","username":"LIU-Yulong"},"change_message_id":"fe37fd0aad98453d472edfef8d6ff19df894870f","unresolved":false,"context_lines":[{"line_number":28,"context_line":"For centralized snat node, we have router scheduling mechanism for l3 router"},{"line_number":29,"context_line":"but we are facing the following problems:"},{"line_number":30,"context_line":""},{"line_number":31,"context_line":"* a) For a public cloud, the user (tenant) may be free to register. Then"},{"line_number":32,"context_line":"  they will have some free quota. The unrestricted growth of tenant"},{"line_number":33,"context_line":"  number will let the router number or any other free resource reach"},{"line_number":34,"context_line":"  a unbearable number of cluster."}],"source_content_type":"text/x-rst","patch_set":5,"id":"bfb3d3c7_ecc80323","line":31,"range":{"start_line":31,"start_character":2,"end_line":31,"end_character":4},"in_reply_to":"bfb3d3c7_fe08b0d9","updated":"2019-05-24 10:54:33.000000000","message":"Done","commit_id":"8669ab4ff25513f6ba2f25bcddd8e1f07655f2d9"},{"author":{"_account_id":4694,"name":"Miguel Lavalle","email":"miguel@mlavalle.com","username":"minsel"},"change_message_id":"ce31dbcc8524a5e5336a66dd125b557d0711a87c","unresolved":false,"context_lines":[{"line_number":33,"context_line":"  number will let the router number or any other free resource reach"},{"line_number":34,"context_line":"  a unbearable number of cluster."},{"line_number":35,"context_line":""},{"line_number":36,"context_line":"* b) Current ``LeastRoutersScheduler`` can not work, since we do not have a"},{"line_number":37,"context_line":"  maximum quantity for l3-agent. So router still is free to schedule to any"},{"line_number":38,"context_line":"  l3-agent. We have no method to limit the number of routes that grow"},{"line_number":39,"context_line":"  indefinitely as tenants grow."}],"source_content_type":"text/x-rst","patch_set":5,"id":"bfb3d3c7_5ef9bcc5","line":36,"range":{"start_line":36,"start_character":2,"end_line":36,"end_character":4},"updated":"2019-05-23 21:56:11.000000000","message":"DItto","commit_id":"8669ab4ff25513f6ba2f25bcddd8e1f07655f2d9"},{"author":{"_account_id":9531,"name":"liuyulong","display_name":"LIU Yulong","email":"i@liuyulong.me","username":"LIU-Yulong"},"change_message_id":"fe37fd0aad98453d472edfef8d6ff19df894870f","unresolved":false,"context_lines":[{"line_number":33,"context_line":"  number will let the router number or any other free resource reach"},{"line_number":34,"context_line":"  a unbearable number of cluster."},{"line_number":35,"context_line":""},{"line_number":36,"context_line":"* b) Current ``LeastRoutersScheduler`` can not work, since we do not have a"},{"line_number":37,"context_line":"  maximum quantity for l3-agent. So router still is free to schedule to any"},{"line_number":38,"context_line":"  l3-agent. We have no method to limit the number of routes that grow"},{"line_number":39,"context_line":"  indefinitely as tenants grow."}],"source_content_type":"text/x-rst","patch_set":5,"id":"bfb3d3c7_0cceb72d","line":36,"range":{"start_line":36,"start_character":2,"end_line":36,"end_character":4},"in_reply_to":"bfb3d3c7_5ef9bcc5","updated":"2019-05-24 10:54:33.000000000","message":"Done","commit_id":"8669ab4ff25513f6ba2f25bcddd8e1f07655f2d9"},{"author":{"_account_id":4694,"name":"Miguel Lavalle","email":"miguel@mlavalle.com","username":"minsel"},"change_message_id":"ce31dbcc8524a5e5336a66dd125b557d0711a87c","unresolved":false,"context_lines":[{"line_number":35,"context_line":""},{"line_number":36,"context_line":"* b) Current ``LeastRoutersScheduler`` can not work, since we do not have a"},{"line_number":37,"context_line":"  maximum quantity for l3-agent. So router still is free to schedule to any"},{"line_number":38,"context_line":"  l3-agent. We have no method to limit the number of routes that grow"},{"line_number":39,"context_line":"  indefinitely as tenants grow."},{"line_number":40,"context_line":""},{"line_number":41,"context_line":"* c) For the virtual machine (nova instance), we basically have CPU and memory"}],"source_content_type":"text/x-rst","patch_set":5,"id":"bfb3d3c7_5e221c54","line":38,"range":{"start_line":38,"start_character":53,"end_line":38,"end_character":59},"updated":"2019-05-23 21:56:11.000000000","message":"routers","commit_id":"8669ab4ff25513f6ba2f25bcddd8e1f07655f2d9"},{"author":{"_account_id":9531,"name":"liuyulong","display_name":"LIU Yulong","email":"i@liuyulong.me","username":"LIU-Yulong"},"change_message_id":"fe37fd0aad98453d472edfef8d6ff19df894870f","unresolved":false,"context_lines":[{"line_number":35,"context_line":""},{"line_number":36,"context_line":"* b) Current ``LeastRoutersScheduler`` can not work, since we do not have a"},{"line_number":37,"context_line":"  maximum quantity for l3-agent. So router still is free to schedule to any"},{"line_number":38,"context_line":"  l3-agent. We have no method to limit the number of routes that grow"},{"line_number":39,"context_line":"  indefinitely as tenants grow."},{"line_number":40,"context_line":""},{"line_number":41,"context_line":"* c) For the virtual machine (nova instance), we basically have CPU and memory"}],"source_content_type":"text/x-rst","patch_set":5,"id":"bfb3d3c7_acce8b2a","line":38,"range":{"start_line":38,"start_character":53,"end_line":38,"end_character":59},"in_reply_to":"bfb3d3c7_5e221c54","updated":"2019-05-24 10:54:33.000000000","message":"Done","commit_id":"8669ab4ff25513f6ba2f25bcddd8e1f07655f2d9"},{"author":{"_account_id":4694,"name":"Miguel Lavalle","email":"miguel@mlavalle.com","username":"minsel"},"change_message_id":"ce31dbcc8524a5e5336a66dd125b557d0711a87c","unresolved":false,"context_lines":[{"line_number":38,"context_line":"  l3-agent. We have no method to limit the number of routes that grow"},{"line_number":39,"context_line":"  indefinitely as tenants grow."},{"line_number":40,"context_line":""},{"line_number":41,"context_line":"* c) For the virtual machine (nova instance), we basically have CPU and memory"},{"line_number":42,"context_line":"  to estimate capacity. But for the centralized SNAT nodes, we have no such"},{"line_number":43,"context_line":"  calculation."},{"line_number":44,"context_line":""}],"source_content_type":"text/x-rst","patch_set":5,"id":"bfb3d3c7_1e0344ba","line":41,"range":{"start_line":41,"start_character":2,"end_line":41,"end_character":4},"updated":"2019-05-23 21:56:11.000000000","message":"DItto","commit_id":"8669ab4ff25513f6ba2f25bcddd8e1f07655f2d9"},{"author":{"_account_id":9531,"name":"liuyulong","display_name":"LIU Yulong","email":"i@liuyulong.me","username":"LIU-Yulong"},"change_message_id":"fe37fd0aad98453d472edfef8d6ff19df894870f","unresolved":false,"context_lines":[{"line_number":38,"context_line":"  l3-agent. We have no method to limit the number of routes that grow"},{"line_number":39,"context_line":"  indefinitely as tenants grow."},{"line_number":40,"context_line":""},{"line_number":41,"context_line":"* c) For the virtual machine (nova instance), we basically have CPU and memory"},{"line_number":42,"context_line":"  to estimate capacity. But for the centralized SNAT nodes, we have no such"},{"line_number":43,"context_line":"  calculation."},{"line_number":44,"context_line":""}],"source_content_type":"text/x-rst","patch_set":5,"id":"bfb3d3c7_ccd3bfd4","line":41,"range":{"start_line":41,"start_character":2,"end_line":41,"end_character":4},"in_reply_to":"bfb3d3c7_1e0344ba","updated":"2019-05-24 10:54:33.000000000","message":"Done","commit_id":"8669ab4ff25513f6ba2f25bcddd8e1f07655f2d9"},{"author":{"_account_id":4694,"name":"Miguel Lavalle","email":"miguel@mlavalle.com","username":"minsel"},"change_message_id":"ce31dbcc8524a5e5336a66dd125b557d0711a87c","unresolved":false,"context_lines":[{"line_number":42,"context_line":"  to estimate capacity. But for the centralized SNAT nodes, we have no such"},{"line_number":43,"context_line":"  calculation."},{"line_number":44,"context_line":""},{"line_number":45,"context_line":"* d) Large sacle l3 resources increase operation difficulties."},{"line_number":46,"context_line":""},{"line_number":47,"context_line":"* e) L3 agent host a router for no compute resources, especially HA router,"},{"line_number":48,"context_line":"  this means no real resources use router functionality. And sometimes"}],"source_content_type":"text/x-rst","patch_set":5,"id":"bfb3d3c7_3efe48ae","line":45,"range":{"start_line":45,"start_character":2,"end_line":45,"end_character":4},"updated":"2019-05-23 21:56:11.000000000","message":"DItto","commit_id":"8669ab4ff25513f6ba2f25bcddd8e1f07655f2d9"},{"author":{"_account_id":4694,"name":"Miguel Lavalle","email":"miguel@mlavalle.com","username":"minsel"},"change_message_id":"ce31dbcc8524a5e5336a66dd125b557d0711a87c","unresolved":false,"context_lines":[{"line_number":42,"context_line":"  to estimate capacity. But for the centralized SNAT nodes, we have no such"},{"line_number":43,"context_line":"  calculation."},{"line_number":44,"context_line":""},{"line_number":45,"context_line":"* d) Large sacle l3 resources increase operation difficulties."},{"line_number":46,"context_line":""},{"line_number":47,"context_line":"* e) L3 agent host a router for no compute resources, especially HA router,"},{"line_number":48,"context_line":"  this means no real resources use router functionality. And sometimes"}],"source_content_type":"text/x-rst","patch_set":5,"id":"bfb3d3c7_9eddf443","line":45,"range":{"start_line":45,"start_character":11,"end_line":45,"end_character":16},"updated":"2019-05-23 21:56:11.000000000","message":"scale","commit_id":"8669ab4ff25513f6ba2f25bcddd8e1f07655f2d9"},{"author":{"_account_id":9531,"name":"liuyulong","display_name":"LIU Yulong","email":"i@liuyulong.me","username":"LIU-Yulong"},"change_message_id":"fe37fd0aad98453d472edfef8d6ff19df894870f","unresolved":false,"context_lines":[{"line_number":42,"context_line":"  to estimate capacity. But for the centralized SNAT nodes, we have no such"},{"line_number":43,"context_line":"  calculation."},{"line_number":44,"context_line":""},{"line_number":45,"context_line":"* d) Large sacle l3 resources increase operation difficulties."},{"line_number":46,"context_line":""},{"line_number":47,"context_line":"* e) L3 agent host a router for no compute resources, especially HA router,"},{"line_number":48,"context_line":"  this means no real resources use router functionality. And sometimes"}],"source_content_type":"text/x-rst","patch_set":5,"id":"bfb3d3c7_6cdc13e4","line":45,"range":{"start_line":45,"start_character":2,"end_line":45,"end_character":4},"in_reply_to":"bfb3d3c7_3efe48ae","updated":"2019-05-24 10:54:33.000000000","message":"Done","commit_id":"8669ab4ff25513f6ba2f25bcddd8e1f07655f2d9"},{"author":{"_account_id":9531,"name":"liuyulong","display_name":"LIU Yulong","email":"i@liuyulong.me","username":"LIU-Yulong"},"change_message_id":"fe37fd0aad98453d472edfef8d6ff19df894870f","unresolved":false,"context_lines":[{"line_number":42,"context_line":"  to estimate capacity. But for the centralized SNAT nodes, we have no such"},{"line_number":43,"context_line":"  calculation."},{"line_number":44,"context_line":""},{"line_number":45,"context_line":"* d) Large sacle l3 resources increase operation difficulties."},{"line_number":46,"context_line":""},{"line_number":47,"context_line":"* e) L3 agent host a router for no compute resources, especially HA router,"},{"line_number":48,"context_line":"  this means no real resources use router functionality. And sometimes"}],"source_content_type":"text/x-rst","patch_set":5,"id":"bfb3d3c7_8cd947f4","line":45,"range":{"start_line":45,"start_character":11,"end_line":45,"end_character":16},"in_reply_to":"bfb3d3c7_9eddf443","updated":"2019-05-24 10:54:33.000000000","message":"Done","commit_id":"8669ab4ff25513f6ba2f25bcddd8e1f07655f2d9"},{"author":{"_account_id":4694,"name":"Miguel Lavalle","email":"miguel@mlavalle.com","username":"minsel"},"change_message_id":"ce31dbcc8524a5e5336a66dd125b557d0711a87c","unresolved":false,"context_lines":[{"line_number":44,"context_line":""},{"line_number":45,"context_line":"* d) Large sacle l3 resources increase operation difficulties."},{"line_number":46,"context_line":""},{"line_number":47,"context_line":"* e) L3 agent host a router for no compute resources, especially HA router,"},{"line_number":48,"context_line":"  this means no real resources use router functionality. And sometimes"},{"line_number":49,"context_line":"  causes waste of IP resources (This can be overcomed by [1]_), but still be"},{"line_number":50,"context_line":"  a problem without such settings."}],"source_content_type":"text/x-rst","patch_set":5,"id":"bfb3d3c7_de2e0c42","line":47,"range":{"start_line":47,"start_character":2,"end_line":47,"end_character":4},"updated":"2019-05-23 21:56:11.000000000","message":"DItto","commit_id":"8669ab4ff25513f6ba2f25bcddd8e1f07655f2d9"},{"author":{"_account_id":9531,"name":"liuyulong","display_name":"LIU Yulong","email":"i@liuyulong.me","username":"LIU-Yulong"},"change_message_id":"fe37fd0aad98453d472edfef8d6ff19df894870f","unresolved":false,"context_lines":[{"line_number":44,"context_line":""},{"line_number":45,"context_line":"* d) Large sacle l3 resources increase operation difficulties."},{"line_number":46,"context_line":""},{"line_number":47,"context_line":"* e) L3 agent host a router for no compute resources, especially HA router,"},{"line_number":48,"context_line":"  this means no real resources use router functionality. And sometimes"},{"line_number":49,"context_line":"  causes waste of IP resources (This can be overcomed by [1]_), but still be"},{"line_number":50,"context_line":"  a problem without such settings."}],"source_content_type":"text/x-rst","patch_set":5,"id":"bfb3d3c7_2ce29ba1","line":47,"range":{"start_line":47,"start_character":2,"end_line":47,"end_character":4},"in_reply_to":"bfb3d3c7_de2e0c42","updated":"2019-05-24 10:54:33.000000000","message":"Done","commit_id":"8669ab4ff25513f6ba2f25bcddd8e1f07655f2d9"},{"author":{"_account_id":4694,"name":"Miguel Lavalle","email":"miguel@mlavalle.com","username":"minsel"},"change_message_id":"ce31dbcc8524a5e5336a66dd125b557d0711a87c","unresolved":false,"context_lines":[{"line_number":46,"context_line":""},{"line_number":47,"context_line":"* e) L3 agent host a router for no compute resources, especially HA router,"},{"line_number":48,"context_line":"  this means no real resources use router functionality. And sometimes"},{"line_number":49,"context_line":"  causes waste of IP resources (This can be overcomed by [1]_), but still be"},{"line_number":50,"context_line":"  a problem without such settings."},{"line_number":51,"context_line":""},{"line_number":52,"context_line":""}],"source_content_type":"text/x-rst","patch_set":5,"id":"bfb3d3c7_3eec08f2","line":49,"range":{"start_line":49,"start_character":44,"end_line":49,"end_character":53},"updated":"2019-05-23 21:56:11.000000000","message":"overcome","commit_id":"8669ab4ff25513f6ba2f25bcddd8e1f07655f2d9"},{"author":{"_account_id":9531,"name":"liuyulong","display_name":"LIU Yulong","email":"i@liuyulong.me","username":"LIU-Yulong"},"change_message_id":"fe37fd0aad98453d472edfef8d6ff19df894870f","unresolved":false,"context_lines":[{"line_number":46,"context_line":""},{"line_number":47,"context_line":"* e) L3 agent host a router for no compute resources, especially HA router,"},{"line_number":48,"context_line":"  this means no real resources use router functionality. And sometimes"},{"line_number":49,"context_line":"  causes waste of IP resources (This can be overcomed by [1]_), but still be"},{"line_number":50,"context_line":"  a problem without such settings."},{"line_number":51,"context_line":""},{"line_number":52,"context_line":""}],"source_content_type":"text/x-rst","patch_set":5,"id":"bfb3d3c7_4cdfcfd5","line":49,"range":{"start_line":49,"start_character":44,"end_line":49,"end_character":53},"in_reply_to":"bfb3d3c7_3eec08f2","updated":"2019-05-24 10:54:33.000000000","message":"Done","commit_id":"8669ab4ff25513f6ba2f25bcddd8e1f07655f2d9"},{"author":{"_account_id":4694,"name":"Miguel Lavalle","email":"miguel@mlavalle.com","username":"minsel"},"change_message_id":"ce31dbcc8524a5e5336a66dd125b557d0711a87c","unresolved":false,"context_lines":[{"line_number":53,"context_line":"Proposed Change"},{"line_number":54,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"},{"line_number":55,"context_line":""},{"line_number":56,"context_line":"The proposal is generally for centralized SNAT nodes, for these node we"},{"line_number":57,"context_line":"can not say it have unlimited service capacity. From this point of view,"},{"line_number":58,"context_line":"we can refer to the bandwidth capacity of the server."},{"line_number":59,"context_line":""}],"source_content_type":"text/x-rst","patch_set":5,"id":"bfb3d3c7_7ecb6071","line":56,"range":{"start_line":56,"start_character":64,"end_line":56,"end_character":68},"updated":"2019-05-23 21:56:11.000000000","message":"nodes","commit_id":"8669ab4ff25513f6ba2f25bcddd8e1f07655f2d9"},{"author":{"_account_id":11975,"name":"Slawek Kaplonski","email":"skaplons@redhat.com","username":"slaweq"},"change_message_id":"5f3084346433084bcca7c2937ac80c972c80cc23","unresolved":false,"context_lines":[{"line_number":55,"context_line":""},{"line_number":56,"context_line":"The proposal is generally for centralized SNAT nodes, for these node we"},{"line_number":57,"context_line":"can not say it have unlimited service capacity. From this point of view,"},{"line_number":58,"context_line":"we can refer to the bandwidth capacity of the server."},{"line_number":59,"context_line":""},{"line_number":60,"context_line":"Server side changes"},{"line_number":61,"context_line":"-------------------"}],"source_content_type":"text/x-rst","patch_set":5,"id":"bfb3d3c7_dbea970d","line":58,"range":{"start_line":58,"start_character":20,"end_line":58,"end_character":52},"updated":"2019-05-24 07:24:48.000000000","message":"why bandwidth capacity? Above You are saying that problem for operators is that users creates many virtual routers which are doing nothing but using resources. And I can agree with that part. But is it bandwidth which is consumed most by such routers? I don\u0027t think so.","commit_id":"8669ab4ff25513f6ba2f25bcddd8e1f07655f2d9"},{"author":{"_account_id":9531,"name":"liuyulong","display_name":"LIU Yulong","email":"i@liuyulong.me","username":"LIU-Yulong"},"change_message_id":"268172d1e3cff769201242bdc328687ecaa78afc","unresolved":false,"context_lines":[{"line_number":55,"context_line":""},{"line_number":56,"context_line":"The proposal is generally for centralized SNAT nodes, for these node we"},{"line_number":57,"context_line":"can not say it have unlimited service capacity. From this point of view,"},{"line_number":58,"context_line":"we can refer to the bandwidth capacity of the server."},{"line_number":59,"context_line":""},{"line_number":60,"context_line":"Server side changes"},{"line_number":61,"context_line":"-------------------"}],"source_content_type":"text/x-rst","patch_set":5,"id":"bfb3d3c7_6cd17323","line":58,"range":{"start_line":58,"start_character":20,"end_line":58,"end_character":52},"in_reply_to":"bfb3d3c7_dbea970d","updated":"2019-05-24 10:15:53.000000000","message":"What will the networking resource consume in network node? It mainly is physical NIC. How to evaluate the NIC capability generally?  It is NIC defined bandwidth specification.\n\nSo how to caculate the router used/available bandwidth? Router centralized floating IP bandwidth QoS, or may be with gateway IP bandwidth QoS for centralized SNAT.\n\nWe will have config for such NIC bandwidth and ratios, this is very similar to VM cpu, doesn\u0027t the host process use the cpu? memory? Such bandwidth should also need prediction and planning.","commit_id":"8669ab4ff25513f6ba2f25bcddd8e1f07655f2d9"},{"author":{"_account_id":11975,"name":"Slawek Kaplonski","email":"skaplons@redhat.com","username":"slaweq"},"change_message_id":"5f3084346433084bcca7c2937ac80c972c80cc23","unresolved":false,"context_lines":[{"line_number":61,"context_line":"-------------------"},{"line_number":62,"context_line":""},{"line_number":63,"context_line":"* Add a ``bandwidth`` attribute for router which indicates the total"},{"line_number":64,"context_line":"  bandwidth of a router can use. This will be a new extension."},{"line_number":65,"context_line":""},{"line_number":66,"context_line":"* Add new OVO resource ``RouterBandwidth`` with relationship table"},{"line_number":67,"context_line":"  between ``Router`` and ``RouterBandwidth``."}],"source_content_type":"text/x-rst","patch_set":5,"id":"bfb3d3c7_dbd3775f","line":64,"updated":"2019-05-24 07:24:48.000000000","message":"bandwidth per physical network? or how? One node can have connectivity to many different networks and L3 agent don\u0027t have knowlege about mappings IIRC. How You want to solve this problem?","commit_id":"8669ab4ff25513f6ba2f25bcddd8e1f07655f2d9"},{"author":{"_account_id":9531,"name":"liuyulong","display_name":"LIU Yulong","email":"i@liuyulong.me","username":"LIU-Yulong"},"change_message_id":"268172d1e3cff769201242bdc328687ecaa78afc","unresolved":false,"context_lines":[{"line_number":61,"context_line":"-------------------"},{"line_number":62,"context_line":""},{"line_number":63,"context_line":"* Add a ``bandwidth`` attribute for router which indicates the total"},{"line_number":64,"context_line":"  bandwidth of a router can use. This will be a new extension."},{"line_number":65,"context_line":""},{"line_number":66,"context_line":"* Add new OVO resource ``RouterBandwidth`` with relationship table"},{"line_number":67,"context_line":"  between ``Router`` and ``RouterBandwidth``."}],"source_content_type":"text/x-rst","patch_set":5,"id":"bfb3d3c7_4c990f77","line":64,"in_reply_to":"bfb3d3c7_dbd3775f","updated":"2019-05-24 10:15:53.000000000","message":"Hmm, this is a nice question, maybe I should expand the \"Potential configurations\" with a map for such multiple external networks.","commit_id":"8669ab4ff25513f6ba2f25bcddd8e1f07655f2d9"},{"author":{"_account_id":11975,"name":"Slawek Kaplonski","email":"skaplons@redhat.com","username":"slaweq"},"change_message_id":"5f3084346433084bcca7c2937ac80c972c80cc23","unresolved":false,"context_lines":[{"line_number":66,"context_line":"* Add new OVO resource ``RouterBandwidth`` with relationship table"},{"line_number":67,"context_line":"  between ``Router`` and ``RouterBandwidth``."},{"line_number":68,"context_line":""},{"line_number":69,"context_line":"* L3 agent reports the total bandwidth with ratio and used bandwidth."},{"line_number":70,"context_line":""},{"line_number":71,"context_line":"* A new scheduler based on the request router bandwidth and agent"},{"line_number":72,"context_line":"  bandwidth usage."}],"source_content_type":"text/x-rst","patch_set":5,"id":"bfb3d3c7_1bbd8f01","line":69,"range":{"start_line":69,"start_character":1,"end_line":69,"end_character":69},"updated":"2019-05-24 07:24:48.000000000","message":"this looks for me like something what already placement service can do :)","commit_id":"8669ab4ff25513f6ba2f25bcddd8e1f07655f2d9"},{"author":{"_account_id":9531,"name":"liuyulong","display_name":"LIU Yulong","email":"i@liuyulong.me","username":"LIU-Yulong"},"change_message_id":"268172d1e3cff769201242bdc328687ecaa78afc","unresolved":false,"context_lines":[{"line_number":66,"context_line":"* Add new OVO resource ``RouterBandwidth`` with relationship table"},{"line_number":67,"context_line":"  between ``Router`` and ``RouterBandwidth``."},{"line_number":68,"context_line":""},{"line_number":69,"context_line":"* L3 agent reports the total bandwidth with ratio and used bandwidth."},{"line_number":70,"context_line":""},{"line_number":71,"context_line":"* A new scheduler based on the request router bandwidth and agent"},{"line_number":72,"context_line":"  bandwidth usage."}],"source_content_type":"text/x-rst","patch_set":5,"id":"bfb3d3c7_ac81abfd","line":69,"range":{"start_line":69,"start_character":1,"end_line":69,"end_character":69},"in_reply_to":"bfb3d3c7_1bbd8f01","updated":"2019-05-24 10:15:53.000000000","message":"Something similar, but we are L3 mercenary.","commit_id":"8669ab4ff25513f6ba2f25bcddd8e1f07655f2d9"},{"author":{"_account_id":4694,"name":"Miguel Lavalle","email":"miguel@mlavalle.com","username":"minsel"},"change_message_id":"ce31dbcc8524a5e5336a66dd125b557d0711a87c","unresolved":false,"context_lines":[{"line_number":74,"context_line":"Potential configurations"},{"line_number":75,"context_line":"------------------------"},{"line_number":76,"context_line":""},{"line_number":77,"context_line":"* ``bandwidth_ratio \u003d 1.0`` for l3-agent."},{"line_number":78,"context_line":""},{"line_number":79,"context_line":"* ``total_bandwidth \u003d 1000000`` kbps (minimum unit of QoS rules) for l3-agent."},{"line_number":80,"context_line":""}],"source_content_type":"text/x-rst","patch_set":5,"id":"bfb3d3c7_9e99746f","line":77,"updated":"2019-05-23 21:56:11.000000000","message":"Does this mean you can oversubscribe an agent if you set this value to say 2.0?","commit_id":"8669ab4ff25513f6ba2f25bcddd8e1f07655f2d9"},{"author":{"_account_id":9531,"name":"liuyulong","display_name":"LIU Yulong","email":"i@liuyulong.me","username":"LIU-Yulong"},"change_message_id":"c659201ca8b6f82736ee5b80e0eebea28210a570","unresolved":false,"context_lines":[{"line_number":74,"context_line":"Potential configurations"},{"line_number":75,"context_line":"------------------------"},{"line_number":76,"context_line":""},{"line_number":77,"context_line":"* ``bandwidth_ratio \u003d 1.0`` for l3-agent."},{"line_number":78,"context_line":""},{"line_number":79,"context_line":"* ``total_bandwidth \u003d 1000000`` kbps (minimum unit of QoS rules) for l3-agent."},{"line_number":80,"context_line":""}],"source_content_type":"text/x-rst","patch_set":5,"id":"bfb3d3c7_8f09402d","line":77,"in_reply_to":"bfb3d3c7_9e99746f","updated":"2019-05-24 01:32:06.000000000","message":"Yes, overcommit ratio mechanism is very common in the cloud.","commit_id":"8669ab4ff25513f6ba2f25bcddd8e1f07655f2d9"},{"author":{"_account_id":11975,"name":"Slawek Kaplonski","email":"skaplons@redhat.com","username":"slaweq"},"change_message_id":"5f3084346433084bcca7c2937ac80c972c80cc23","unresolved":false,"context_lines":[{"line_number":86,"context_line":""},{"line_number":87,"context_line":"Router"},{"line_number":88,"context_line":"++++++"},{"line_number":89,"context_line":"* 1. User create a router with specified or default bandwidth."},{"line_number":90,"context_line":"* 2. Scheduler router according to the bandwidth."},{"line_number":91,"context_line":"* 3. Available agent selected (successfully scheduled) return 200 to user."},{"line_number":92,"context_line":"* 4. Failed to schedule return 400 to user."}],"source_content_type":"text/x-rst","patch_set":5,"id":"bfb3d3c7_fb3d1b8f","line":89,"updated":"2019-05-24 07:24:48.000000000","message":"How You want to limit bandwidth for a router? Using existing QoS policies or in some different way?","commit_id":"8669ab4ff25513f6ba2f25bcddd8e1f07655f2d9"},{"author":{"_account_id":9531,"name":"liuyulong","display_name":"LIU Yulong","email":"i@liuyulong.me","username":"LIU-Yulong"},"change_message_id":"268172d1e3cff769201242bdc328687ecaa78afc","unresolved":false,"context_lines":[{"line_number":86,"context_line":""},{"line_number":87,"context_line":"Router"},{"line_number":88,"context_line":"++++++"},{"line_number":89,"context_line":"* 1. User create a router with specified or default bandwidth."},{"line_number":90,"context_line":"* 2. Scheduler router according to the bandwidth."},{"line_number":91,"context_line":"* 3. Available agent selected (successfully scheduled) return 200 to user."},{"line_number":92,"context_line":"* 4. Failed to schedule return 400 to user."}],"source_content_type":"text/x-rst","patch_set":5,"id":"bfb3d3c7_8c7ce71f","line":89,"in_reply_to":"bfb3d3c7_fb3d1b8f","updated":"2019-05-24 10:15:53.000000000","message":"Once more, router centralized floating IP bandwidth QoS, or may be with gateway IP bandwidth QoS for centralized SNAT, these traffic bandwidth will be considered for the router used/available bandwidth caculation. But this could be more likely another RFE, IMO.","commit_id":"8669ab4ff25513f6ba2f25bcddd8e1f07655f2d9"},{"author":{"_account_id":11975,"name":"Slawek Kaplonski","email":"skaplons@redhat.com","username":"slaweq"},"change_message_id":"432ed6e17b1f1e526c260945694554694c3bac5e","unresolved":false,"context_lines":[{"line_number":12,"context_line":""},{"line_number":13,"context_line":"For a large scale public cloud it meets scale issue about L3-agent. Most"},{"line_number":14,"context_line":"cloud service provider does not charge for the neutron virtual router."},{"line_number":15,"context_line":"This can become a headache for the operators. Every tenant may create"},{"line_number":16,"context_line":"free routers for doing nothing. But neutron will create many resource"},{"line_number":17,"context_line":"for it, especially the HA scenario, there will be namespaces, keepalived"},{"line_number":18,"context_line":"processes, and monitor processes. It will absolutely increase the failure"},{"line_number":19,"context_line":"risk, especially for agent restart. For cluster view of resource total"}],"source_content_type":"text/x-rst","patch_set":6,"id":"7faddb67_f61eba72","line":16,"range":{"start_line":15,"start_character":46,"end_line":16,"end_character":31},"updated":"2019-07-26 11:31:18.000000000","message":"One more thing. When I\u0027m reading this specs I have feeling that You are describing different problem here and You are trying to solve a bit different problem.\nIf real problem are routers \"which are doing nothing\" than resources like memory or cpu might be important also, not only bandwidth.\nAnd if this is really problem for You, maybe much simpler solution would be to change existing scheduler mechanism to do \"lazy\" scheduling of routers, only when it really have connected some interface. That way routers which are doing nothing will not create namespaces, haproxies and use other resources.\n\nIf however problem is with routers which are really consume bandwidth, than it\u0027s IMO more complicated problem to solve and we should check how nova are doing it together with placement and do something similar.","commit_id":"57863abe62267fdabb4db60a04568e493fe64563"},{"author":{"_account_id":9531,"name":"liuyulong","display_name":"LIU Yulong","email":"i@liuyulong.me","username":"LIU-Yulong"},"change_message_id":"efaa6bccf710787a6732a0084bea875e5fcfcc6c","unresolved":false,"context_lines":[{"line_number":12,"context_line":""},{"line_number":13,"context_line":"For a large scale public cloud it meets scale issue about L3-agent. Most"},{"line_number":14,"context_line":"cloud service provider does not charge for the neutron virtual router."},{"line_number":15,"context_line":"This can become a headache for the operators. Every tenant may create"},{"line_number":16,"context_line":"free routers for doing nothing. But neutron will create many resource"},{"line_number":17,"context_line":"for it, especially the HA scenario, there will be namespaces, keepalived"},{"line_number":18,"context_line":"processes, and monitor processes. It will absolutely increase the failure"},{"line_number":19,"context_line":"risk, especially for agent restart. For cluster view of resource total"}],"source_content_type":"text/x-rst","patch_set":6,"id":"e8683ce6_3b92f632","line":16,"range":{"start_line":15,"start_character":46,"end_line":16,"end_character":31},"in_reply_to":"7faddb67_f61eba72","updated":"2020-11-26 07:56:07.000000000","message":"Oh, sorry for the late reply. Yes, the lazy schedule and the capacity are different problem. No matter when the scheduling happens, the physical hosts are always has its capacity. This spec is aim to limit the total router amount/bandwidth of a network node. We do not consider the scheduling opportunity, but give the current neutron a chance to limit the running router number on a network node.\n\nThe main concern of this spec is to aviod that \"your network node has too many routers to work as usual. It can not burden more scheduled routers. Otherwise, the running resources of your network may face the risk of link broken or HA state change.\"\n\nSo this spec will provider two ways to describe the capacity: one is the total number of routers for a single network node you insist;  another is the physical bandwidth of a network node.","commit_id":"57863abe62267fdabb4db60a04568e493fe64563"},{"author":{"_account_id":11975,"name":"Slawek Kaplonski","email":"skaplons@redhat.com","username":"slaweq"},"change_message_id":"1338444d595837f34889d6bb7ac86eacbf8fa07b","unresolved":false,"context_lines":[{"line_number":37,"context_line":"* Current ``LeastRoutersScheduler`` can not work, since we do not have a"},{"line_number":38,"context_line":"  maximum quantity for l3-agent. So router still is free to schedule to any"},{"line_number":39,"context_line":"  l3-agent. We have no method to limit the number of routers that grow"},{"line_number":40,"context_line":"  indefinitely as tenants grow."},{"line_number":41,"context_line":""},{"line_number":42,"context_line":"* For the virtual machine (nova instance), we basically have CPU and memory"},{"line_number":43,"context_line":"  to estimate capacity. But for the centralized SNAT nodes, we have no such"}],"source_content_type":"text/x-rst","patch_set":6,"id":"7faddb67_94dc3419","line":40,"updated":"2019-07-25 09:19:03.000000000","message":"You can use some external tools and add new network node if current node will reach some capacity configured by You.","commit_id":"57863abe62267fdabb4db60a04568e493fe64563"},{"author":{"_account_id":11975,"name":"Slawek Kaplonski","email":"skaplons@redhat.com","username":"slaweq"},"change_message_id":"53a329d9eea6c2e39d1cbe6ceb6d6d2d898b41e5","unresolved":false,"context_lines":[{"line_number":37,"context_line":"* Current ``LeastRoutersScheduler`` can not work, since we do not have a"},{"line_number":38,"context_line":"  maximum quantity for l3-agent. So router still is free to schedule to any"},{"line_number":39,"context_line":"  l3-agent. We have no method to limit the number of routers that grow"},{"line_number":40,"context_line":"  indefinitely as tenants grow."},{"line_number":41,"context_line":""},{"line_number":42,"context_line":"* For the virtual machine (nova instance), we basically have CPU and memory"},{"line_number":43,"context_line":"  to estimate capacity. But for the centralized SNAT nodes, we have no such"}],"source_content_type":"text/x-rst","patch_set":6,"id":"ff570b3c_f2dc9e19","line":40,"in_reply_to":"7faddb67_15459b43","updated":"2020-11-23 11:28:36.000000000","message":"But with Your solution new routers will end up with error \"No valid host found\" which Nova is famous of already. Is that really better for end user?","commit_id":"57863abe62267fdabb4db60a04568e493fe64563"},{"author":{"_account_id":9531,"name":"liuyulong","display_name":"LIU Yulong","email":"i@liuyulong.me","username":"LIU-Yulong"},"change_message_id":"d08bca35f5c544ee3360c6da03ba04960a9d8fbd","unresolved":false,"context_lines":[{"line_number":37,"context_line":"* Current ``LeastRoutersScheduler`` can not work, since we do not have a"},{"line_number":38,"context_line":"  maximum quantity for l3-agent. So router still is free to schedule to any"},{"line_number":39,"context_line":"  l3-agent. We have no method to limit the number of routers that grow"},{"line_number":40,"context_line":"  indefinitely as tenants grow."},{"line_number":41,"context_line":""},{"line_number":42,"context_line":"* For the virtual machine (nova instance), we basically have CPU and memory"},{"line_number":43,"context_line":"  to estimate capacity. But for the centralized SNAT nodes, we have no such"}],"source_content_type":"text/x-rst","patch_set":6,"id":"7faddb67_15459b43","line":40,"in_reply_to":"7faddb67_94dc3419","updated":"2019-07-25 15:06:27.000000000","message":"For instance, what external tool? Did you mean some monitor daemon? Yes, it can be one way you get the router number of the agent, but you still do not prevent the unlimited router growth.\nFor that \u0027new network node\u0027 thing, in some cloud life cycle, it usually takes a lot of manpower and time to put a new batch of machines on the shelf. In a standardized process, this usually takes a long time, including some hardware provisioning and testing work etc. When you get the alarm and accomplish the new machines schedule, the router number may have exceeded the threshold.","commit_id":"57863abe62267fdabb4db60a04568e493fe64563"},{"author":{"_account_id":11975,"name":"Slawek Kaplonski","email":"skaplons@redhat.com","username":"slaweq"},"change_message_id":"84eb6d29233ff53ab787d92c09257758db14d79e","unresolved":true,"context_lines":[{"line_number":37,"context_line":"* Current ``LeastRoutersScheduler`` can not work, since we do not have a"},{"line_number":38,"context_line":"  maximum quantity for l3-agent. So router still is free to schedule to any"},{"line_number":39,"context_line":"  l3-agent. We have no method to limit the number of routers that grow"},{"line_number":40,"context_line":"  indefinitely as tenants grow."},{"line_number":41,"context_line":""},{"line_number":42,"context_line":"* For the virtual machine (nova instance), we basically have CPU and memory"},{"line_number":43,"context_line":"  to estimate capacity. But for the centralized SNAT nodes, we have no such"}],"source_content_type":"text/x-rst","patch_set":6,"id":"8fcc9d34_b378be8d","line":40,"in_reply_to":"dc88b7b7_137756a8","updated":"2020-12-08 08:55:10.000000000","message":"The argument with many ports and overloaded ovs-agent and/or L3 agent is totally valid and I agree with it.","commit_id":"57863abe62267fdabb4db60a04568e493fe64563"},{"author":{"_account_id":9531,"name":"liuyulong","display_name":"LIU Yulong","email":"i@liuyulong.me","username":"LIU-Yulong"},"change_message_id":"efaa6bccf710787a6732a0084bea875e5fcfcc6c","unresolved":false,"context_lines":[{"line_number":37,"context_line":"* Current ``LeastRoutersScheduler`` can not work, since we do not have a"},{"line_number":38,"context_line":"  maximum quantity for l3-agent. So router still is free to schedule to any"},{"line_number":39,"context_line":"  l3-agent. We have no method to limit the number of routers that grow"},{"line_number":40,"context_line":"  indefinitely as tenants grow."},{"line_number":41,"context_line":""},{"line_number":42,"context_line":"* For the virtual machine (nova instance), we basically have CPU and memory"},{"line_number":43,"context_line":"  to estimate capacity. But for the centralized SNAT nodes, we have no such"}],"source_content_type":"text/x-rst","patch_set":6,"id":"dc88b7b7_137756a8","line":40,"in_reply_to":"ff570b3c_f2dc9e19","updated":"2020-11-26 07:56:07.000000000","message":"IMHO, API \u0027fast failed\u0027 is better than created but your system is down, then the resource is not really uesable. New created routers\u0027 amount growth can sometimes hurt the running ones. Router related ports are continuously pluged into the ovs-bridge, there will be too many ports on ovs-bridge to process/work properly. You can create 400 router to a network node, then you may see your L3-agent and ovs-agent have no response, or possibly dead.","commit_id":"57863abe62267fdabb4db60a04568e493fe64563"},{"author":{"_account_id":11975,"name":"Slawek Kaplonski","email":"skaplons@redhat.com","username":"slaweq"},"change_message_id":"1338444d595837f34889d6bb7ac86eacbf8fa07b","unresolved":false,"context_lines":[{"line_number":41,"context_line":""},{"line_number":42,"context_line":"* For the virtual machine (nova instance), we basically have CPU and memory"},{"line_number":43,"context_line":"  to estimate capacity. But for the centralized SNAT nodes, we have no such"},{"line_number":44,"context_line":"  calculation."},{"line_number":45,"context_line":""},{"line_number":46,"context_line":"* Large scale l3 resources increase operation difficulties."},{"line_number":47,"context_line":""}],"source_content_type":"text/x-rst","patch_set":6,"id":"7faddb67_7402d871","line":44,"updated":"2019-07-25 09:19:03.000000000","message":"And for vm it is easy as You know how many cores/ram/disk instance will use in the moment when You want to schedule it. For router it is harder as You don\u0027t have this information during router creation and scheduling process.","commit_id":"57863abe62267fdabb4db60a04568e493fe64563"},{"author":{"_account_id":9531,"name":"liuyulong","display_name":"LIU Yulong","email":"i@liuyulong.me","username":"LIU-Yulong"},"change_message_id":"d08bca35f5c544ee3360c6da03ba04960a9d8fbd","unresolved":false,"context_lines":[{"line_number":41,"context_line":""},{"line_number":42,"context_line":"* For the virtual machine (nova instance), we basically have CPU and memory"},{"line_number":43,"context_line":"  to estimate capacity. But for the centralized SNAT nodes, we have no such"},{"line_number":44,"context_line":"  calculation."},{"line_number":45,"context_line":""},{"line_number":46,"context_line":"* Large scale l3 resources increase operation difficulties."},{"line_number":47,"context_line":""}],"source_content_type":"text/x-rst","patch_set":6,"id":"7faddb67_c3f2ec6f","line":44,"in_reply_to":"7faddb67_7402d871","updated":"2019-07-25 15:06:27.000000000","message":"This proposal is adding that information. A scheduler is a very basic routine for hosting resources based on some capacity.","commit_id":"57863abe62267fdabb4db60a04568e493fe64563"},{"author":{"_account_id":11975,"name":"Slawek Kaplonski","email":"skaplons@redhat.com","username":"slaweq"},"change_message_id":"1338444d595837f34889d6bb7ac86eacbf8fa07b","unresolved":false,"context_lines":[{"line_number":58,"context_line":"can not say it have unlimited service capacity. From this point of view,"},{"line_number":59,"context_line":"we can refer to the bandwidth capacity of the server. Because the networking"},{"line_number":60,"context_line":"resources mainly consume the physical NIC in network node, and NIC has its"},{"line_number":61,"context_line":"defined specification, aka bandwidth. So before building a robust large cloud"},{"line_number":62,"context_line":"platform, such bandwidth should also need prediction and planning."},{"line_number":63,"context_line":""},{"line_number":64,"context_line":"Server side changes"}],"source_content_type":"text/x-rst","patch_set":6,"id":"7faddb67_3439c03e","line":61,"updated":"2019-07-25 09:19:03.000000000","message":"This is IMO not true according to Your description from L13-L20. You said there that problem is if users are creating routers which are doing nothing but consume resources for namespace/keepalived process/monitor process. Those can consume memory/number of opened files/cpu but if router is not doing anythig it will not consume bandwidth so why You want to base scheduling on bandwidth?","commit_id":"57863abe62267fdabb4db60a04568e493fe64563"},{"author":{"_account_id":9531,"name":"liuyulong","display_name":"LIU Yulong","email":"i@liuyulong.me","username":"LIU-Yulong"},"change_message_id":"4b0b30535b67de3d3716b6f20a90654501505d3e","unresolved":false,"context_lines":[{"line_number":58,"context_line":"can not say it have unlimited service capacity. From this point of view,"},{"line_number":59,"context_line":"we can refer to the bandwidth capacity of the server. Because the networking"},{"line_number":60,"context_line":"resources mainly consume the physical NIC in network node, and NIC has its"},{"line_number":61,"context_line":"defined specification, aka bandwidth. So before building a robust large cloud"},{"line_number":62,"context_line":"platform, such bandwidth should also need prediction and planning."},{"line_number":63,"context_line":""},{"line_number":64,"context_line":"Server side changes"}],"source_content_type":"text/x-rst","patch_set":6,"id":"7faddb67_1fd051c9","line":61,"in_reply_to":"7faddb67_2bae36e7","updated":"2019-07-26 04:36:19.000000000","message":"No, neutron virtual routers now are no conscious of memory or CPU.","commit_id":"57863abe62267fdabb4db60a04568e493fe64563"},{"author":{"_account_id":9531,"name":"liuyulong","display_name":"LIU Yulong","email":"i@liuyulong.me","username":"LIU-Yulong"},"change_message_id":"d08bca35f5c544ee3360c6da03ba04960a9d8fbd","unresolved":false,"context_lines":[{"line_number":58,"context_line":"can not say it have unlimited service capacity. From this point of view,"},{"line_number":59,"context_line":"we can refer to the bandwidth capacity of the server. Because the networking"},{"line_number":60,"context_line":"resources mainly consume the physical NIC in network node, and NIC has its"},{"line_number":61,"context_line":"defined specification, aka bandwidth. So before building a robust large cloud"},{"line_number":62,"context_line":"platform, such bandwidth should also need prediction and planning."},{"line_number":63,"context_line":""},{"line_number":64,"context_line":"Server side changes"}],"source_content_type":"text/x-rst","patch_set":6,"id":"7faddb67_75f2cf08","line":61,"in_reply_to":"7faddb67_3439c03e","updated":"2019-07-25 15:06:27.000000000","message":"I wrote this here in line 60:\n\"\"\"\nBecause the networking\nresources mainly consume the physical NIC in network node\n\"\"\"","commit_id":"57863abe62267fdabb4db60a04568e493fe64563"},{"author":{"_account_id":11975,"name":"Slawek Kaplonski","email":"skaplons@redhat.com","username":"slaweq"},"change_message_id":"02e15a8057c5fb4ff5daffe07db90edc21c8b083","unresolved":false,"context_lines":[{"line_number":58,"context_line":"can not say it have unlimited service capacity. From this point of view,"},{"line_number":59,"context_line":"we can refer to the bandwidth capacity of the server. Because the networking"},{"line_number":60,"context_line":"resources mainly consume the physical NIC in network node, and NIC has its"},{"line_number":61,"context_line":"defined specification, aka bandwidth. So before building a robust large cloud"},{"line_number":62,"context_line":"platform, such bandwidth should also need prediction and planning."},{"line_number":63,"context_line":""},{"line_number":64,"context_line":"Server side changes"}],"source_content_type":"text/x-rst","patch_set":6,"id":"7faddb67_2bae36e7","line":61,"in_reply_to":"7faddb67_75f2cf08","updated":"2019-07-25 16:14:44.000000000","message":"But You also wrote:\n\n\"Every tenant may create\nfree routers for doing nothing. But neutron will create many resource\nfor it, especially the HA scenario, there will be namespaces, keepalived\nprocesses, and monitor processes.\"\n\nAnd IIUC descrition of Your problem which You are tryin to solve those routers are main issue, no?\n\nAlso if we will take a look at physical routers - they also have CPU and memory and those are very important resources for them. So You shouldn\u0027t limit Your proposal only to bandwidth.","commit_id":"57863abe62267fdabb4db60a04568e493fe64563"},{"author":{"_account_id":11975,"name":"Slawek Kaplonski","email":"skaplons@redhat.com","username":"slaweq"},"change_message_id":"1338444d595837f34889d6bb7ac86eacbf8fa07b","unresolved":false,"context_lines":[{"line_number":65,"context_line":"-------------------"},{"line_number":66,"context_line":""},{"line_number":67,"context_line":"* Add a ``bandwidth`` attribute for router which indicates the total"},{"line_number":68,"context_line":"  bandwidth of a router can use. This will be a new extension."},{"line_number":69,"context_line":""},{"line_number":70,"context_line":"* Add new OVO resource ``RouterBandwidth`` with relationship table"},{"line_number":71,"context_line":"  between ``Router`` and ``RouterBandwidth``."}],"source_content_type":"text/x-rst","patch_set":6,"id":"7faddb67_8772d210","line":68,"updated":"2019-07-25 09:19:03.000000000","message":"there is already bandwidth in qos and it can be associated with any of router\u0027s ports. Which bandwidth You would want to have here?\n\nAlso what this new \"bandwidth\" would really mean? Max total bandwidth on all interfaces? max bandwidth on each interface? max bandwidth on external gateway interface?","commit_id":"57863abe62267fdabb4db60a04568e493fe64563"},{"author":{"_account_id":11975,"name":"Slawek Kaplonski","email":"skaplons@redhat.com","username":"slaweq"},"change_message_id":"432ed6e17b1f1e526c260945694554694c3bac5e","unresolved":false,"context_lines":[{"line_number":65,"context_line":"-------------------"},{"line_number":66,"context_line":""},{"line_number":67,"context_line":"* Add a ``bandwidth`` attribute for router which indicates the total"},{"line_number":68,"context_line":"  bandwidth of a router can use. This will be a new extension."},{"line_number":69,"context_line":""},{"line_number":70,"context_line":"* Add new OVO resource ``RouterBandwidth`` with relationship table"},{"line_number":71,"context_line":"  between ``Router`` and ``RouterBandwidth``."}],"source_content_type":"text/x-rst","patch_set":6,"id":"7faddb67_76c08ae1","line":68,"in_reply_to":"7faddb67_029234a0","updated":"2019-07-26 11:31:18.000000000","message":"I\u0027m not saying that we shouldn\u0027t do it. But we should do it in proper way, not in the fastest way ;)","commit_id":"57863abe62267fdabb4db60a04568e493fe64563"},{"author":{"_account_id":9531,"name":"liuyulong","display_name":"LIU Yulong","email":"i@liuyulong.me","username":"LIU-Yulong"},"change_message_id":"d08bca35f5c544ee3360c6da03ba04960a9d8fbd","unresolved":false,"context_lines":[{"line_number":65,"context_line":"-------------------"},{"line_number":66,"context_line":""},{"line_number":67,"context_line":"* Add a ``bandwidth`` attribute for router which indicates the total"},{"line_number":68,"context_line":"  bandwidth of a router can use. This will be a new extension."},{"line_number":69,"context_line":""},{"line_number":70,"context_line":"* Add new OVO resource ``RouterBandwidth`` with relationship table"},{"line_number":71,"context_line":"  between ``Router`` and ``RouterBandwidth``."}],"source_content_type":"text/x-rst","patch_set":6,"id":"7faddb67_b561273c","line":68,"in_reply_to":"7faddb67_8772d210","updated":"2019-07-25 15:06:27.000000000","message":"It\u0027s not a QoS, it is a specification of one router external bandwidth capacity, for more detail, it is the router\u0027s restricted maximum centralized NAT bandwidth. For now it will be used only for schedule.\n\nAnd I have one extra idea is to limit the floating IP creation under such router with \u0027total bandwidth\u0027. We can add all the floating IP bandwidth and gateway IP bandwidth. Then compare if the new floating IP will let the total exceed the specification.","commit_id":"57863abe62267fdabb4db60a04568e493fe64563"},{"author":{"_account_id":9531,"name":"liuyulong","display_name":"LIU Yulong","email":"i@liuyulong.me","username":"LIU-Yulong"},"change_message_id":"4b0b30535b67de3d3716b6f20a90654501505d3e","unresolved":false,"context_lines":[{"line_number":65,"context_line":"-------------------"},{"line_number":66,"context_line":""},{"line_number":67,"context_line":"* Add a ``bandwidth`` attribute for router which indicates the total"},{"line_number":68,"context_line":"  bandwidth of a router can use. This will be a new extension."},{"line_number":69,"context_line":""},{"line_number":70,"context_line":"* Add new OVO resource ``RouterBandwidth`` with relationship table"},{"line_number":71,"context_line":"  between ``Router`` and ``RouterBandwidth``."}],"source_content_type":"text/x-rst","patch_set":6,"id":"7faddb67_029234a0","line":68,"in_reply_to":"7faddb67_8bf5eaef","updated":"2019-07-26 04:36:19.000000000","message":"L3 does not count the on the \u0027port\u0027 QoS. Router now has floating IP QoS and gateway IP QoS.\n\nLike I said, neutron router scheduler does not involve some resources else. Nova uses placement, IMO, is because the instances consume tons of resources which are not easily managed by nova itself, like volume, port bandwidth, PCI-devices, etc.\n`Router` is a main neutron resource, it should have a robust scheduling mechanism. And it is not out of networking scope, neutron should handle it. And, for now, it can be achieved.","commit_id":"57863abe62267fdabb4db60a04568e493fe64563"},{"author":{"_account_id":11975,"name":"Slawek Kaplonski","email":"skaplons@redhat.com","username":"slaweq"},"change_message_id":"02e15a8057c5fb4ff5daffe07db90edc21c8b083","unresolved":false,"context_lines":[{"line_number":65,"context_line":"-------------------"},{"line_number":66,"context_line":""},{"line_number":67,"context_line":"* Add a ``bandwidth`` attribute for router which indicates the total"},{"line_number":68,"context_line":"  bandwidth of a router can use. This will be a new extension."},{"line_number":69,"context_line":""},{"line_number":70,"context_line":"* Add new OVO resource ``RouterBandwidth`` with relationship table"},{"line_number":71,"context_line":"  between ``Router`` and ``RouterBandwidth``."}],"source_content_type":"text/x-rst","patch_set":6,"id":"7faddb67_8bf5eaef","line":68,"in_reply_to":"7faddb67_b561273c","updated":"2019-07-25 16:14:44.000000000","message":"So what is QoS policy with max bw limit associated with gateway port? Isn\u0027t it describing the same thing in fact?\n\nAbout Your second idea: again it\u0027s something what we already solved for instances and guarantee bw limit using cooperation with placement :)","commit_id":"57863abe62267fdabb4db60a04568e493fe64563"},{"author":{"_account_id":11975,"name":"Slawek Kaplonski","email":"skaplons@redhat.com","username":"slaweq"},"change_message_id":"84eb6d29233ff53ab787d92c09257758db14d79e","unresolved":true,"context_lines":[{"line_number":67,"context_line":"* Add a ``bandwidth`` attribute for router which indicates the total"},{"line_number":68,"context_line":"  bandwidth of a router can use. This will be a new extension."},{"line_number":69,"context_line":""},{"line_number":70,"context_line":"* Add new OVO resource ``RouterBandwidth`` with relationship table"},{"line_number":71,"context_line":"  between ``Router`` and ``RouterBandwidth``."},{"line_number":72,"context_line":""},{"line_number":73,"context_line":"* L3 agent reports the total bandwidth with ratio and used bandwidth."}],"source_content_type":"text/x-rst","patch_set":6,"id":"5dec58f6_0a8742d9","line":70,"range":{"start_line":70,"start_character":25,"end_line":70,"end_character":40},"updated":"2020-12-08 08:55:10.000000000","message":"Sorry but I still don\u0027t really understand the concept of \"RouterBandwidth\" - router has many ports connected to different networks. Some of them are connected to some provider networks, some may be connected to tunnel networks.\nPorts connected to the provider networks may use different physical networks so may have totally different bandwidth available.\nSo what exactly this \"RouterBandwidth\" attribute would represent?","commit_id":"57863abe62267fdabb4db60a04568e493fe64563"},{"author":{"_account_id":9531,"name":"liuyulong","display_name":"LIU Yulong","email":"i@liuyulong.me","username":"LIU-Yulong"},"change_message_id":"110b3fb13600f41d2b53f601e717d63667b20cb2","unresolved":true,"context_lines":[{"line_number":67,"context_line":"* Add a ``bandwidth`` attribute for router which indicates the total"},{"line_number":68,"context_line":"  bandwidth of a router can use. This will be a new extension."},{"line_number":69,"context_line":""},{"line_number":70,"context_line":"* Add new OVO resource ``RouterBandwidth`` with relationship table"},{"line_number":71,"context_line":"  between ``Router`` and ``RouterBandwidth``."},{"line_number":72,"context_line":""},{"line_number":73,"context_line":"* L3 agent reports the total bandwidth with ratio and used bandwidth."}],"source_content_type":"text/x-rst","patch_set":6,"id":"9df55468_5b97a626","line":70,"range":{"start_line":70,"start_character":25,"end_line":70,"end_character":40},"in_reply_to":"5dec58f6_0a8742d9","updated":"2020-12-10 11:13:04.000000000","message":"Please see my comments in Line 71 to your former question. You can consider the L3 Router is now a container, it has a max bandwidth capacity, this \"RouterBandwidth\" stands for.\nThe sum of all L3 IPs\u0027 (under one router) bandwidth should not exceed the Router\u0027s (total) Bandwidth.","commit_id":"57863abe62267fdabb4db60a04568e493fe64563"},{"author":{"_account_id":11975,"name":"Slawek Kaplonski","email":"skaplons@redhat.com","username":"slaweq"},"change_message_id":"1338444d595837f34889d6bb7ac86eacbf8fa07b","unresolved":false,"context_lines":[{"line_number":68,"context_line":"  bandwidth of a router can use. This will be a new extension."},{"line_number":69,"context_line":""},{"line_number":70,"context_line":"* Add new OVO resource ``RouterBandwidth`` with relationship table"},{"line_number":71,"context_line":"  between ``Router`` and ``RouterBandwidth``."},{"line_number":72,"context_line":""},{"line_number":73,"context_line":"* L3 agent reports the total bandwidth with ratio and used bandwidth."},{"line_number":74,"context_line":""}],"source_content_type":"text/x-rst","patch_set":6,"id":"7faddb67_1427e414","line":71,"updated":"2019-07-25 09:19:03.000000000","message":"This is another \"implementation\" of QoS IMO. Why You don\u0027t want to reuse what we already have?","commit_id":"57863abe62267fdabb4db60a04568e493fe64563"},{"author":{"_account_id":9531,"name":"liuyulong","display_name":"LIU Yulong","email":"i@liuyulong.me","username":"LIU-Yulong"},"change_message_id":"d08bca35f5c544ee3360c6da03ba04960a9d8fbd","unresolved":false,"context_lines":[{"line_number":68,"context_line":"  bandwidth of a router can use. This will be a new extension."},{"line_number":69,"context_line":""},{"line_number":70,"context_line":"* Add new OVO resource ``RouterBandwidth`` with relationship table"},{"line_number":71,"context_line":"  between ``Router`` and ``RouterBandwidth``."},{"line_number":72,"context_line":""},{"line_number":73,"context_line":"* L3 agent reports the total bandwidth with ratio and used bandwidth."},{"line_number":74,"context_line":""}],"source_content_type":"text/x-rst","patch_set":6,"id":"7faddb67_15c03b2a","line":71,"in_reply_to":"7faddb67_1427e414","updated":"2019-07-25 15:06:27.000000000","message":"It\u0027s not a QoS, it is a specification of one router bandwidth capacity.","commit_id":"57863abe62267fdabb4db60a04568e493fe64563"},{"author":{"_account_id":11975,"name":"Slawek Kaplonski","email":"skaplons@redhat.com","username":"slaweq"},"change_message_id":"1338444d595837f34889d6bb7ac86eacbf8fa07b","unresolved":false,"context_lines":[{"line_number":70,"context_line":"* Add new OVO resource ``RouterBandwidth`` with relationship table"},{"line_number":71,"context_line":"  between ``Router`` and ``RouterBandwidth``."},{"line_number":72,"context_line":""},{"line_number":73,"context_line":"* L3 agent reports the total bandwidth with ratio and used bandwidth."},{"line_number":74,"context_line":""},{"line_number":75,"context_line":"* A new scheduler based on the request router bandwidth and agent"},{"line_number":76,"context_line":"  bandwidth usage."}],"source_content_type":"text/x-rst","patch_set":6,"id":"7faddb67_07bca267","line":73,"updated":"2019-07-25 09:19:03.000000000","message":"total ratio per physical network? or summary? Also please note that currently L3 router don\u0027t have information about bridge mappings and physical networks - it\u0027s matter of L2 agent.\nAnd such agent already have possibility to report this bandwidth \"per physical network\" to placement service. Why not use this information for scheduling?","commit_id":"57863abe62267fdabb4db60a04568e493fe64563"},{"author":{"_account_id":9531,"name":"liuyulong","display_name":"LIU Yulong","email":"i@liuyulong.me","username":"LIU-Yulong"},"change_message_id":"d08bca35f5c544ee3360c6da03ba04960a9d8fbd","unresolved":false,"context_lines":[{"line_number":70,"context_line":"* Add new OVO resource ``RouterBandwidth`` with relationship table"},{"line_number":71,"context_line":"  between ``Router`` and ``RouterBandwidth``."},{"line_number":72,"context_line":""},{"line_number":73,"context_line":"* L3 agent reports the total bandwidth with ratio and used bandwidth."},{"line_number":74,"context_line":""},{"line_number":75,"context_line":"* A new scheduler based on the request router bandwidth and agent"},{"line_number":76,"context_line":"  bandwidth usage."}],"source_content_type":"text/x-rst","patch_set":6,"id":"7faddb67_151cbbbf","line":73,"in_reply_to":"7faddb67_07bca267","updated":"2019-07-25 15:06:27.000000000","message":"\u003e total ratio per physical network? or summary? Also please note that\n\nboth can be a potential solution\n\n \u003e currently L3 router don\u0027t have information about bridge mappings\n \u003e and physical networks - it\u0027s matter of L2 agent.\n\nand we are going to add this\n\n \u003e And such agent already have possibility to report this bandwidth\n \u003e \"per physical network\" to placement service. Why not use this\n \u003e information for scheduling?\n\nI\u0027m not very keen on placement, it increase the system complexity, and neutron can do this work by itself, since we will not involve some resource else.","commit_id":"57863abe62267fdabb4db60a04568e493fe64563"},{"author":{"_account_id":11975,"name":"Slawek Kaplonski","email":"skaplons@redhat.com","username":"slaweq"},"change_message_id":"02e15a8057c5fb4ff5daffe07db90edc21c8b083","unresolved":false,"context_lines":[{"line_number":70,"context_line":"* Add new OVO resource ``RouterBandwidth`` with relationship table"},{"line_number":71,"context_line":"  between ``Router`` and ``RouterBandwidth``."},{"line_number":72,"context_line":""},{"line_number":73,"context_line":"* L3 agent reports the total bandwidth with ratio and used bandwidth."},{"line_number":74,"context_line":""},{"line_number":75,"context_line":"* A new scheduler based on the request router bandwidth and agent"},{"line_number":76,"context_line":"  bandwidth usage."}],"source_content_type":"text/x-rst","patch_set":6,"id":"7faddb67_eb533ee1","line":73,"in_reply_to":"7faddb67_151cbbbf","updated":"2019-07-25 16:14:44.000000000","message":"\u003e \u003e total ratio per physical network? or summary? Also please note\n \u003e that\n \u003e \n \u003e both can be a potential solution\n \u003e \n \u003e \u003e currently L3 router don\u0027t have information about bridge mappings\n \u003e \u003e and physical networks - it\u0027s matter of L2 agent.\n \u003e \n \u003e and we are going to add this\n\nBut this isn\u0027t something what L3 agent should know IMO. It\u0027s L2 agent\u0027s thing.\n\n \u003e \n \u003e \u003e And such agent already have possibility to report this bandwidth\n \u003e \u003e \"per physical network\" to placement service. Why not use this\n \u003e \u003e information for scheduling?\n \u003e \n \u003e I\u0027m not very keen on placement, it increase the system complexity,\n \u003e and neutron can do this work by itself, since we will not involve\n \u003e some resource else.\n\nNeutron can do many things if we will implement them but the question is: what is actually neutron and what is its goal? Should it do everything? I\u0027m really not sure about that.","commit_id":"57863abe62267fdabb4db60a04568e493fe64563"},{"author":{"_account_id":11975,"name":"Slawek Kaplonski","email":"skaplons@redhat.com","username":"slaweq"},"change_message_id":"84eb6d29233ff53ab787d92c09257758db14d79e","unresolved":true,"context_lines":[{"line_number":70,"context_line":"* Add new OVO resource ``RouterBandwidth`` with relationship table"},{"line_number":71,"context_line":"  between ``Router`` and ``RouterBandwidth``."},{"line_number":72,"context_line":""},{"line_number":73,"context_line":"* L3 agent reports the total bandwidth with ratio and used bandwidth."},{"line_number":74,"context_line":""},{"line_number":75,"context_line":"* A new scheduler based on the request router bandwidth and agent"},{"line_number":76,"context_line":"  bandwidth usage."}],"source_content_type":"text/x-rst","patch_set":6,"id":"d77e7895_eb1ec79c","line":73,"in_reply_to":"7faddb67_5fbd6966","updated":"2020-12-08 08:55:10.000000000","message":"personally I\u0027m still not fan of mixing L2 concepts (like bridge mappings) into L3 agent. But I would like to know opinions of others on that.","commit_id":"57863abe62267fdabb4db60a04568e493fe64563"},{"author":{"_account_id":9531,"name":"liuyulong","display_name":"LIU Yulong","email":"i@liuyulong.me","username":"LIU-Yulong"},"change_message_id":"4b0b30535b67de3d3716b6f20a90654501505d3e","unresolved":false,"context_lines":[{"line_number":70,"context_line":"* Add new OVO resource ``RouterBandwidth`` with relationship table"},{"line_number":71,"context_line":"  between ``Router`` and ``RouterBandwidth``."},{"line_number":72,"context_line":""},{"line_number":73,"context_line":"* L3 agent reports the total bandwidth with ratio and used bandwidth."},{"line_number":74,"context_line":""},{"line_number":75,"context_line":"* A new scheduler based on the request router bandwidth and agent"},{"line_number":76,"context_line":"  bandwidth usage."}],"source_content_type":"text/x-rst","patch_set":6,"id":"7faddb67_5fbd6966","line":73,"in_reply_to":"7faddb67_eb533ee1","updated":"2019-07-26 04:36:19.000000000","message":"\u003e \u003e \u003e total ratio per physical network? or summary? Also please note\n \u003e \u003e that\n \u003e \u003e\n \u003e \u003e both can be a potential solution\n \u003e \u003e\n \u003e \u003e \u003e currently L3 router don\u0027t have information about bridge\n \u003e mappings\n \u003e \u003e \u003e and physical networks - it\u0027s matter of L2 agent.\n \u003e \u003e\n \u003e \u003e and we are going to add this\n \u003e \n \u003e But this isn\u0027t something what L3 agent should know IMO. It\u0027s L2\n\nYou may consider that as the L3 agent is consuming the physical L3 bandwidth. : )\nIn other perpective, neutron L2 and L3 are now a bit coupled. L3 agent has many configuration like: external_network_bridge (removed now, but existed once), interface_driver which indicate the L2 things. L2 agent has some configuration like: enable_distributed_routing which is for L3.\n\n \u003e agent\u0027s thing.\n \u003e \n \u003e \u003e\n \u003e \u003e \u003e And such agent already have possibility to report this\n \u003e bandwidth\n \u003e \u003e \u003e \"per physical network\" to placement service. Why not use this\n \u003e \u003e \u003e information for scheduling?\n \u003e \u003e\n \u003e \u003e I\u0027m not very keen on placement, it increase the system\n \u003e complexity,\n \u003e \u003e and neutron can do this work by itself, since we will not involve\n \u003e \u003e some resource else.\n \u003e \n \u003e Neutron can do many things if we will implement them but the\n \u003e question is: what is actually neutron and what is its goal? Should\n \u003e it do everything? I\u0027m really not sure about that.\n\nLike I said, neutron router scheduler does not involve some resources else. Nova uses placement, IMO, is because the instances consume tons of resources which are not easily managed by nova itself, like volume, port bandwidth, PCI-devices, etc.\n`Router` is a main neutron resource, it should have a robust scheduling mechanism. And it is not out of networking scope, neutron should handle it. And, for now, it can be achieved.","commit_id":"57863abe62267fdabb4db60a04568e493fe64563"},{"author":{"_account_id":9531,"name":"liuyulong","display_name":"LIU Yulong","email":"i@liuyulong.me","username":"LIU-Yulong"},"change_message_id":"110b3fb13600f41d2b53f601e717d63667b20cb2","unresolved":true,"context_lines":[{"line_number":70,"context_line":"* Add new OVO resource ``RouterBandwidth`` with relationship table"},{"line_number":71,"context_line":"  between ``Router`` and ``RouterBandwidth``."},{"line_number":72,"context_line":""},{"line_number":73,"context_line":"* L3 agent reports the total bandwidth with ratio and used bandwidth."},{"line_number":74,"context_line":""},{"line_number":75,"context_line":"* A new scheduler based on the request router bandwidth and agent"},{"line_number":76,"context_line":"  bandwidth usage."}],"source_content_type":"text/x-rst","patch_set":6,"id":"2a8780d7_6bea35f7","line":73,"in_reply_to":"d77e7895_eb1ec79c","updated":"2020-12-10 11:13:04.000000000","message":"Again, all these bandwidth capacity or scheduling methods are for L3 IPs. And the bandwidth is L3 IP\u0027s bandwidth.","commit_id":"57863abe62267fdabb4db60a04568e493fe64563"},{"author":{"_account_id":11975,"name":"Slawek Kaplonski","email":"skaplons@redhat.com","username":"slaweq"},"change_message_id":"1338444d595837f34889d6bb7ac86eacbf8fa07b","unresolved":false,"context_lines":[{"line_number":73,"context_line":"* L3 agent reports the total bandwidth with ratio and used bandwidth."},{"line_number":74,"context_line":""},{"line_number":75,"context_line":"* A new scheduler based on the request router bandwidth and agent"},{"line_number":76,"context_line":"  bandwidth usage."},{"line_number":77,"context_line":""},{"line_number":78,"context_line":"Potential configurations"},{"line_number":79,"context_line":"------------------------"}],"source_content_type":"text/x-rst","patch_set":6,"id":"7faddb67_87a73250","line":76,"updated":"2019-07-25 09:19:03.000000000","message":"How You may know during new router creation to which physical networks it will be connected, and from which physical interface it will consume bandwidth?","commit_id":"57863abe62267fdabb4db60a04568e493fe64563"},{"author":{"_account_id":11975,"name":"Slawek Kaplonski","email":"skaplons@redhat.com","username":"slaweq"},"change_message_id":"02e15a8057c5fb4ff5daffe07db90edc21c8b083","unresolved":false,"context_lines":[{"line_number":73,"context_line":"* L3 agent reports the total bandwidth with ratio and used bandwidth."},{"line_number":74,"context_line":""},{"line_number":75,"context_line":"* A new scheduler based on the request router bandwidth and agent"},{"line_number":76,"context_line":"  bandwidth usage."},{"line_number":77,"context_line":""},{"line_number":78,"context_line":"Potential configurations"},{"line_number":79,"context_line":"------------------------"}],"source_content_type":"text/x-rst","patch_set":6,"id":"7faddb67_6b274e66","line":76,"in_reply_to":"7faddb67_551c7312","updated":"2019-07-25 16:14:44.000000000","message":"\u003e One solution is when the router is not collected to external\n \u003e network, it will not schedule to any one. When it try to connect to\n \u003e the external network, schedule it. Yes, it may get failed. But this\n \u003e will be my another proposal.\n\nAnd what about routers which have only internal ports? How You will place them? And where?\n\n \u003e Another is considering the network node as one entirety, no matter\n \u003e the NICs, interfaces or networks. Just schedule one has the free\n \u003e capacity.\n\nFree capacity where? On which NIC?\nI really have feeling that what You are trying to do is simply express maximum number of routers per agent but in \"strange\" way, like:\n\n    router \u003d 1000 kbps\n    agent_capacity \u003d 100000 kbps\n\n    number_of_routers_on_agent \u003d agent_capacity / router \u003d 100\n\nSo why You don\u0027t simply propose new scheduler driver which will have limited number of routers which can be scheduled on agent? Without all this bandwidth stuff.\n\n \u003e I have no more thought of a better way for this now. I will\n \u003e appreciate if you can give us some.","commit_id":"57863abe62267fdabb4db60a04568e493fe64563"},{"author":{"_account_id":9531,"name":"liuyulong","display_name":"LIU Yulong","email":"i@liuyulong.me","username":"LIU-Yulong"},"change_message_id":"4b0b30535b67de3d3716b6f20a90654501505d3e","unresolved":false,"context_lines":[{"line_number":73,"context_line":"* L3 agent reports the total bandwidth with ratio and used bandwidth."},{"line_number":74,"context_line":""},{"line_number":75,"context_line":"* A new scheduler based on the request router bandwidth and agent"},{"line_number":76,"context_line":"  bandwidth usage."},{"line_number":77,"context_line":""},{"line_number":78,"context_line":"Potential configurations"},{"line_number":79,"context_line":"------------------------"}],"source_content_type":"text/x-rst","patch_set":6,"id":"7faddb67_22b5f0ee","line":76,"in_reply_to":"7faddb67_6b274e66","updated":"2019-07-26 04:36:19.000000000","message":"\u003e \u003e One solution is when the router is not collected to external\n \u003e \u003e network, it will not schedule to any one. When it try to connect\n \u003e to\n \u003e \u003e the external network, schedule it. Yes, it may get failed. But\n \u003e this\n \u003e \u003e will be my another proposal.\n \u003e \n \u003e And what about routers which have only internal ports? How You will\n \u003e place them? And where?\n\nFor DVR router, the compute local router can do the east west traffic. This can be a problem for none-dvr router. Maybe split the NAT traffic and east-west traffic can be a potential solution for none-dvr. Schedule only when the NAT traffic is needed.\n\n \u003e \n \u003e \u003e Another is considering the network node as one entirety, no\n \u003e matter\n \u003e \u003e the NICs, interfaces or networks. Just schedule one has the free\n \u003e \u003e capacity.\n \u003e \n \u003e Free capacity where? On which NIC?\n \u003e I really have feeling that what You are trying to do is simply\n \u003e express maximum number of routers per agent but in \"strange\" way,\n \u003e like:\n \u003e \n \u003e router \u003d 1000 kbps\n \u003e agent_capacity \u003d 100000 kbps\n \u003e \n \u003e number_of_routers_on_agent \u003d agent_capacity / router \u003d 100\n \u003e \n \u003e So why You don\u0027t simply propose new scheduler driver which will\n \u003e have limited number of routers which can be scheduled on agent?\n \u003e Without all this bandwidth stuff.\n\nYes, it will look like that, a simple number calculation.\nBut it is aiming to the main resource that router will consume. \n\n \u003e \n \u003e \u003e I have no more thought of a better way for this now. I will\n \u003e \u003e appreciate if you can give us some.","commit_id":"57863abe62267fdabb4db60a04568e493fe64563"},{"author":{"_account_id":9531,"name":"liuyulong","display_name":"LIU Yulong","email":"i@liuyulong.me","username":"LIU-Yulong"},"change_message_id":"d08bca35f5c544ee3360c6da03ba04960a9d8fbd","unresolved":false,"context_lines":[{"line_number":73,"context_line":"* L3 agent reports the total bandwidth with ratio and used bandwidth."},{"line_number":74,"context_line":""},{"line_number":75,"context_line":"* A new scheduler based on the request router bandwidth and agent"},{"line_number":76,"context_line":"  bandwidth usage."},{"line_number":77,"context_line":""},{"line_number":78,"context_line":"Potential configurations"},{"line_number":79,"context_line":"------------------------"}],"source_content_type":"text/x-rst","patch_set":6,"id":"7faddb67_551c7312","line":76,"in_reply_to":"7faddb67_87a73250","updated":"2019-07-25 15:06:27.000000000","message":"One solution is when the router is not collected to external network, it will not schedule to any one. When it try to connect to the external network, schedule it. Yes, it may get failed. But this will be my another proposal.\nAnother is considering the network node as one entirety, no matter the NICs, interfaces or networks. Just schedule one has the free capacity.\nI have no more thought of a better way for this now. I will appreciate if you can give us some.","commit_id":"57863abe62267fdabb4db60a04568e493fe64563"},{"author":{"_account_id":11975,"name":"Slawek Kaplonski","email":"skaplons@redhat.com","username":"slaweq"},"change_message_id":"1338444d595837f34889d6bb7ac86eacbf8fa07b","unresolved":false,"context_lines":[{"line_number":80,"context_line":""},{"line_number":81,"context_line":"For neutron server side:"},{"line_number":82,"context_line":""},{"line_number":83,"context_line":"* ``router_default_bandwidth \u003d 1000000`` kbps for router API."},{"line_number":84,"context_line":""},{"line_number":85,"context_line":"In agent side due to multile external networks, the following list config will"},{"line_number":86,"context_line":"be needed:"}],"source_content_type":"text/x-rst","patch_set":6,"id":"7faddb67_34678054","line":83,"updated":"2019-07-25 09:19:03.000000000","message":"Why not use QoS policies for that?","commit_id":"57863abe62267fdabb4db60a04568e493fe64563"},{"author":{"_account_id":9531,"name":"liuyulong","display_name":"LIU Yulong","email":"i@liuyulong.me","username":"LIU-Yulong"},"change_message_id":"d08bca35f5c544ee3360c6da03ba04960a9d8fbd","unresolved":false,"context_lines":[{"line_number":80,"context_line":""},{"line_number":81,"context_line":"For neutron server side:"},{"line_number":82,"context_line":""},{"line_number":83,"context_line":"* ``router_default_bandwidth \u003d 1000000`` kbps for router API."},{"line_number":84,"context_line":""},{"line_number":85,"context_line":"In agent side due to multile external networks, the following list config will"},{"line_number":86,"context_line":"be needed:"}],"source_content_type":"text/x-rst","patch_set":6,"id":"7faddb67_b53607b4","line":83,"in_reply_to":"7faddb67_34678054","updated":"2019-07-25 15:06:27.000000000","message":"You can assume it as a nominal specification. Something like the NIC bandwidth defined specification.","commit_id":"57863abe62267fdabb4db60a04568e493fe64563"},{"author":{"_account_id":11975,"name":"Slawek Kaplonski","email":"skaplons@redhat.com","username":"slaweq"},"change_message_id":"02e15a8057c5fb4ff5daffe07db90edc21c8b083","unresolved":false,"context_lines":[{"line_number":80,"context_line":""},{"line_number":81,"context_line":"For neutron server side:"},{"line_number":82,"context_line":""},{"line_number":83,"context_line":"* ``router_default_bandwidth \u003d 1000000`` kbps for router API."},{"line_number":84,"context_line":""},{"line_number":85,"context_line":"In agent side due to multile external networks, the following list config will"},{"line_number":86,"context_line":"be needed:"}],"source_content_type":"text/x-rst","patch_set":6,"id":"7faddb67_eb1a5e15","line":83,"in_reply_to":"7faddb67_b53607b4","updated":"2019-07-25 16:14:44.000000000","message":"I will repeat my question: which NIC? there can be many of them on one host :)","commit_id":"57863abe62267fdabb4db60a04568e493fe64563"},{"author":{"_account_id":9531,"name":"liuyulong","display_name":"LIU Yulong","email":"i@liuyulong.me","username":"LIU-Yulong"},"change_message_id":"4b0b30535b67de3d3716b6f20a90654501505d3e","unresolved":false,"context_lines":[{"line_number":80,"context_line":""},{"line_number":81,"context_line":"For neutron server side:"},{"line_number":82,"context_line":""},{"line_number":83,"context_line":"* ``router_default_bandwidth \u003d 1000000`` kbps for router API."},{"line_number":84,"context_line":""},{"line_number":85,"context_line":"In agent side due to multile external networks, the following list config will"},{"line_number":86,"context_line":"be needed:"}],"source_content_type":"text/x-rst","patch_set":6,"id":"7faddb67_9f068120","line":83,"in_reply_to":"7faddb67_eb1a5e15","updated":"2019-07-26 04:36:19.000000000","message":"The \u0027NIC\u0027 here is not for router to connect. It is just an example something like this:\n\"\"\"\n# ethtool em1\nSettings for em1:\n\t...\n\tAdvertised FEC modes: Not reported\n\tSpeed: 10000Mb/s\n\"\"\"\nThe \u0027bandwidth\u0027 attribute for neutron router is like that \"Speed\" for NIC.","commit_id":"57863abe62267fdabb4db60a04568e493fe64563"},{"author":{"_account_id":11975,"name":"Slawek Kaplonski","email":"skaplons@redhat.com","username":"slaweq"},"change_message_id":"1338444d595837f34889d6bb7ac86eacbf8fa07b","unresolved":false,"context_lines":[{"line_number":93,"context_line":"     \"total_bandwidth\": 1000000},  # kbps of one NIC for routers to use"},{"line_number":94,"context_line":"    {\"interface\": \"eth1\","},{"line_number":95,"context_line":"     \"bandwidth_ratio\": 1.5,"},{"line_number":96,"context_line":"     \"total_bandwidth\": 1000000}"},{"line_number":97,"context_line":"  ]"},{"line_number":98,"context_line":""},{"line_number":99,"context_line":"The neutron basic workflow"}],"source_content_type":"text/x-rst","patch_set":6,"id":"7faddb67_f207c617","line":96,"updated":"2019-07-25 09:19:03.000000000","message":"This is already done in L2 agents, see https://github.com/openstack/neutron/blob/master/neutron/plugins/ml2/drivers/openvswitch/agent/ovs_neutron_agent.py#L198","commit_id":"57863abe62267fdabb4db60a04568e493fe64563"},{"author":{"_account_id":9531,"name":"liuyulong","display_name":"LIU Yulong","email":"i@liuyulong.me","username":"LIU-Yulong"},"change_message_id":"4b0b30535b67de3d3716b6f20a90654501505d3e","unresolved":false,"context_lines":[{"line_number":93,"context_line":"     \"total_bandwidth\": 1000000},  # kbps of one NIC for routers to use"},{"line_number":94,"context_line":"    {\"interface\": \"eth1\","},{"line_number":95,"context_line":"     \"bandwidth_ratio\": 1.5,"},{"line_number":96,"context_line":"     \"total_bandwidth\": 1000000}"},{"line_number":97,"context_line":"  ]"},{"line_number":98,"context_line":""},{"line_number":99,"context_line":"The neutron basic workflow"}],"source_content_type":"text/x-rst","patch_set":6,"id":"7faddb67_e23a78cb","line":96,"in_reply_to":"7faddb67_0beb3adc","updated":"2019-07-26 04:36:19.000000000","message":"\u003e One more thing related to this: how You want to map interface to\n \u003e physical_network? In ovs agent we have bridge mappings for that,\n\nYes, for now it will be in such way.\n\n \u003e here how You exactly want to do it? And why we should duplicate\n \u003e this with L2 agent?\n\nYou may consider it is a bandwidth for L3, or datagram transimit rate. For neutron router, we can assume it is \u0027NAT\u0027 rate.","commit_id":"57863abe62267fdabb4db60a04568e493fe64563"},{"author":{"_account_id":11975,"name":"Slawek Kaplonski","email":"skaplons@redhat.com","username":"slaweq"},"change_message_id":"02e15a8057c5fb4ff5daffe07db90edc21c8b083","unresolved":false,"context_lines":[{"line_number":93,"context_line":"     \"total_bandwidth\": 1000000},  # kbps of one NIC for routers to use"},{"line_number":94,"context_line":"    {\"interface\": \"eth1\","},{"line_number":95,"context_line":"     \"bandwidth_ratio\": 1.5,"},{"line_number":96,"context_line":"     \"total_bandwidth\": 1000000}"},{"line_number":97,"context_line":"  ]"},{"line_number":98,"context_line":""},{"line_number":99,"context_line":"The neutron basic workflow"}],"source_content_type":"text/x-rst","patch_set":6,"id":"7faddb67_0beb3adc","line":96,"in_reply_to":"7faddb67_15d59be5","updated":"2019-07-25 16:14:44.000000000","message":"One more thing related to this: how You want to map interface to physical_network? In ovs agent we have bridge mappings for that, here how You exactly want to do it? And why we should duplicate this with L2 agent?","commit_id":"57863abe62267fdabb4db60a04568e493fe64563"},{"author":{"_account_id":9531,"name":"liuyulong","display_name":"LIU Yulong","email":"i@liuyulong.me","username":"LIU-Yulong"},"change_message_id":"d08bca35f5c544ee3360c6da03ba04960a9d8fbd","unresolved":false,"context_lines":[{"line_number":93,"context_line":"     \"total_bandwidth\": 1000000},  # kbps of one NIC for routers to use"},{"line_number":94,"context_line":"    {\"interface\": \"eth1\","},{"line_number":95,"context_line":"     \"bandwidth_ratio\": 1.5,"},{"line_number":96,"context_line":"     \"total_bandwidth\": 1000000}"},{"line_number":97,"context_line":"  ]"},{"line_number":98,"context_line":""},{"line_number":99,"context_line":"The neutron basic workflow"}],"source_content_type":"text/x-rst","patch_set":6,"id":"7faddb67_15d59be5","line":96,"in_reply_to":"7faddb67_f207c617","updated":"2019-07-25 15:06:27.000000000","message":"Thanks, I know it.","commit_id":"57863abe62267fdabb4db60a04568e493fe64563"}],"specs/ussuri/l3-agent-capacity.rst":[{"author":{"_account_id":6854,"name":"YAMAMOTO Takashi","email":"yamamoto@midokura.com","username":"yamamoto"},"change_message_id":"6f12bf7f3c724b94e2b915c21d3032a54f3943d9","unresolved":false,"context_lines":[{"line_number":70,"context_line":"Server side changes"},{"line_number":71,"context_line":"-------------------"},{"line_number":72,"context_line":""},{"line_number":73,"context_line":"* Add a ``bandwidth`` attribute for router which indicates the total"},{"line_number":74,"context_line":"  bandwidth of a router can use. This will be a new extension."},{"line_number":75,"context_line":""},{"line_number":76,"context_line":"* Add new OVO resource ``RouterBandwidth`` with relationship table"},{"line_number":77,"context_line":"  between ``Router`` and ``RouterBandwidth``."}],"source_content_type":"text/x-rst","patch_set":7,"id":"3fa7e38b_94173754","line":74,"range":{"start_line":73,"start_character":59,"end_line":74,"end_character":31},"updated":"2020-01-17 15:05:09.000000000","message":"does this attribute mean to allow api users to give estimates as a hint for scheduling?\nor, will there be some qos mechanism in the datapath to enforce this?","commit_id":"f331230cb817ec476eda1cd188ee06b052220e32"},{"author":{"_account_id":9531,"name":"liuyulong","display_name":"LIU Yulong","email":"i@liuyulong.me","username":"LIU-Yulong"},"change_message_id":"e5aa6f82e57b4c8a90c52f2ffff137228dd15dd6","unresolved":false,"context_lines":[{"line_number":70,"context_line":"Server side changes"},{"line_number":71,"context_line":"-------------------"},{"line_number":72,"context_line":""},{"line_number":73,"context_line":"* Add a ``bandwidth`` attribute for router which indicates the total"},{"line_number":74,"context_line":"  bandwidth of a router can use. This will be a new extension."},{"line_number":75,"context_line":""},{"line_number":76,"context_line":"* Add new OVO resource ``RouterBandwidth`` with relationship table"},{"line_number":77,"context_line":"  between ``Router`` and ``RouterBandwidth``."}],"source_content_type":"text/x-rst","patch_set":7,"id":"ff570b3c_ecf456f3","line":74,"range":{"start_line":73,"start_character":59,"end_line":74,"end_character":31},"in_reply_to":"3fa7e38b_94173754","updated":"2020-05-20 15:11:37.000000000","message":"It is the hints for scheduler. No QoS introduced is in such approach. But the floating IPs and gateway IPs bandwidth will be considered to caculate the bandwidth usage of the router.","commit_id":"f331230cb817ec476eda1cd188ee06b052220e32"},{"author":{"_account_id":6854,"name":"YAMAMOTO Takashi","email":"yamamoto@midokura.com","username":"yamamoto"},"change_message_id":"6f12bf7f3c724b94e2b915c21d3032a54f3943d9","unresolved":false,"context_lines":[{"line_number":78,"context_line":""},{"line_number":79,"context_line":"* L3 agent reports the total bandwidth with ratio and used bandwidth."},{"line_number":80,"context_line":""},{"line_number":81,"context_line":"* A new scheduler based on the request router bandwidth and agent"},{"line_number":82,"context_line":"  bandwidth usage."},{"line_number":83,"context_line":""},{"line_number":84,"context_line":"Potential configurations"},{"line_number":85,"context_line":"------------------------"}],"source_content_type":"text/x-rst","patch_set":7,"id":"3fa7e38b_9449778e","line":82,"range":{"start_line":81,"start_character":60,"end_line":82,"end_character":17},"updated":"2020-01-17 15:05:09.000000000","message":"does this mean a measured value on datapath? or just the sum of scheduled routers?","commit_id":"f331230cb817ec476eda1cd188ee06b052220e32"},{"author":{"_account_id":9531,"name":"liuyulong","display_name":"LIU Yulong","email":"i@liuyulong.me","username":"LIU-Yulong"},"change_message_id":"e5aa6f82e57b4c8a90c52f2ffff137228dd15dd6","unresolved":false,"context_lines":[{"line_number":78,"context_line":""},{"line_number":79,"context_line":"* L3 agent reports the total bandwidth with ratio and used bandwidth."},{"line_number":80,"context_line":""},{"line_number":81,"context_line":"* A new scheduler based on the request router bandwidth and agent"},{"line_number":82,"context_line":"  bandwidth usage."},{"line_number":83,"context_line":""},{"line_number":84,"context_line":"Potential configurations"},{"line_number":85,"context_line":"------------------------"}],"source_content_type":"text/x-rst","patch_set":7,"id":"ff570b3c_ccd4f24a","line":82,"range":{"start_line":81,"start_character":60,"end_line":82,"end_character":17},"in_reply_to":"3fa7e38b_9449778e","updated":"2020-05-20 15:11:37.000000000","message":"The later, scheduled routers total bandwidht.","commit_id":"f331230cb817ec476eda1cd188ee06b052220e32"},{"author":{"_account_id":6854,"name":"YAMAMOTO Takashi","email":"yamamoto@midokura.com","username":"yamamoto"},"change_message_id":"6f12bf7f3c724b94e2b915c21d3032a54f3943d9","unresolved":false,"context_lines":[{"line_number":86,"context_line":""},{"line_number":87,"context_line":"For neutron server side:"},{"line_number":88,"context_line":""},{"line_number":89,"context_line":"* ``router_default_bandwidth \u003d 1000000`` kbps for router API."},{"line_number":90,"context_line":""},{"line_number":91,"context_line":"In agent side due to multile external networks, the following list config will"},{"line_number":92,"context_line":"be needed:"}],"source_content_type":"text/x-rst","patch_set":7,"id":"3fa7e38b_9460970e","line":89,"range":{"start_line":89,"start_character":0,"end_line":89,"end_character":61},"updated":"2020-01-17 15:05:09.000000000","message":"do you mean neutron.conf?","commit_id":"f331230cb817ec476eda1cd188ee06b052220e32"},{"author":{"_account_id":9531,"name":"liuyulong","display_name":"LIU Yulong","email":"i@liuyulong.me","username":"LIU-Yulong"},"change_message_id":"e5aa6f82e57b4c8a90c52f2ffff137228dd15dd6","unresolved":false,"context_lines":[{"line_number":86,"context_line":""},{"line_number":87,"context_line":"For neutron server side:"},{"line_number":88,"context_line":""},{"line_number":89,"context_line":"* ``router_default_bandwidth \u003d 1000000`` kbps for router API."},{"line_number":90,"context_line":""},{"line_number":91,"context_line":"In agent side due to multile external networks, the following list config will"},{"line_number":92,"context_line":"be needed:"}],"source_content_type":"text/x-rst","patch_set":7,"id":"ff570b3c_6cb96613","line":89,"range":{"start_line":89,"start_character":0,"end_line":89,"end_character":61},"in_reply_to":"3fa7e38b_9460970e","updated":"2020-05-20 15:11:37.000000000","message":"Yes, for neutron server side the L3 plugin only uses the neutron.conf.","commit_id":"f331230cb817ec476eda1cd188ee06b052220e32"}],"specs/wallaby/l3-agent-capacity.rst":[{"author":{"_account_id":11975,"name":"Slawek Kaplonski","email":"skaplons@redhat.com","username":"slaweq"},"change_message_id":"d59c0b7968d1ec44ccafc22c95adeac5fcb62fef","unresolved":true,"context_lines":[{"line_number":71,"context_line":"-------------------"},{"line_number":72,"context_line":""},{"line_number":73,"context_line":"* Add a ``bandwidth`` attribute for router which indicates the total"},{"line_number":74,"context_line":"  bandwidth of a router can use. This will be a new extension."},{"line_number":75,"context_line":""},{"line_number":76,"context_line":"* Add new OVO resource ``RouterBandwidth`` with relationship table"},{"line_number":77,"context_line":"  between ``Router`` and ``RouterBandwidth``."}],"source_content_type":"text/x-rst","patch_set":9,"id":"f15149eb_e179ce00","line":74,"updated":"2021-02-03 08:31:28.000000000","message":"this is not what was decided and approved in the drivers meeting. Please check https://bugs.launchpad.net/neutron/+bug/1828494/comments/10 and update this spec accordingly.","commit_id":"fe379ee6bf1226946dddc575ffbc6b1781e0a39a"},{"author":{"_account_id":11975,"name":"Slawek Kaplonski","email":"skaplons@redhat.com","username":"slaweq"},"change_message_id":"e0cea0b2684f86cfad7f2e4c5d776440b33c19b8","unresolved":true,"context_lines":[{"line_number":71,"context_line":"-------------------"},{"line_number":72,"context_line":""},{"line_number":73,"context_line":"* Add a ``bandwidth`` attribute for router which indicates the total"},{"line_number":74,"context_line":"  bandwidth of a router can use. This will be a new extension."},{"line_number":75,"context_line":""},{"line_number":76,"context_line":"* Add new OVO resource ``RouterBandwidth`` with relationship table"},{"line_number":77,"context_line":"  between ``Router`` and ``RouterBandwidth``."}],"source_content_type":"text/x-rst","patch_set":9,"id":"1b020222_29bb9341","line":74,"in_reply_to":"c05cb3d2_ef21f634","updated":"2021-04-26 09:18:36.000000000","message":"From Your comment it looks like You want to have much more complete \"bandwidth aware scheduling\" of the routers. In such case I think this spec should be changed and You should consider usage of the placement API for that as this is the goal of that project.\nAlso You should consider the cases where router can use different physical network so will consume totally different bandwidth than e.g. other routers.\nThere is also problem that currently L3 agent\u0027s aren\u0027t aware about bridge mappings and phys nets at all and they should be probably with such feature.\nThat\u0027s why we decided to go with much simpler \"max_routers\" value for the scheduler as it wouldn\u0027t require all those things mentioned by me above.","commit_id":"fe379ee6bf1226946dddc575ffbc6b1781e0a39a"},{"author":{"_account_id":9531,"name":"liuyulong","display_name":"LIU Yulong","email":"i@liuyulong.me","username":"LIU-Yulong"},"change_message_id":"2240ee875e01b5e64d85218c708cefa412e44a51","unresolved":true,"context_lines":[{"line_number":71,"context_line":"-------------------"},{"line_number":72,"context_line":""},{"line_number":73,"context_line":"* Add a ``bandwidth`` attribute for router which indicates the total"},{"line_number":74,"context_line":"  bandwidth of a router can use. This will be a new extension."},{"line_number":75,"context_line":""},{"line_number":76,"context_line":"* Add new OVO resource ``RouterBandwidth`` with relationship table"},{"line_number":77,"context_line":"  between ``Router`` and ``RouterBandwidth``."}],"source_content_type":"text/x-rst","patch_set":9,"id":"c05cb3d2_ef21f634","line":74,"in_reply_to":"f15149eb_e179ce00","updated":"2021-02-03 08:59:10.000000000","message":"Please take a look at line 129 [Alternatives], that should be what you guys approved. But IMO, that \"routers_max\" looks a bit rigid for a large scale cloud deployment. While one agent has only one router, the floating IP\u0027s total bandwidth under it may have exceed the maximum of the phyical NIC. More bad thing is that more routers with more floating IPs can still be hosted on this same host. Maybe someday you will raise another attribute \"floatingips_max\" for router. But the problem is basically same, one floating IP\u0027s bandwidth can be set higher. When the number of routers is less than \"routers_max\", and each router\u0027s floating IP number is less than \"floatingips_max\", the total bandwidth of all floating IPs in this host still exceeds physical device‘s spec.\n\nFor this proposal here, it contains three dimensions of bandwidth:\n1. the physical NIC\u0027s bandwidth\n2. the total bandwidth a neutron Router sould serve (here this line proposed)\n3. the floating IPs total bandwidth under one Router\n\nSo, things will be clear:\n1. how many router can a host run is based on the total bandwidth of routers\n2. how many floating IPs can a router serve is based on the router\u0027s bandwidth attribute (here this line proposed)\n\nSo, maybe we can reconsider these problems at the meeting.","commit_id":"fe379ee6bf1226946dddc575ffbc6b1781e0a39a"},{"author":{"_account_id":11975,"name":"Slawek Kaplonski","email":"skaplons@redhat.com","username":"slaweq"},"change_message_id":"e0cea0b2684f86cfad7f2e4c5d776440b33c19b8","unresolved":true,"context_lines":[{"line_number":99,"context_line":"     \"total_bandwidth\": 1000000},  # kbps of one NIC for routers to use"},{"line_number":100,"context_line":"    {\"interface\": \"eth1\","},{"line_number":101,"context_line":"     \"bandwidth_ratio\": 1.5,"},{"line_number":102,"context_line":"     \"total_bandwidth\": 1000000}"},{"line_number":103,"context_line":"  ]"},{"line_number":104,"context_line":""},{"line_number":105,"context_line":"The neutron basic workflow"}],"source_content_type":"text/x-rst","patch_set":9,"id":"331c9f1b_6cbac41f","line":102,"updated":"2021-04-26 09:18:36.000000000","message":"we are mixing here things from L2 and L3 agent. IMO L3 agent shouldn\u0027t be aware about bridge mappings and bandwidth on each phys interface. L2 agent is already able to report that to placement so we should reuse it if that would be really needed.","commit_id":"fe379ee6bf1226946dddc575ffbc6b1781e0a39a"}]}