)]}'
{"specs/mitaka/approved/libvirt-emulator-threads-policy.rst":[{"author":{"_account_id":7730,"name":"Sahid Orentino Ferdjaoui","email":"sahid.ferdjaoui@industrialdiscipline.com","username":"sahid"},"change_message_id":"ad1835909242f9d9f06b05ad58a124921a1d8cd5","unresolved":false,"context_lines":[{"line_number":82,"context_line":"virtual instance having zero or more \"system psuedo-CPUs\". For"},{"line_number":83,"context_line":"example, we might say that all the various QEMU threads account"},{"line_number":84,"context_line":"for 1 system psuedo-CPU, while a Xen stub-domain\u0027s 2 vCPUs, may"},{"line_number":85,"context_line":"correspond to the 2 system psuedo-CPUs. Since the permitted vCPU"},{"line_number":86,"context_line":"allocation is recorded against the flavour, it is natural to use"},{"line_number":87,"context_line":"a flavour extra spec to express the number of system psuedo-CPUs."},{"line_number":88,"context_line":"So the new property"}],"source_content_type":"text/x-rst","patch_set":1,"id":"ba15a1d1_9974d09c","line":85,"updated":"2015-09-22 15:42:45.000000000","message":"s/psuedo/pseudo","commit_id":"86f6574d253cf7a3639cedc0c5dcb8e30b42dfdf"},{"author":{"_account_id":7730,"name":"Sahid Orentino Ferdjaoui","email":"sahid.ferdjaoui@industrialdiscipline.com","username":"sahid"},"change_message_id":"ad1835909242f9d9f06b05ad58a124921a1d8cd5","unresolved":false,"context_lines":[{"line_number":110,"context_line":"* cpu_overcommit\u003d1.5"},{"line_number":111,"context_line":""},{"line_number":112,"context_line":"The filter will look for a host with free CPU resource equivalent"},{"line_number":113,"context_line":"to 15, instead of 12."},{"line_number":114,"context_line":""},{"line_number":115,"context_line":"The NUMA schedular filter will also need to be changed to account"},{"line_number":116,"context_line":"for the extra threads, which will in turn include changing the"}],"source_content_type":"text/x-rst","patch_set":1,"id":"ba15a1d1_c6e027cf","line":113,"updated":"2015-09-22 15:42:45.000000000","message":"This example is not clear for me.\n\nI cannot see how the flavor is asking for 15 CPUs, for what I have understand; It requests 10 (8+2) so if host have 1.5 overcommit we can \"consider\" it requests for 7 CPUs. Actually it more like it requests for 10 CPUs and we look at host with N CPUs * cpu_overcommit \u003e\u003d 10.","commit_id":"86f6574d253cf7a3639cedc0c5dcb8e30b42dfdf"},{"author":{"_account_id":1779,"name":"Daniel Berrange","email":"berrange@redhat.com","username":"berrange"},"change_message_id":"a49a4545d43659f7a47f1db7db10a3d1782b0433","unresolved":false,"context_lines":[{"line_number":110,"context_line":"* cpu_overcommit\u003d1.5"},{"line_number":111,"context_line":""},{"line_number":112,"context_line":"The filter will look for a host with free CPU resource equivalent"},{"line_number":113,"context_line":"to 15, instead of 12."},{"line_number":114,"context_line":""},{"line_number":115,"context_line":"The NUMA schedular filter will also need to be changed to account"},{"line_number":116,"context_line":"for the extra threads, which will in turn include changing the"}],"source_content_type":"text/x-rst","patch_set":1,"id":"ba15a1d1_9937f0af","line":113,"in_reply_to":"ba15a1d1_c6e027cf","updated":"2015-09-22 15:48:54.000000000","message":"Oh, we\u0027re looking at it from different perspectives. Your way is probably clearer in terms of what the scheduler actually implements as logic.","commit_id":"86f6574d253cf7a3639cedc0c5dcb8e30b42dfdf"},{"author":{"_account_id":7730,"name":"Sahid Orentino Ferdjaoui","email":"sahid.ferdjaoui@industrialdiscipline.com","username":"sahid"},"change_message_id":"ad1835909242f9d9f06b05ad58a124921a1d8cd5","unresolved":false,"context_lines":[{"line_number":129,"context_line":"a full dedicatd host pCPU to run on. To allow this to be controlled"},{"line_number":130,"context_line":"there will be a second flavour extra spec"},{"line_number":131,"context_line":""},{"line_number":132,"context_line":"* hw:cpu_system_policy\u003dshared|dedicated"},{"line_number":133,"context_line":""},{"line_number":134,"context_line":"Similarly to the current hw:cpu_policy parameter, this will indicate"},{"line_number":135,"context_line":"whether the system psuedo-CPUs are to be given dedicated pCPUs of"}],"source_content_type":"text/x-rst","patch_set":1,"id":"ba15a1d1_b902f407","line":132,"updated":"2015-09-22 15:42:45.000000000","message":"I do not believe we should to have this option. If cpu_system\u003dN is set we should to consider we want N dedicated CPUs for system.\n\nProblem is we will have to \"dedicate\" CPUs even for a system_policy\u003dshared.","commit_id":"86f6574d253cf7a3639cedc0c5dcb8e30b42dfdf"},{"author":{"_account_id":1779,"name":"Daniel Berrange","email":"berrange@redhat.com","username":"berrange"},"change_message_id":"a49a4545d43659f7a47f1db7db10a3d1782b0433","unresolved":false,"context_lines":[{"line_number":129,"context_line":"a full dedicatd host pCPU to run on. To allow this to be controlled"},{"line_number":130,"context_line":"there will be a second flavour extra spec"},{"line_number":131,"context_line":""},{"line_number":132,"context_line":"* hw:cpu_system_policy\u003dshared|dedicated"},{"line_number":133,"context_line":""},{"line_number":134,"context_line":"Similarly to the current hw:cpu_policy parameter, this will indicate"},{"line_number":135,"context_line":"whether the system psuedo-CPUs are to be given dedicated pCPUs of"}],"source_content_type":"text/x-rst","patch_set":1,"id":"ba15a1d1_b92f543d","line":132,"in_reply_to":"ba15a1d1_b902f407","updated":"2015-09-22 15:48:54.000000000","message":"It is certainly simpler if we didn\u0027t have this. eg just assume that hw:cpu_policy applies to the system psuedo-CPUs too. I just added it as I thought there might be a need for to have hw:cpu_policy\u003ddedicated and hw:cpu_system_policy\u003dshared. That\u0027s probably overkill unless someone steps forward with a clear requirement.","commit_id":"86f6574d253cf7a3639cedc0c5dcb8e30b42dfdf"},{"author":{"_account_id":7730,"name":"Sahid Orentino Ferdjaoui","email":"sahid.ferdjaoui@industrialdiscipline.com","username":"sahid"},"change_message_id":"43d043abc6e786ce75a6e2dec2abb993e7a94ee4","unresolved":false,"context_lines":[{"line_number":124,"context_line":"on the mask determined by the NUMA filter. The current default"},{"line_number":125,"context_line":"logic of floating emulator threads across all host pCPUs associated"},{"line_number":126,"context_line":"with vCPUs will only be used if hw:cpu_system is not set."},{"line_number":127,"context_line":""},{"line_number":128,"context_line":"Alternatives"},{"line_number":129,"context_line":"------------"},{"line_number":130,"context_line":""}],"source_content_type":"text/x-rst","patch_set":2,"id":"ba15a1d1_cc0b16d4","line":127,"updated":"2015-09-24 07:52:10.000000000","message":"As a best practice we may want to have emulator threads running on a different NUMA host than vCPUs.  How can we tune this configuration to answer that ?\n\nFor example you have used before do we have to consider cpu_system IDs will be 8, 9 ?\n\n  vcpus\u003d8\n  hw:cpu_system\u003d2\n  hw:numa_node\u003d2\n  hw:numa_node.0 \u003d 0-7\n  hw:numa_node.1 \u003d 8,9\n\nWe probably need something clear since this will influence development.","commit_id":"510046234ffdce31165c141bafb7838ab561e37b"},{"author":{"_account_id":7730,"name":"Sahid Orentino Ferdjaoui","email":"sahid.ferdjaoui@industrialdiscipline.com","username":"sahid"},"change_message_id":"82f3d81f84972589ec881b9ea1b3f60c1f510a34","unresolved":false,"context_lines":[{"line_number":133,"context_line":"have 6 system threads, running across 2 CPUs - we don\u0027t know how"},{"line_number":134,"context_line":"many threads will be present at any time. So we can\u0027t allow fine"},{"line_number":135,"context_line":"grained control. So we merely need a property to specify the"},{"line_number":136,"context_line":"single guest NUMA node to associate the system psuedo-CPUs with"},{"line_number":137,"context_line":"e.g. the following says to associate with NUMA node 3."},{"line_number":138,"context_line":""},{"line_number":139,"context_line":"* hw:numa_cpu_system\u003d3"}],"source_content_type":"text/x-rst","patch_set":3,"id":"9a1a9d01_56172e51","line":136,"updated":"2015-09-25 07:40:32.000000000","message":"s/psuedo-CPUs/pseudo-CPUs :)","commit_id":"1a856738867334869a358e20d3133e771d6442bd"},{"author":{"_account_id":7730,"name":"Sahid Orentino Ferdjaoui","email":"sahid.ferdjaoui@industrialdiscipline.com","username":"sahid"},"change_message_id":"82f3d81f84972589ec881b9ea1b3f60c1f510a34","unresolved":false,"context_lines":[{"line_number":136,"context_line":"single guest NUMA node to associate the system psuedo-CPUs with"},{"line_number":137,"context_line":"e.g. the following says to associate with NUMA node 3."},{"line_number":138,"context_line":""},{"line_number":139,"context_line":"* hw:numa_cpu_system\u003d3"},{"line_number":140,"context_line":""},{"line_number":141,"context_line":"Alternatives"},{"line_number":142,"context_line":"------------"}],"source_content_type":"text/x-rst","patch_set":3,"id":"9a1a9d01_79ffc301","line":139,"updated":"2015-09-25 07:40:32.000000000","message":"Yeah sounds good, my initial solution was clearly not good since I mixed vCPU IDs and host CPU IDs.","commit_id":"1a856738867334869a358e20d3133e771d6442bd"},{"author":{"_account_id":1779,"name":"Daniel Berrange","email":"berrange@redhat.com","username":"berrange"},"change_message_id":"61da97b4d4ed5428366963233cebc22967d12040","unresolved":false,"context_lines":[{"line_number":136,"context_line":"single guest NUMA node to associate the system psuedo-CPUs with"},{"line_number":137,"context_line":"e.g. the following says to associate with NUMA node 3."},{"line_number":138,"context_line":""},{"line_number":139,"context_line":"* hw:numa_cpu_system\u003d3"},{"line_number":140,"context_line":""},{"line_number":141,"context_line":"Alternatives"},{"line_number":142,"context_line":"------------"}],"source_content_type":"text/x-rst","patch_set":3,"id":"9a1a9d01_9c63a5b1","line":139,"in_reply_to":"9a1a9d01_39ddcb42","updated":"2015-09-25 08:23:13.000000000","message":"I don\u0027t think you would ever want to do such a setup. It would guarantee that all your I/O operations would cause cross-node memory transfer which would seriously hurt the performance. You really really want to have the system threads running on the same NUMA node as at least one of your vCPUs","commit_id":"1a856738867334869a358e20d3133e771d6442bd"},{"author":{"_account_id":7730,"name":"Sahid Orentino Ferdjaoui","email":"sahid.ferdjaoui@industrialdiscipline.com","username":"sahid"},"change_message_id":"b061e93fbd06bb3fc88dc9ef4125507d70b0f8ca","unresolved":false,"context_lines":[{"line_number":136,"context_line":"single guest NUMA node to associate the system psuedo-CPUs with"},{"line_number":137,"context_line":"e.g. the following says to associate with NUMA node 3."},{"line_number":138,"context_line":""},{"line_number":139,"context_line":"* hw:numa_cpu_system\u003d3"},{"line_number":140,"context_line":""},{"line_number":141,"context_line":"Alternatives"},{"line_number":142,"context_line":"------------"}],"source_content_type":"text/x-rst","patch_set":3,"id":"9a1a9d01_39ddcb42","line":139,"in_reply_to":"9a1a9d01_79ffc301","updated":"2015-09-25 07:53:12.000000000","message":"That say.. I\u0027m not sure in the current implementation we can specify for instance 2 guests NUMA nodes and set all vCPUs on node 0.\n\n  vcpus\u003d4\n  numa_nodes \u003d 2\n  numa_node.0 \u003d 0-3\n\nWe can probably change that behavior, but I think we should keep the same syntax instead of adding a new option.\n\n  numa_node.1 \u003d cpu_system","commit_id":"1a856738867334869a358e20d3133e771d6442bd"},{"author":{"_account_id":7730,"name":"Sahid Orentino Ferdjaoui","email":"sahid.ferdjaoui@industrialdiscipline.com","username":"sahid"},"change_message_id":"942c122984335c03369c26100209cc4ee6025c51","unresolved":false,"context_lines":[{"line_number":136,"context_line":"single guest NUMA node to associate the system psuedo-CPUs with"},{"line_number":137,"context_line":"e.g. the following says to associate with NUMA node 3."},{"line_number":138,"context_line":""},{"line_number":139,"context_line":"* hw:numa_cpu_system\u003d3"},{"line_number":140,"context_line":""},{"line_number":141,"context_line":"Alternatives"},{"line_number":142,"context_line":"------------"}],"source_content_type":"text/x-rst","patch_set":3,"id":"9a1a9d01_bcad498a","line":139,"in_reply_to":"9a1a9d01_9c63a5b1","updated":"2015-09-25 08:37:53.000000000","message":"I\u0027m thinking about realtime context: We want RT-vCPUs be completely isolated on a host NUMA nodes to avoid any interference which can impact latency.\n\nThese workloads are intended to be used in a private cloud with dedicated hosts for running realtime. Operator will have to configure flavor topology to have always realtime cpu running on isolated host NUMA nodes.\n\n vcpus \u003d 2\n hw:cpu_system \u003d 1\n hw:cpu_realtime_mask \u003d ^1\n hw:numa_nodes \u003d 2\n hw:numa_node.0 \u003d 0 # realtime scheduler\n hw:numa_node.1 \u003d 1 # regular scheduler\n hw:numa_cpu_system \u003d 1\n\nOops so your solution is working perfectly :)","commit_id":"1a856738867334869a358e20d3133e771d6442bd"}]}