)]}'
{"specs/mitaka/approved/numa-optimization-ppc.rst":[{"author":{"_account_id":5511,"name":"Nikola Dipanov","email":"ndipanov@redhat.com","username":"ndipanov"},"change_message_id":"17354005c254cdae851b15c544fe5ad8b48b2fed","unresolved":false,"context_lines":[{"line_number":50,"context_line":"to Openstack. Since PPC architecture involves core based scheduling,"},{"line_number":51,"context_line":"estimated changes would be needed at the following places:"},{"line_number":52,"context_line":""},{"line_number":53,"context_line":"* Account for threads per core on the host while deciding NUMA cell"},{"line_number":54,"context_line":"  affinity on the host."},{"line_number":55,"context_line":""},{"line_number":56,"context_line":"* Do CPU pinning by having a knowledge of the threads inside each core"},{"line_number":57,"context_line":"  instead of assuming cores with a single thread."}],"source_content_type":"text/x-rst","patch_set":3,"id":"9a8ffd7b_c28a3464","line":54,"range":{"start_line":53,"start_character":2,"end_line":54,"end_character":21},"updated":"2015-12-02 11:36:45.000000000","message":"So this is the bit that makes or breaks this idea imho.\n\nI\u0027d like to understand more about how you plan to do these? do we just estimate? do we do some types of tables lookups based on they processor info we have?\n\nIn almost all of the cases above - none of that stuff should live in Nova and is likely much better suited as a feature-add for libvirt","commit_id":"62e11aa55f6c9d2098287b6b49cda2712108d933"},{"author":{"_account_id":5511,"name":"Nikola Dipanov","email":"ndipanov@redhat.com","username":"ndipanov"},"change_message_id":"f7dd64545afdd9dc04ac9558ee1ee5e892aca02d","unresolved":false,"context_lines":[{"line_number":51,"context_line":"estimated changes would be needed at the following places:"},{"line_number":52,"context_line":""},{"line_number":53,"context_line":"* Account for threads per core on the host while deciding NUMA cell"},{"line_number":54,"context_line":"  affinity on the host."},{"line_number":55,"context_line":""},{"line_number":56,"context_line":"* Do CPU pinning by having a knowledge of the threads inside each core"},{"line_number":57,"context_line":"  instead of assuming cores with a single thread."}],"source_content_type":"text/x-rst","patch_set":3,"id":"9a8ffd7b_c5b31e4c","line":54,"in_reply_to":"9a8ffd7b_a24b70b0","updated":"2015-12-02 11:57:57.000000000","message":"\u003e This host topology should be an indications on what the core to thread ratio is...\n\nI don\u0027t understand this. It will be an indication how? Based on what other data, if the data we have from libvirt is incomplete?\n\nYou really should expand on this with a concrete example, otherwise IMHO this spec does not carry enough information for us to decide if it\u0027s something we want to do. In the general case - yes we want to make sure Nova works for all architectures, but not at the expense of introducing brittle code or code with umanageable dependencies for the Nova project as we are not in the business of supporting hardware platfroms but supporting (some ?) virtualizations technologies.","commit_id":"62e11aa55f6c9d2098287b6b49cda2712108d933"},{"author":{"_account_id":9775,"name":"Sudipta Biswas","email":"sbiswas7@in.ibm.com","username":"sudswas"},"change_message_id":"23eb297f6a2e27150c642ffe552421506bbd60c1","unresolved":false,"context_lines":[{"line_number":51,"context_line":"estimated changes would be needed at the following places:"},{"line_number":52,"context_line":""},{"line_number":53,"context_line":"* Account for threads per core on the host while deciding NUMA cell"},{"line_number":54,"context_line":"  affinity on the host."},{"line_number":55,"context_line":""},{"line_number":56,"context_line":"* Do CPU pinning by having a knowledge of the threads inside each core"},{"line_number":57,"context_line":"  instead of assuming cores with a single thread."}],"source_content_type":"text/x-rst","patch_set":3,"id":"9a8ffd7b_a24b70b0","line":54,"in_reply_to":"9a8ffd7b_c28a3464","updated":"2015-12-02 11:49:54.000000000","message":"AFAIK, this data can be obtained from the cpu_info field in the compute_nodes table.\nThe topology of the host cpu can be obtained via this?\n\nThis host topology should be an indications on what the core to thread ratio is...So I guess no lookup is necessary. Do you agree?","commit_id":"62e11aa55f6c9d2098287b6b49cda2712108d933"},{"author":{"_account_id":12898,"name":"Tony Breeds","email":"tony@bakeyournoodle.com","username":"tonyb"},"change_message_id":"4d5633bd78a29345bc112709b2510f154e4b5d50","unresolved":false,"context_lines":[{"line_number":63,"context_line":"  length of the VM cell length (as is done today) to choose the best"},{"line_number":64,"context_line":"  host_cell for the instance_cell. Note, the online cpus shown by"},{"line_number":65,"context_line":"  libvirt in case of PowerPC denote the primary thread of each core."},{"line_number":66,"context_line":"  This way, it becomes a good representation of the core itself."},{"line_number":67,"context_line":""},{"line_number":68,"context_line":"* Do CPU pinning by having a knowledge of the threads inside each core"},{"line_number":69,"context_line":"  instead of assuming cores with a single thread. Since we"}],"source_content_type":"text/x-rst","patch_set":4,"id":"7a740942_5572e2e6","line":66,"updated":"2015-12-04 01:49:29.000000000","message":"This explanation doesn\u0027t match what I see with libvirt (1.2.16) on a Power8  where libvirt says I have \"sockets\u003d1, cores\u003d192, threads\u003d1\"  Which is clearly wrong.  The h/w is \"sockets\u003d2, cores\u003d12, threads\u003d8\",  still the same total cores, 192, 24 of which are primary.  Where are you getting threads_per_core from?\n\nAlso on ppc threads_per_core is somewhat dynamic.  The kernel *should* hide some of that but it does make placing instances more challenging.\n\nI\u0027m all for optimising but lets start with correct.","commit_id":"4a0c9d1fdba1ec71916e5e1a8fed03502f1a0cc5"},{"author":{"_account_id":9775,"name":"Sudipta Biswas","email":"sbiswas7@in.ibm.com","username":"sudswas"},"change_message_id":"21a579ab63cf57eeedd689e42b78afe538e9cf6e","unresolved":false,"context_lines":[{"line_number":63,"context_line":"  length of the VM cell length (as is done today) to choose the best"},{"line_number":64,"context_line":"  host_cell for the instance_cell. Note, the online cpus shown by"},{"line_number":65,"context_line":"  libvirt in case of PowerPC denote the primary thread of each core."},{"line_number":66,"context_line":"  This way, it becomes a good representation of the core itself."},{"line_number":67,"context_line":""},{"line_number":68,"context_line":"* Do CPU pinning by having a knowledge of the threads inside each core"},{"line_number":69,"context_line":"  instead of assuming cores with a single thread. Since we"}],"source_content_type":"text/x-rst","patch_set":4,"id":"7a740942_b0a44da6","line":66,"in_reply_to":"7a740942_5572e2e6","updated":"2015-12-04 17:59:15.000000000","message":"Hi Tony, You make very interesting observations and I would hope to clarify a few here.\n\nI completely agree to your comment on the minimum libvirt version. I have libvirt 1.2.19 - and it\u0027s fixed on this one. I would like to go back and check the minimum version and document that here.\n\nSecondly to comment on the dynamic split core that you mentioned above. \nIn customer environments, IMHO compute hosts which would be used for pinning are separate from the ones that would be used for general placement . This ensures that the host will have no floating guest vcpus (they will all be pinned) - and hence the dynamic split core won\u0027t be in effect.  In such a case, the admin may also choose to disable split core completely. Thoughts?","commit_id":"4a0c9d1fdba1ec71916e5e1a8fed03502f1a0cc5"},{"author":{"_account_id":5511,"name":"Nikola Dipanov","email":"ndipanov@redhat.com","username":"ndipanov"},"change_message_id":"7489eb642863eb8f82bc557639e1340957186c2f","unresolved":false,"context_lines":[{"line_number":63,"context_line":"  length of the VM cell length (as is done today) to choose the best"},{"line_number":64,"context_line":"  host_cell for the instance_cell. Note, the online cpus shown by"},{"line_number":65,"context_line":"  libvirt in case of PowerPC denote the primary thread of each core."},{"line_number":66,"context_line":"  This way, it becomes a good representation of the core itself."},{"line_number":67,"context_line":""},{"line_number":68,"context_line":"* Do CPU pinning by having a knowledge of the threads inside each core"},{"line_number":69,"context_line":"  instead of assuming cores with a single thread. Since we"}],"source_content_type":"text/x-rst","patch_set":4,"id":"7a740942_9a462322","line":66,"in_reply_to":"7a740942_5572e2e6","updated":"2015-12-04 12:05:04.000000000","message":"I don\u0027t have access to a Power box ATM - this is good info! Thanks Tony.","commit_id":"4a0c9d1fdba1ec71916e5e1a8fed03502f1a0cc5"},{"author":{"_account_id":5511,"name":"Nikola Dipanov","email":"ndipanov@redhat.com","username":"ndipanov"},"change_message_id":"93ec9076c5fa1467a75e03bf54bdc58089ad7778","unresolved":false,"context_lines":[{"line_number":63,"context_line":"  length of the VM cell length (as is done today) to choose the best"},{"line_number":64,"context_line":"  host_cell for the instance_cell. Note, the online cpus shown by"},{"line_number":65,"context_line":"  libvirt in case of PowerPC denote the primary thread of each core."},{"line_number":66,"context_line":"  This way, it becomes a good representation of the core itself."},{"line_number":67,"context_line":""},{"line_number":68,"context_line":"* Do CPU pinning by having a knowledge of the threads inside each core"},{"line_number":69,"context_line":"  instead of assuming cores with a single thread. Since we"}],"source_content_type":"text/x-rst","patch_set":4,"id":"7a740942_7a5a5785","line":66,"in_reply_to":"7a740942_9a462322","updated":"2015-12-04 12:12:17.000000000","message":"\u003e This explanation doesn\u0027t match what I see with libvirt (1.2.16) on a Power8 where libvirt says I have \"sockets\u003d1, cores\u003d192, threads\u003d1\"\n\nActually thinking about this more - if this is true - we should not merge the patch where we enable NUMA pinning for PPC as it will do the wrong thing and fail to boot an instance  IIUC.","commit_id":"4a0c9d1fdba1ec71916e5e1a8fed03502f1a0cc5"},{"author":{"_account_id":12898,"name":"Tony Breeds","email":"tony@bakeyournoodle.com","username":"tonyb"},"change_message_id":"6fd41c8cad2bffbacfcf405cc4bf01089ee93efb","unresolved":false,"context_lines":[{"line_number":63,"context_line":"  length of the VM cell length (as is done today) to choose the best"},{"line_number":64,"context_line":"  host_cell for the instance_cell. Note, the online cpus shown by"},{"line_number":65,"context_line":"  libvirt in case of PowerPC denote the primary thread of each core."},{"line_number":66,"context_line":"  This way, it becomes a good representation of the core itself."},{"line_number":67,"context_line":""},{"line_number":68,"context_line":"* Do CPU pinning by having a knowledge of the threads inside each core"},{"line_number":69,"context_line":"  instead of assuming cores with a single thread. Since we"}],"source_content_type":"text/x-rst","patch_set":4,"id":"7a740942_0aa5cca0","line":66,"in_reply_to":"7a740942_b0a44da6","updated":"2015-12-06 23:46:28.000000000","message":"I\u0027ll need to digest that carefully.  Knowing that 1.2.19 is somewhat better is good information.  I\u0027ll run up that tarball and see what else is fixed.  The commit wont fix the socket detection (but the SHA you point out could be part of a larger series I haven\u0027t checked).\n\nThe micro-threading specifically static will make life harder for nova/libvirt.  but you\u0027re probably right that admins that want to provide stable NUMA to guests will probably do so outside of libvirt/nova","commit_id":"4a0c9d1fdba1ec71916e5e1a8fed03502f1a0cc5"},{"author":{"_account_id":12898,"name":"Tony Breeds","email":"tony@bakeyournoodle.com","username":"tonyb"},"change_message_id":"4d5633bd78a29345bc112709b2510f154e4b5d50","unresolved":false,"context_lines":[{"line_number":158,"context_line":"Testing"},{"line_number":159,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d"},{"line_number":160,"context_line":""},{"line_number":161,"context_line":"The changes will be exercised through unit tests."},{"line_number":162,"context_line":""},{"line_number":163,"context_line":"Documentation Impact"},{"line_number":164,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"}],"source_content_type":"text/x-rst","patch_set":4,"id":"7a740942_0de21404","line":161,"updated":"2015-12-04 01:49:29.000000000","message":"You need to elaborate here.  We have issues when the uint tests run on x86 and need to mock arch etc to be ppc.  This isn\u0027t a simple write 5 or 6 unit tests.","commit_id":"4a0c9d1fdba1ec71916e5e1a8fed03502f1a0cc5"},{"author":{"_account_id":5511,"name":"Nikola Dipanov","email":"ndipanov@redhat.com","username":"ndipanov"},"change_message_id":"7489eb642863eb8f82bc557639e1340957186c2f","unresolved":false,"context_lines":[{"line_number":158,"context_line":"Testing"},{"line_number":159,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d"},{"line_number":160,"context_line":""},{"line_number":161,"context_line":"The changes will be exercised through unit tests."},{"line_number":162,"context_line":""},{"line_number":163,"context_line":"Documentation Impact"},{"line_number":164,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"}],"source_content_type":"text/x-rst","patch_set":4,"id":"7a740942_1ae55336","line":161,"in_reply_to":"7a740942_0de21404","updated":"2015-12-04 12:05:04.000000000","message":"Well in all fairness we do mock most of what libvirt gives us back so the author likely meant that any code in Nova will be tested this way. Would be excellent to clarify though.","commit_id":"4a0c9d1fdba1ec71916e5e1a8fed03502f1a0cc5"},{"author":{"_account_id":12898,"name":"Tony Breeds","email":"tony@bakeyournoodle.com","username":"tonyb"},"change_message_id":"6fd41c8cad2bffbacfcf405cc4bf01089ee93efb","unresolved":false,"context_lines":[{"line_number":158,"context_line":"Testing"},{"line_number":159,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d"},{"line_number":160,"context_line":""},{"line_number":161,"context_line":"The changes will be exercised through unit tests."},{"line_number":162,"context_line":""},{"line_number":163,"context_line":"Documentation Impact"},{"line_number":164,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"}],"source_content_type":"text/x-rst","patch_set":4,"id":"7a740942_8ad45c20","line":161,"in_reply_to":"7a740942_1ae55336","updated":"2015-12-06 23:46:28.000000000","message":"Sure.  I thought there were issues mocking get_arch and it causing problems with _get_guest_config() but I could be misremembering.  If we can just mock \"all the things\" and make this testable on both x86 and power then #winning :)","commit_id":"4a0c9d1fdba1ec71916e5e1a8fed03502f1a0cc5"}]}