)]}'
{"specs/queens/approved/libvirt-emulator-threads-policy.rst":[{"author":{"_account_id":11604,"name":"sean mooney","email":"smooney@redhat.com","username":"sean-k-mooney"},"change_message_id":"7b69fedbd6fac8d0ba9668cd7614ce12c491c53d","unresolved":false,"context_lines":[{"line_number":80,"context_line":"Proposed change"},{"line_number":81,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"},{"line_number":82,"context_line":""},{"line_number":83,"context_line":"A pre-requisite for enabling the emulator threads placement policy"},{"line_number":84,"context_line":"feature on a flavor is that it must also have ‘hw:cpu_policy’ set to"},{"line_number":85,"context_line":"‘dedicated’."},{"line_number":86,"context_line":""},{"line_number":87,"context_line":"Each hypervisor has a different architecture, for example QEMU has"},{"line_number":88,"context_line":"emulator threads, while Xen has stub-domains. To avoid favoring any"}],"source_content_type":"text/x-rst","patch_set":1,"id":"5f4e5783_ca6eccfe","line":85,"range":{"start_line":83,"start_character":0,"end_line":85,"end_character":12},"updated":"2017-10-17 20:02:04.000000000","message":"why? i can still see a usecase for this if it is set to shared. e.g. as an operator if i am not chargeing you for the cost of the emulator thread i may want to dedicate a pool of cpus for all my instances to use when spawning the emulator thread.","commit_id":"b9604184a9c79353a57d3ac5250d7be9989350c4"},{"author":{"_account_id":11604,"name":"sean mooney","email":"smooney@redhat.com","username":"sean-k-mooney"},"change_message_id":"0bef9802555c7644c10405ec4c654d31d6a23134","unresolved":false,"context_lines":[{"line_number":82,"context_line":""},{"line_number":83,"context_line":"A pre-requisite for enabling the emulator threads placement policy"},{"line_number":84,"context_line":"feature on a flavor is that it must also have ‘hw:cpu_policy’ set to"},{"line_number":85,"context_line":"‘dedicated’."},{"line_number":86,"context_line":""},{"line_number":87,"context_line":"Each hypervisor has a different architecture, for example QEMU has"},{"line_number":88,"context_line":"emulator threads, while Xen has stub-domains. To avoid favoring any"}],"source_content_type":"text/x-rst","patch_set":1,"id":"3f4b6375_4712935a","line":85,"in_reply_to":"","updated":"2017-10-19 02:25:57.000000000","message":"Their is a slight performance hit with heavy io but yes if not realtime the this change as a whole is not needed but i still dont see that as a reason to not allow tis for shared cores. There should be no downside  to using the host policy for shared cores either right or am i missing something. For isolate the cpu policy dedicated makes sense.","commit_id":"b9604184a9c79353a57d3ac5250d7be9989350c4"},{"author":{"_account_id":11604,"name":"sean mooney","email":"smooney@redhat.com","username":"sean-k-mooney"},"change_message_id":"371177e0d488e48a2bc0c3dc3136b0fa72555d51","unresolved":false,"context_lines":[{"line_number":82,"context_line":""},{"line_number":83,"context_line":"A pre-requisite for enabling the emulator threads placement policy"},{"line_number":84,"context_line":"feature on a flavor is that it must also have ‘hw:cpu_policy’ set to"},{"line_number":85,"context_line":"‘dedicated’."},{"line_number":86,"context_line":""},{"line_number":87,"context_line":"Each hypervisor has a different architecture, for example QEMU has"},{"line_number":88,"context_line":"emulator threads, while Xen has stub-domains. To avoid favoring any"}],"source_content_type":"text/x-rst","patch_set":1,"id":"3f4b6375_17ed86b6","line":85,"in_reply_to":"3f4b6375_344fb1be","updated":"2017-11-02 04:12:56.000000000","message":"I don\u0027t see that as a downside. The stricter scheduling will give better performance without disallowing oversubscription like pinning would","commit_id":"b9604184a9c79353a57d3ac5250d7be9989350c4"},{"author":{"_account_id":7730,"name":"Sahid Orentino Ferdjaoui","email":"sahid.ferdjaoui@industrialdiscipline.com","username":"sahid"},"change_message_id":"e82791525d27ad41ed357aa3e496bf20998e840a","unresolved":false,"context_lines":[{"line_number":82,"context_line":""},{"line_number":83,"context_line":"A pre-requisite for enabling the emulator threads placement policy"},{"line_number":84,"context_line":"feature on a flavor is that it must also have ‘hw:cpu_policy’ set to"},{"line_number":85,"context_line":"‘dedicated’."},{"line_number":86,"context_line":""},{"line_number":87,"context_line":"Each hypervisor has a different architecture, for example QEMU has"},{"line_number":88,"context_line":"emulator threads, while Xen has stub-domains. To avoid favoring any"}],"source_content_type":"text/x-rst","patch_set":1,"id":"3f4b6375_91e68216","line":85,"in_reply_to":"3f4b6375_4712935a","updated":"2017-10-19 07:32:18.000000000","message":"It\u0027s not a performance problem or something. If the guests vCPUs threads are moving across the pCPUs available for Nova so pinning emulator threads do not make sense as you can understand.\n\nWhen using pinning (cpu_policy\u003ddedicated) operators have the pre-requistes to isolates all the guests on certain hosts by using aggregates.","commit_id":"b9604184a9c79353a57d3ac5250d7be9989350c4"},{"author":{"_account_id":8768,"name":"Chris Friesen","email":"chris.friesen@windriver.com","username":"cbf123"},"change_message_id":"fd4a6cb72364ddbda8ddf7d950651cbe4b34472d","unresolved":false,"context_lines":[{"line_number":82,"context_line":""},{"line_number":83,"context_line":"A pre-requisite for enabling the emulator threads placement policy"},{"line_number":84,"context_line":"feature on a flavor is that it must also have ‘hw:cpu_policy’ set to"},{"line_number":85,"context_line":"‘dedicated’."},{"line_number":86,"context_line":""},{"line_number":87,"context_line":"Each hypervisor has a different architecture, for example QEMU has"},{"line_number":88,"context_line":"emulator threads, while Xen has stub-domains. To avoid favoring any"}],"source_content_type":"text/x-rst","patch_set":1,"id":"3f4b6375_344fb1be","line":85,"in_reply_to":"3f4b6375_91e68216","updated":"2017-10-19 15:41:58.000000000","message":"Also, if you have non-dedicated CPUs, then setting emulator_threads_policy to \"host\" will mean that you can\u0027t schedule on any hosts that don\u0027t reserve CPUs via overhead_pin_set (or emulator_pin_set, or whatever we call it).  This is a net loss of available compute nodes for no significant gain.","commit_id":"b9604184a9c79353a57d3ac5250d7be9989350c4"},{"author":{"_account_id":8768,"name":"Chris Friesen","email":"chris.friesen@windriver.com","username":"cbf123"},"change_message_id":"5c9714b8dafb640c025c3bc46b66a1f6bb9724fe","unresolved":false,"context_lines":[{"line_number":80,"context_line":"Proposed change"},{"line_number":81,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"},{"line_number":82,"context_line":""},{"line_number":83,"context_line":"A pre-requisite for enabling the emulator threads placement policy"},{"line_number":84,"context_line":"feature on a flavor is that it must also have ‘hw:cpu_policy’ set to"},{"line_number":85,"context_line":"‘dedicated’."},{"line_number":86,"context_line":""},{"line_number":87,"context_line":"Each hypervisor has a different architecture, for example QEMU has"},{"line_number":88,"context_line":"emulator threads, while Xen has stub-domains. To avoid favoring any"}],"source_content_type":"text/x-rst","patch_set":1,"id":"3f4b6375_56d5b2b8","line":85,"range":{"start_line":83,"start_character":0,"end_line":85,"end_character":12},"in_reply_to":"5f4e5783_ca6eccfe","updated":"2017-10-18 15:49:56.000000000","message":"When not using realtime, I don\u0027t see a downside of affining the emulator threads with the CPU threads.","commit_id":"b9604184a9c79353a57d3ac5250d7be9989350c4"},{"author":{"_account_id":8768,"name":"Chris Friesen","email":"chris.friesen@windriver.com","username":"cbf123"},"change_message_id":"e4ba9a012ceedb0467f202082f6e7802273620ed","unresolved":false,"context_lines":[{"line_number":87,"context_line":"Each hypervisor has a different architecture, for example QEMU has"},{"line_number":88,"context_line":"emulator threads, while Xen has stub-domains. To avoid favoring any"},{"line_number":89,"context_line":"specific implementation, the idea is to extend"},{"line_number":90,"context_line":"`estimate_instance_overhead` to return 1 additional host CPU to take"},{"line_number":91,"context_line":"into account during claim. A user which expresses the desire to"},{"line_number":92,"context_line":"isolate emulator threads must use a flavor configured to accept that"},{"line_number":93,"context_line":"specification as:"},{"line_number":94,"context_line":""}],"source_content_type":"text/x-rst","patch_set":1,"id":"5f4e5783_0cb1b7e1","line":91,"range":{"start_line":90,"start_character":0,"end_line":91,"end_character":25},"updated":"2017-10-17 04:57:24.000000000","message":"with \"hw:cpu_emulator_threads\u003dhost\" it no longer makes sense to consume one extra CPU in the claim...this should be documented.","commit_id":"b9604184a9c79353a57d3ac5250d7be9989350c4"},{"author":{"_account_id":11604,"name":"sean mooney","email":"smooney@redhat.com","username":"sean-k-mooney"},"change_message_id":"371177e0d488e48a2bc0c3dc3136b0fa72555d51","unresolved":false,"context_lines":[{"line_number":88,"context_line":"emulator threads, while Xen has stub-domains. To avoid favoring any"},{"line_number":89,"context_line":"specific implementation, the idea is to extend"},{"line_number":90,"context_line":"`estimate_instance_overhead` to return 1 additional host CPU to take"},{"line_number":91,"context_line":"into account during claim. A user which expresses the desire to"},{"line_number":92,"context_line":"isolate emulator threads must use a flavor configured to accept that"},{"line_number":93,"context_line":"specification as:"},{"line_number":94,"context_line":""}],"source_content_type":"text/x-rst","patch_set":1,"id":"3f4b6375_97acd69f","line":91,"in_reply_to":"3f4b6375_b1ba9edb","updated":"2017-11-02 04:12:56.000000000","message":"You need to asser that earlier in the paragraph as written this sates that we will allways consume an additional core","commit_id":"b9604184a9c79353a57d3ac5250d7be9989350c4"},{"author":{"_account_id":11604,"name":"sean mooney","email":"smooney@redhat.com","username":"sean-k-mooney"},"change_message_id":"7b69fedbd6fac8d0ba9668cd7614ce12c491c53d","unresolved":false,"context_lines":[{"line_number":87,"context_line":"Each hypervisor has a different architecture, for example QEMU has"},{"line_number":88,"context_line":"emulator threads, while Xen has stub-domains. To avoid favoring any"},{"line_number":89,"context_line":"specific implementation, the idea is to extend"},{"line_number":90,"context_line":"`estimate_instance_overhead` to return 1 additional host CPU to take"},{"line_number":91,"context_line":"into account during claim. A user which expresses the desire to"},{"line_number":92,"context_line":"isolate emulator threads must use a flavor configured to accept that"},{"line_number":93,"context_line":"specification as:"},{"line_number":94,"context_line":""}],"source_content_type":"text/x-rst","patch_set":1,"id":"5f4e5783_caa00c45","line":91,"range":{"start_line":90,"start_character":0,"end_line":91,"end_character":25},"in_reply_to":"5f4e5783_0cb1b7e1","updated":"2017-10-17 20:02:04.000000000","message":"agreed, it would make sense if not use a pool of host cpus when we are doing allocation per vm.","commit_id":"b9604184a9c79353a57d3ac5250d7be9989350c4"},{"author":{"_account_id":7730,"name":"Sahid Orentino Ferdjaoui","email":"sahid.ferdjaoui@industrialdiscipline.com","username":"sahid"},"change_message_id":"e82791525d27ad41ed357aa3e496bf20998e840a","unresolved":false,"context_lines":[{"line_number":87,"context_line":"Each hypervisor has a different architecture, for example QEMU has"},{"line_number":88,"context_line":"emulator threads, while Xen has stub-domains. To avoid favoring any"},{"line_number":89,"context_line":"specific implementation, the idea is to extend"},{"line_number":90,"context_line":"`estimate_instance_overhead` to return 1 additional host CPU to take"},{"line_number":91,"context_line":"into account during claim. A user which expresses the desire to"},{"line_number":92,"context_line":"isolate emulator threads must use a flavor configured to accept that"},{"line_number":93,"context_line":"specification as:"},{"line_number":94,"context_line":""}],"source_content_type":"text/x-rst","patch_set":1,"id":"3f4b6375_b1ba9edb","line":91,"range":{"start_line":90,"start_character":0,"end_line":91,"end_character":25},"in_reply_to":"5f4e5783_caa00c45","updated":"2017-10-19 07:32:18.000000000","message":"Please consider the whole paragraph we are talking about \u0027hw:cpu_emulator_threads\u003disolate\u0027","commit_id":"b9604184a9c79353a57d3ac5250d7be9989350c4"},{"author":{"_account_id":8768,"name":"Chris Friesen","email":"chris.friesen@windriver.com","username":"cbf123"},"change_message_id":"e4ba9a012ceedb0467f202082f6e7802273620ed","unresolved":false,"context_lines":[{"line_number":94,"context_line":""},{"line_number":95,"context_line":"* hw:cpu_emulator_threads\u003disolate"},{"line_number":96,"context_line":""},{"line_number":97,"context_line":"Would say that this instance is to be considered to consume 1"},{"line_number":98,"context_line":"additional host CPU. That pCPU used to make running emulator threads"},{"line_number":99,"context_line":"is going to always be configured on the related guest NUMA node ID 0,"},{"line_number":100,"context_line":"to make it predictable for users. Currently there is no desire to make"},{"line_number":101,"context_line":"customizable the number of host CPUs running emulator threads since"}],"source_content_type":"text/x-rst","patch_set":1,"id":"5f4e5783_ace4e30d","line":98,"range":{"start_line":97,"start_character":0,"end_line":98,"end_character":20},"updated":"2017-10-17 04:57:24.000000000","message":"This sentence is not grammatical.","commit_id":"b9604184a9c79353a57d3ac5250d7be9989350c4"},{"author":{"_account_id":7730,"name":"Sahid Orentino Ferdjaoui","email":"sahid.ferdjaoui@industrialdiscipline.com","username":"sahid"},"change_message_id":"e82791525d27ad41ed357aa3e496bf20998e840a","unresolved":false,"context_lines":[{"line_number":94,"context_line":""},{"line_number":95,"context_line":"* hw:cpu_emulator_threads\u003disolate"},{"line_number":96,"context_line":""},{"line_number":97,"context_line":"Would say that this instance is to be considered to consume 1"},{"line_number":98,"context_line":"additional host CPU. That pCPU used to make running emulator threads"},{"line_number":99,"context_line":"is going to always be configured on the related guest NUMA node ID 0,"},{"line_number":100,"context_line":"to make it predictable for users. Currently there is no desire to make"},{"line_number":101,"context_line":"customizable the number of host CPUs running emulator threads since"}],"source_content_type":"text/x-rst","patch_set":1,"id":"3f4b6375_b1af3e93","line":98,"range":{"start_line":97,"start_character":0,"end_line":98,"end_character":20},"in_reply_to":"5f4e5783_0d051e11","updated":"2017-10-19 07:32:18.000000000","message":"This part has already been approved actually. I would update any of the suggestions you can give to me.","commit_id":"b9604184a9c79353a57d3ac5250d7be9989350c4"},{"author":{"_account_id":11604,"name":"sean mooney","email":"smooney@redhat.com","username":"sean-k-mooney"},"change_message_id":"7b69fedbd6fac8d0ba9668cd7614ce12c491c53d","unresolved":false,"context_lines":[{"line_number":94,"context_line":""},{"line_number":95,"context_line":"* hw:cpu_emulator_threads\u003disolate"},{"line_number":96,"context_line":""},{"line_number":97,"context_line":"Would say that this instance is to be considered to consume 1"},{"line_number":98,"context_line":"additional host CPU. That pCPU used to make running emulator threads"},{"line_number":99,"context_line":"is going to always be configured on the related guest NUMA node ID 0,"},{"line_number":100,"context_line":"to make it predictable for users. Currently there is no desire to make"},{"line_number":101,"context_line":"customizable the number of host CPUs running emulator threads since"}],"source_content_type":"text/x-rst","patch_set":1,"id":"5f4e5783_0d051e11","line":98,"range":{"start_line":97,"start_character":0,"end_line":98,"end_character":20},"in_reply_to":"5f4e5783_ace4e30d","updated":"2017-10-17 20:02:04.000000000","message":"its a run on form before the example.\nso it should start with \"which says that ...\"","commit_id":"b9604184a9c79353a57d3ac5250d7be9989350c4"},{"author":{"_account_id":11604,"name":"sean mooney","email":"smooney@redhat.com","username":"sean-k-mooney"},"change_message_id":"7b69fedbd6fac8d0ba9668cd7614ce12c491c53d","unresolved":false,"context_lines":[{"line_number":96,"context_line":""},{"line_number":97,"context_line":"Would say that this instance is to be considered to consume 1"},{"line_number":98,"context_line":"additional host CPU. That pCPU used to make running emulator threads"},{"line_number":99,"context_line":"is going to always be configured on the related guest NUMA node ID 0,"},{"line_number":100,"context_line":"to make it predictable for users. Currently there is no desire to make"},{"line_number":101,"context_line":"customizable the number of host CPUs running emulator threads since"},{"line_number":102,"context_line":"only one should work for almost every use case. If in the future there"},{"line_number":103,"context_line":"is a desire to isolate more than one host CPU to run emulator threads,"}],"source_content_type":"text/x-rst","patch_set":1,"id":"5f4e5783_2dec5a28","line":100,"range":{"start_line":99,"start_character":0,"end_line":100,"end_character":32},"updated":"2017-10-17 20:02:04.000000000","message":"users should not care or know this low-level behaviour.\n\nthe only numa affinity we should provide is that if the vm has a numa topology then the emulator thread will be form associated with one of the numa nodes of the guest.","commit_id":"b9604184a9c79353a57d3ac5250d7be9989350c4"},{"author":{"_account_id":11604,"name":"sean mooney","email":"smooney@redhat.com","username":"sean-k-mooney"},"change_message_id":"371177e0d488e48a2bc0c3dc3136b0fa72555d51","unresolved":false,"context_lines":[{"line_number":97,"context_line":"Would say that this instance is to be considered to consume 1"},{"line_number":98,"context_line":"additional host CPU. That pCPU used to make running emulator threads"},{"line_number":99,"context_line":"is going to always be configured on the related guest NUMA node ID 0,"},{"line_number":100,"context_line":"to make it predictable for users. Currently there is no desire to make"},{"line_number":101,"context_line":"customizable the number of host CPUs running emulator threads since"},{"line_number":102,"context_line":"only one should work for almost every use case. If in the future there"},{"line_number":103,"context_line":"is a desire to isolate more than one host CPU to run emulator threads,"}],"source_content_type":"text/x-rst","patch_set":1,"id":"3f4b6375_b7aeb27e","line":100,"in_reply_to":"","updated":"2017-11-02 04:12:56.000000000","message":"Can you link to where that was approved because this will limit scheduling efficiency in any case where guest cpus or ram topology are not systematic","commit_id":"b9604184a9c79353a57d3ac5250d7be9989350c4"},{"author":{"_account_id":11604,"name":"sean mooney","email":"smooney@redhat.com","username":"sean-k-mooney"},"change_message_id":"0bef9802555c7644c10405ec4c654d31d6a23134","unresolved":false,"context_lines":[{"line_number":97,"context_line":"Would say that this instance is to be considered to consume 1"},{"line_number":98,"context_line":"additional host CPU. That pCPU used to make running emulator threads"},{"line_number":99,"context_line":"is going to always be configured on the related guest NUMA node ID 0,"},{"line_number":100,"context_line":"to make it predictable for users. Currently there is no desire to make"},{"line_number":101,"context_line":"customizable the number of host CPUs running emulator threads since"},{"line_number":102,"context_line":"only one should work for almost every use case. If in the future there"},{"line_number":103,"context_line":"is a desire to isolate more than one host CPU to run emulator threads,"}],"source_content_type":"text/x-rst","patch_set":1,"id":"3f4b6375_67a90fde","line":100,"in_reply_to":"","updated":"2017-10-19 02:25:57.000000000","message":"Qemu does not support multiple pci roots in the guest so all guest devices will be associated with guest numa node 0. That said this should work for guests that are not numa affined also. It will eork best if they have numa affinity but it should not be a requirement","commit_id":"b9604184a9c79353a57d3ac5250d7be9989350c4"},{"author":{"_account_id":7730,"name":"Sahid Orentino Ferdjaoui","email":"sahid.ferdjaoui@industrialdiscipline.com","username":"sahid"},"change_message_id":"e82791525d27ad41ed357aa3e496bf20998e840a","unresolved":false,"context_lines":[{"line_number":97,"context_line":"Would say that this instance is to be considered to consume 1"},{"line_number":98,"context_line":"additional host CPU. That pCPU used to make running emulator threads"},{"line_number":99,"context_line":"is going to always be configured on the related guest NUMA node ID 0,"},{"line_number":100,"context_line":"to make it predictable for users. Currently there is no desire to make"},{"line_number":101,"context_line":"customizable the number of host CPUs running emulator threads since"},{"line_number":102,"context_line":"only one should work for almost every use case. If in the future there"},{"line_number":103,"context_line":"is a desire to isolate more than one host CPU to run emulator threads,"}],"source_content_type":"text/x-rst","patch_set":1,"id":"3f4b6375_b1885e1f","line":100,"in_reply_to":"3f4b6375_67a90fde","updated":"2017-10-19 07:32:18.000000000","message":"It\u0027s an important technical detail for end-users. That is said. It\u0027s already approved and implemented I\u0027m not going to change that part except if you propose to me a better phrasing","commit_id":"b9604184a9c79353a57d3ac5250d7be9989350c4"},{"author":{"_account_id":8768,"name":"Chris Friesen","email":"chris.friesen@windriver.com","username":"cbf123"},"change_message_id":"85703132c3465b96352864a3103749b80959aa8a","unresolved":false,"context_lines":[{"line_number":97,"context_line":"Would say that this instance is to be considered to consume 1"},{"line_number":98,"context_line":"additional host CPU. That pCPU used to make running emulator threads"},{"line_number":99,"context_line":"is going to always be configured on the related guest NUMA node ID 0,"},{"line_number":100,"context_line":"to make it predictable for users. Currently there is no desire to make"},{"line_number":101,"context_line":"customizable the number of host CPUs running emulator threads since"},{"line_number":102,"context_line":"only one should work for almost every use case. If in the future there"},{"line_number":103,"context_line":"is a desire to isolate more than one host CPU to run emulator threads,"}],"source_content_type":"text/x-rst","patch_set":1,"id":"3f4b6375_63006702","line":100,"in_reply_to":"3f4b6375_b7aeb27e","updated":"2017-11-02 18:26:11.000000000","message":"Sean...the previous version of the spec was approved for Ocata.","commit_id":"b9604184a9c79353a57d3ac5250d7be9989350c4"},{"author":{"_account_id":8768,"name":"Chris Friesen","email":"chris.friesen@windriver.com","username":"cbf123"},"change_message_id":"5c9714b8dafb640c025c3bc46b66a1f6bb9724fe","unresolved":false,"context_lines":[{"line_number":96,"context_line":""},{"line_number":97,"context_line":"Would say that this instance is to be considered to consume 1"},{"line_number":98,"context_line":"additional host CPU. That pCPU used to make running emulator threads"},{"line_number":99,"context_line":"is going to always be configured on the related guest NUMA node ID 0,"},{"line_number":100,"context_line":"to make it predictable for users. Currently there is no desire to make"},{"line_number":101,"context_line":"customizable the number of host CPUs running emulator threads since"},{"line_number":102,"context_line":"only one should work for almost every use case. If in the future there"},{"line_number":103,"context_line":"is a desire to isolate more than one host CPU to run emulator threads,"}],"source_content_type":"text/x-rst","patch_set":1,"id":"3f4b6375_d655221f","line":100,"range":{"start_line":99,"start_character":0,"end_line":100,"end_character":32},"in_reply_to":"5f4e5783_2dec5a28","updated":"2017-10-18 15:49:56.000000000","message":"I think it\u0027s valid to expose this, because it has implications on which guest NUMA node should be used for I/O.  (If I drive I/O in the guest from one NUMA node but my emulator I/O threads are on another NUMA node that\u0027s going to cause cross-NUMA traffic.)\n\nAlso, I suspect many guests will default to using CPU 0 as a non-realtime CPU, in which case they\u0027re likely going to be driving I/O from CPU 0.","commit_id":"b9604184a9c79353a57d3ac5250d7be9989350c4"},{"author":{"_account_id":11604,"name":"sean mooney","email":"smooney@redhat.com","username":"sean-k-mooney"},"change_message_id":"7b69fedbd6fac8d0ba9668cd7614ce12c491c53d","unresolved":false,"context_lines":[{"line_number":104,"context_line":"we would implement instead I/O threads to add granularity on"},{"line_number":105,"context_line":"dedicating used resources to run guests on host CPUs."},{"line_number":106,"context_line":""},{"line_number":107,"context_line":"As we said an additional pCPU is going to be consumed but this first"},{"line_number":108,"context_line":"implementation is not going to update the user quotas, that in a"},{"line_number":109,"context_line":"spirit of simplicity since quotas already leak on different scenarios."},{"line_number":110,"context_line":""},{"line_number":111,"context_line":"To extend the flexibility and address use-cases where resources on"},{"line_number":112,"context_line":"hosts are limited. We are introducing \u0027overhead_pin_set\u0027 option on"}],"source_content_type":"text/x-rst","patch_set":1,"id":"5f4e5783_6d5372e3","line":109,"range":{"start_line":107,"start_character":1,"end_line":109,"end_character":70},"updated":"2017-10-17 20:02:04.000000000","message":"in the host case i think that is ok. in the isolate case we should be charging the guest for the extra cpu we are allocating for them.","commit_id":"b9604184a9c79353a57d3ac5250d7be9989350c4"},{"author":{"_account_id":11604,"name":"sean mooney","email":"smooney@redhat.com","username":"sean-k-mooney"},"change_message_id":"371177e0d488e48a2bc0c3dc3136b0fa72555d51","unresolved":false,"context_lines":[{"line_number":106,"context_line":""},{"line_number":107,"context_line":"As we said an additional pCPU is going to be consumed but this first"},{"line_number":108,"context_line":"implementation is not going to update the user quotas, that in a"},{"line_number":109,"context_line":"spirit of simplicity since quotas already leak on different scenarios."},{"line_number":110,"context_line":""},{"line_number":111,"context_line":"To extend the flexibility and address use-cases where resources on"},{"line_number":112,"context_line":"hosts are limited. We are introducing \u0027overhead_pin_set\u0027 option on"}],"source_content_type":"text/x-rst","patch_set":1,"id":"3f4b6375_970f3687","line":109,"in_reply_to":"3f4b6375_71c8a654","updated":"2017-11-02 04:12:56.000000000","message":"Well this spec has not been merge yet so it has not been approved","commit_id":"b9604184a9c79353a57d3ac5250d7be9989350c4"},{"author":{"_account_id":7730,"name":"Sahid Orentino Ferdjaoui","email":"sahid.ferdjaoui@industrialdiscipline.com","username":"sahid"},"change_message_id":"e82791525d27ad41ed357aa3e496bf20998e840a","unresolved":false,"context_lines":[{"line_number":104,"context_line":"we would implement instead I/O threads to add granularity on"},{"line_number":105,"context_line":"dedicating used resources to run guests on host CPUs."},{"line_number":106,"context_line":""},{"line_number":107,"context_line":"As we said an additional pCPU is going to be consumed but this first"},{"line_number":108,"context_line":"implementation is not going to update the user quotas, that in a"},{"line_number":109,"context_line":"spirit of simplicity since quotas already leak on different scenarios."},{"line_number":110,"context_line":""},{"line_number":111,"context_line":"To extend the flexibility and address use-cases where resources on"},{"line_number":112,"context_line":"hosts are limited. We are introducing \u0027overhead_pin_set\u0027 option on"}],"source_content_type":"text/x-rst","patch_set":1,"id":"3f4b6375_71c8a654","line":109,"range":{"start_line":107,"start_character":1,"end_line":109,"end_character":70},"in_reply_to":"5f4e5783_6d5372e3","updated":"2017-10-19 07:32:18.000000000","message":"That has already been discussed and approved. Thanks for your suggestion.","commit_id":"b9604184a9c79353a57d3ac5250d7be9989350c4"},{"author":{"_account_id":8768,"name":"Chris Friesen","email":"chris.friesen@windriver.com","username":"cbf123"},"change_message_id":"e4ba9a012ceedb0467f202082f6e7802273620ed","unresolved":false,"context_lines":[{"line_number":108,"context_line":"implementation is not going to update the user quotas, that in a"},{"line_number":109,"context_line":"spirit of simplicity since quotas already leak on different scenarios."},{"line_number":110,"context_line":""},{"line_number":111,"context_line":"To extend the flexibility and address use-cases where resources on"},{"line_number":112,"context_line":"hosts are limited. We are introducing \u0027overhead_pin_set\u0027 option on"},{"line_number":113,"context_line":"compute node. Operators will have ability to reserve host CPUs for"},{"line_number":114,"context_line":"hypervisor overhead."},{"line_number":115,"context_line":""},{"line_number":116,"context_line":"* hw:cpu_emulator_threads\u003dhost"}],"source_content_type":"text/x-rst","patch_set":1,"id":"5f4e5783_ec853b08","line":113,"range":{"start_line":111,"start_character":0,"end_line":113,"end_character":12},"updated":"2017-10-17 04:57:24.000000000","message":"This should be one sentence, and you should probably describe what \"overhead_pin_set\" represents.  Presumably it\u0027s a set of pCPUs described in the same syntax as \"cpu_pin\"set?\n\nAlso, I think \"emulator_pin_set\" would make more sense.\n\nWhat happens if someone modifies the pin set in the config file and restarts nova-compute when there are running instances using the \"old\" value?  Would we leave them as-is or verify the values at nova-compute startup and re-affine them as necessary?","commit_id":"b9604184a9c79353a57d3ac5250d7be9989350c4"},{"author":{"_account_id":7730,"name":"Sahid Orentino Ferdjaoui","email":"sahid.ferdjaoui@industrialdiscipline.com","username":"sahid"},"change_message_id":"04c9212578350426ae49df4f573bc670df4c201d","unresolved":false,"context_lines":[{"line_number":108,"context_line":"implementation is not going to update the user quotas, that in a"},{"line_number":109,"context_line":"spirit of simplicity since quotas already leak on different scenarios."},{"line_number":110,"context_line":""},{"line_number":111,"context_line":"To extend the flexibility and address use-cases where resources on"},{"line_number":112,"context_line":"hosts are limited. We are introducing \u0027overhead_pin_set\u0027 option on"},{"line_number":113,"context_line":"compute node. Operators will have ability to reserve host CPUs for"},{"line_number":114,"context_line":"hypervisor overhead."},{"line_number":115,"context_line":""},{"line_number":116,"context_line":"* hw:cpu_emulator_threads\u003dhost"}],"source_content_type":"text/x-rst","patch_set":1,"id":"3f4b6375_6e1e75bb","line":113,"range":{"start_line":111,"start_character":0,"end_line":113,"end_character":12},"in_reply_to":"3f4b6375_1ab85f74","updated":"2017-10-19 18:15:44.000000000","message":"Thanks for this Chris. I updated the spec using your suggestion. I have also added a note to make clear that the option introduced will be a subset of the CPUs available for Nova by vcpu_pin_set (if defined).\n\nAbout the name of the option - I\u0027m not totally convinced by hypervisor_pin_set so I did not changed it. But if we can have a third person to take the final decision I will update the spec. Perhaps Jay or Sean. Seems fair for you?","commit_id":"b9604184a9c79353a57d3ac5250d7be9989350c4"},{"author":{"_account_id":8768,"name":"Chris Friesen","email":"chris.friesen@windriver.com","username":"cbf123"},"change_message_id":"fd4a6cb72364ddbda8ddf7d950651cbe4b34472d","unresolved":false,"context_lines":[{"line_number":108,"context_line":"implementation is not going to update the user quotas, that in a"},{"line_number":109,"context_line":"spirit of simplicity since quotas already leak on different scenarios."},{"line_number":110,"context_line":""},{"line_number":111,"context_line":"To extend the flexibility and address use-cases where resources on"},{"line_number":112,"context_line":"hosts are limited. We are introducing \u0027overhead_pin_set\u0027 option on"},{"line_number":113,"context_line":"compute node. Operators will have ability to reserve host CPUs for"},{"line_number":114,"context_line":"hypervisor overhead."},{"line_number":115,"context_line":""},{"line_number":116,"context_line":"* hw:cpu_emulator_threads\u003dhost"}],"source_content_type":"text/x-rst","patch_set":1,"id":"3f4b6375_1ab85f74","line":113,"range":{"start_line":111,"start_character":0,"end_line":113,"end_character":12},"in_reply_to":"3f4b6375_b1129eb0","updated":"2017-10-19 15:41:58.000000000","message":"\"overhead\" just seems overly generic.  I think \"hypervisor_pin_set\" would be more specific, but I can live with it either way.\n\nI think the new section would be clearer if written something like this:\n\nTo extend the flexibility and address use-cases where resources on hosts are limited, we are adding the ability for operators to reserve a set of host CPUs for running hypervisor work (emulator threads, I/O threads, etc.).  These host CPUs would be shared by all instances configured to use them (as opposed to reserving a whole extra CPU per instance with the \"isolate\" option.)\n\nThe reserved host CPUs are specified via the \u0027hypervisor_pin_set\u0027 config option.  This is analogous to the existing \"vcpu_pin_set\" and is formatted the same way.\n\nIn order to run the hypervisor work on these reserved host CPUs, the admin must set \"hw:cpu_emulator_threads\u003dhost\" in the flavor extra-spec.","commit_id":"b9604184a9c79353a57d3ac5250d7be9989350c4"},{"author":{"_account_id":11604,"name":"sean mooney","email":"smooney@redhat.com","username":"sean-k-mooney"},"change_message_id":"7b69fedbd6fac8d0ba9668cd7614ce12c491c53d","unresolved":false,"context_lines":[{"line_number":108,"context_line":"implementation is not going to update the user quotas, that in a"},{"line_number":109,"context_line":"spirit of simplicity since quotas already leak on different scenarios."},{"line_number":110,"context_line":""},{"line_number":111,"context_line":"To extend the flexibility and address use-cases where resources on"},{"line_number":112,"context_line":"hosts are limited. We are introducing \u0027overhead_pin_set\u0027 option on"},{"line_number":113,"context_line":"compute node. Operators will have ability to reserve host CPUs for"},{"line_number":114,"context_line":"hypervisor overhead."},{"line_number":115,"context_line":""},{"line_number":116,"context_line":"* hw:cpu_emulator_threads\u003dhost"}],"source_content_type":"text/x-rst","patch_set":1,"id":"5f4e5783_eddbe24b","line":113,"range":{"start_line":111,"start_character":0,"end_line":113,"end_character":12},"in_reply_to":"5f4e5783_ec853b08","updated":"2017-10-17 20:02:04.000000000","message":"emulator_pin_set would not be as hypervisor agnostic but i do prefer that term also.\n\nas for updates that is a good question. i want to say we would update them but i think for simplicity we would not.\n\n\ninstead leave it to be corrected when the vm is rebooted and document that for the deplorer. \n\nas a guidline i think we should treat it the same as what we do if we update the vcpu_pin_set.","commit_id":"b9604184a9c79353a57d3ac5250d7be9989350c4"},{"author":{"_account_id":7730,"name":"Sahid Orentino Ferdjaoui","email":"sahid.ferdjaoui@industrialdiscipline.com","username":"sahid"},"change_message_id":"e82791525d27ad41ed357aa3e496bf20998e840a","unresolved":false,"context_lines":[{"line_number":108,"context_line":"implementation is not going to update the user quotas, that in a"},{"line_number":109,"context_line":"spirit of simplicity since quotas already leak on different scenarios."},{"line_number":110,"context_line":""},{"line_number":111,"context_line":"To extend the flexibility and address use-cases where resources on"},{"line_number":112,"context_line":"hosts are limited. We are introducing \u0027overhead_pin_set\u0027 option on"},{"line_number":113,"context_line":"compute node. Operators will have ability to reserve host CPUs for"},{"line_number":114,"context_line":"hypervisor overhead."},{"line_number":115,"context_line":""},{"line_number":116,"context_line":"* hw:cpu_emulator_threads\u003dhost"}],"source_content_type":"text/x-rst","patch_set":1,"id":"3f4b6375_b1129eb0","line":113,"range":{"start_line":111,"start_character":0,"end_line":113,"end_character":12},"in_reply_to":"5f4e5783_eddbe24b","updated":"2017-10-19 07:32:18.000000000","message":"I choose \u0027overhead_pin_set\u0027 because it refers to vcpu_pin_set and also refers to the name we use to refer instances overhead (see: driver.estimate_instance_overhead()).\n\n\u0027emulator_pin_set\u0027 would be a specific config for libvirt, it\u0027s not what we want. The code to handle the *pin_set option are going to live in /virt/hardware/ so any of the other drivers can take advantage of it.\n\nYou can see the implementation here [0].\n\nIf you have a better name I can still update it but that one looks to keep coherence with what we have.\n\n[0] https://review.openstack.org/#/c/510897/","commit_id":"b9604184a9c79353a57d3ac5250d7be9989350c4"},{"author":{"_account_id":11604,"name":"sean mooney","email":"smooney@redhat.com","username":"sean-k-mooney"},"change_message_id":"7b69fedbd6fac8d0ba9668cd7614ce12c491c53d","unresolved":false,"context_lines":[{"line_number":130,"context_line":"are used, it will be critical to separate emulator threads for"},{"line_number":131,"context_line":"different KVM instances."},{"line_number":132,"context_line":""},{"line_number":133,"context_line":"Another option is to hardcode an assumption that the vCPUs number set"},{"line_number":134,"context_line":"against the flavour implicitly includes 1 vCPUs for emulator. eg a"},{"line_number":135,"context_line":"vCPU value of 5 would imply 4 actual vCPUs and 1 system pseudo-vCPU."},{"line_number":136,"context_line":"This would likely be extremely confusing to tenant users, and"},{"line_number":137,"context_line":"developers alike."},{"line_number":138,"context_line":""},{"line_number":139,"context_line":"Do nothing is always an option. If we did nothing, then it would limit"},{"line_number":140,"context_line":"the types of workload that can be run on Nova. This would have a"}],"source_content_type":"text/x-rst","patch_set":1,"id":"5f4e5783_080ceca2","line":137,"range":{"start_line":133,"start_character":0,"end_line":137,"end_character":17},"updated":"2017-10-17 20:02:04.000000000","message":"on hypervisors like hyperv or with nova lxd it would also make no sense","commit_id":"b9604184a9c79353a57d3ac5250d7be9989350c4"},{"author":{"_account_id":8768,"name":"Chris Friesen","email":"chris.friesen@windriver.com","username":"cbf123"},"change_message_id":"e4ba9a012ceedb0467f202082f6e7802273620ed","unresolved":false,"context_lines":[{"line_number":154,"context_line":""},{"line_number":155,"context_line":"* emulator_threads_policy\u003dCPUEmulatorThreadsPolicy()"},{"line_number":156,"context_line":""},{"line_number":157,"context_line":"This field will be implemented as an enum with two options:"},{"line_number":158,"context_line":""},{"line_number":159,"context_line":"* share - The emulator threads float across the pCPUs associated to"},{"line_number":160,"context_line":"  the guest."}],"source_content_type":"text/x-rst","patch_set":1,"id":"5f4e5783_0c989767","line":157,"range":{"start_line":157,"start_character":47,"end_line":157,"end_character":50},"updated":"2017-10-17 04:57:24.000000000","message":"three options","commit_id":"b9604184a9c79353a57d3ac5250d7be9989350c4"},{"author":{"_account_id":7730,"name":"Sahid Orentino Ferdjaoui","email":"sahid.ferdjaoui@industrialdiscipline.com","username":"sahid"},"change_message_id":"e82791525d27ad41ed357aa3e496bf20998e840a","unresolved":false,"context_lines":[{"line_number":154,"context_line":""},{"line_number":155,"context_line":"* emulator_threads_policy\u003dCPUEmulatorThreadsPolicy()"},{"line_number":156,"context_line":""},{"line_number":157,"context_line":"This field will be implemented as an enum with two options:"},{"line_number":158,"context_line":""},{"line_number":159,"context_line":"* share - The emulator threads float across the pCPUs associated to"},{"line_number":160,"context_line":"  the guest."}],"source_content_type":"text/x-rst","patch_set":1,"id":"3f4b6375_117972fb","line":157,"range":{"start_line":157,"start_character":47,"end_line":157,"end_character":50},"in_reply_to":"5f4e5783_0c989767","updated":"2017-10-19 07:32:18.000000000","message":"Thanks, nice catch.","commit_id":"b9604184a9c79353a57d3ac5250d7be9989350c4"},{"author":{"_account_id":8768,"name":"Chris Friesen","email":"chris.friesen@windriver.com","username":"cbf123"},"change_message_id":"e4ba9a012ceedb0467f202082f6e7802273620ed","unresolved":false,"context_lines":[{"line_number":162,"context_line":"* host - The emulator threads are isolated on the pCPUs set configure"},{"line_number":163,"context_line":"         with \u0027overhead_pin_set\u0027"},{"line_number":164,"context_line":""},{"line_number":165,"context_line":"By default \u0027shared\u0027 will be used. It\u0027s important to note that: Since"},{"line_number":166,"context_line":"[1] on kernel the load-balancing on CPU isolated from the kernel"},{"line_number":167,"context_line":"command line using \u0027isolcpus\u003d\u0027 has been removed. It means that the"},{"line_number":168,"context_line":"emulator threads are not going to float on the union of pCPUs"},{"line_number":169,"context_line":"dedicated to the guest but instead be constrained to the pCPU running"},{"line_number":170,"context_line":"vCPU 0."}],"source_content_type":"text/x-rst","patch_set":1,"id":"5f4e5783_8cf24799","line":167,"range":{"start_line":165,"start_character":63,"end_line":167,"end_character":48},"updated":"2017-10-17 04:57:24.000000000","message":"This is a bit unclear as written.  Maybe something like:\n\nSince [1] was merged into the kernel source, any CPU isolated from the scheduler using \"isolcpus\u003d\" on the kernel boot args will not be load-balanced by the kernel scheduler.","commit_id":"b9604184a9c79353a57d3ac5250d7be9989350c4"},{"author":{"_account_id":7730,"name":"Sahid Orentino Ferdjaoui","email":"sahid.ferdjaoui@industrialdiscipline.com","username":"sahid"},"change_message_id":"e82791525d27ad41ed357aa3e496bf20998e840a","unresolved":false,"context_lines":[{"line_number":162,"context_line":"* host - The emulator threads are isolated on the pCPUs set configure"},{"line_number":163,"context_line":"         with \u0027overhead_pin_set\u0027"},{"line_number":164,"context_line":""},{"line_number":165,"context_line":"By default \u0027shared\u0027 will be used. It\u0027s important to note that: Since"},{"line_number":166,"context_line":"[1] on kernel the load-balancing on CPU isolated from the kernel"},{"line_number":167,"context_line":"command line using \u0027isolcpus\u003d\u0027 has been removed. It means that the"},{"line_number":168,"context_line":"emulator threads are not going to float on the union of pCPUs"},{"line_number":169,"context_line":"dedicated to the guest but instead be constrained to the pCPU running"},{"line_number":170,"context_line":"vCPU 0."}],"source_content_type":"text/x-rst","patch_set":1,"id":"3f4b6375_31fa2e42","line":167,"range":{"start_line":165,"start_character":63,"end_line":167,"end_character":48},"in_reply_to":"5f4e5783_4870a408","updated":"2017-10-19 07:32:18.000000000","message":"Thanks I appreciate to see suggestion. That is said that part has already been approved but if we can have something better I\u0027m OK to update.","commit_id":"b9604184a9c79353a57d3ac5250d7be9989350c4"},{"author":{"_account_id":11604,"name":"sean mooney","email":"smooney@redhat.com","username":"sean-k-mooney"},"change_message_id":"7b69fedbd6fac8d0ba9668cd7614ce12c491c53d","unresolved":false,"context_lines":[{"line_number":162,"context_line":"* host - The emulator threads are isolated on the pCPUs set configure"},{"line_number":163,"context_line":"         with \u0027overhead_pin_set\u0027"},{"line_number":164,"context_line":""},{"line_number":165,"context_line":"By default \u0027shared\u0027 will be used. It\u0027s important to note that: Since"},{"line_number":166,"context_line":"[1] on kernel the load-balancing on CPU isolated from the kernel"},{"line_number":167,"context_line":"command line using \u0027isolcpus\u003d\u0027 has been removed. It means that the"},{"line_number":168,"context_line":"emulator threads are not going to float on the union of pCPUs"},{"line_number":169,"context_line":"dedicated to the guest but instead be constrained to the pCPU running"},{"line_number":170,"context_line":"vCPU 0."}],"source_content_type":"text/x-rst","patch_set":1,"id":"5f4e5783_4870a408","line":167,"range":{"start_line":165,"start_character":63,"end_line":167,"end_character":48},"in_reply_to":"5f4e5783_8cf24799","updated":"2017-10-17 20:02:04.000000000","message":"huh 2015 i taught that was disabled much farther back then that.","commit_id":"b9604184a9c79353a57d3ac5250d7be9989350c4"},{"author":{"_account_id":11604,"name":"sean mooney","email":"smooney@redhat.com","username":"sean-k-mooney"},"change_message_id":"7b69fedbd6fac8d0ba9668cd7614ce12c491c53d","unresolved":false,"context_lines":[{"line_number":165,"context_line":"By default \u0027shared\u0027 will be used. It\u0027s important to note that: Since"},{"line_number":166,"context_line":"[1] on kernel the load-balancing on CPU isolated from the kernel"},{"line_number":167,"context_line":"command line using \u0027isolcpus\u003d\u0027 has been removed. It means that the"},{"line_number":168,"context_line":"emulator threads are not going to float on the union of pCPUs"},{"line_number":169,"context_line":"dedicated to the guest but instead be constrained to the pCPU running"},{"line_number":170,"context_line":"vCPU 0."},{"line_number":171,"context_line":""},{"line_number":172,"context_line":"The InstanceNUMACell object will be extended to have a new field where"},{"line_number":173,"context_line":"physical CPUs ID will be stored and used by the driver layer to pin"}],"source_content_type":"text/x-rst","patch_set":1,"id":"5f4e5783_e3a9b38f","line":170,"range":{"start_line":168,"start_character":0,"end_line":170,"end_character":6},"updated":"2017-10-17 20:02:04.000000000","message":"that is only true if you use isolcpus to isolate the cpus specifed in the nova vcpu_pin_set.\n\nnova does not isolate cpus form the kernel so this section in not really relevent to this spec though people should be aware of this behaviour.\n\ni would remove everything afer \"By default \u0027shared\u0027 will be used.\" or move it to a note in other deployer impact as it has nothing to do with the data model.","commit_id":"b9604184a9c79353a57d3ac5250d7be9989350c4"},{"author":{"_account_id":11604,"name":"sean mooney","email":"smooney@redhat.com","username":"sean-k-mooney"},"change_message_id":"371177e0d488e48a2bc0c3dc3136b0fa72555d51","unresolved":false,"context_lines":[{"line_number":167,"context_line":"command line using \u0027isolcpus\u003d\u0027 has been removed. It means that the"},{"line_number":168,"context_line":"emulator threads are not going to float on the union of pCPUs"},{"line_number":169,"context_line":"dedicated to the guest but instead be constrained to the pCPU running"},{"line_number":170,"context_line":"vCPU 0."},{"line_number":171,"context_line":""},{"line_number":172,"context_line":"The InstanceNUMACell object will be extended to have a new field where"},{"line_number":173,"context_line":"physical CPUs ID will be stored and used by the driver layer to pin"}],"source_content_type":"text/x-rst","patch_set":1,"id":"3f4b6375_174906b6","line":170,"in_reply_to":"","updated":"2017-11-02 04:12:56.000000000","message":"This is completely misleading though this has to be removed or this spec should not be merged","commit_id":"b9604184a9c79353a57d3ac5250d7be9989350c4"},{"author":{"_account_id":7730,"name":"Sahid Orentino Ferdjaoui","email":"sahid.ferdjaoui@industrialdiscipline.com","username":"sahid"},"change_message_id":"e82791525d27ad41ed357aa3e496bf20998e840a","unresolved":false,"context_lines":[{"line_number":165,"context_line":"By default \u0027shared\u0027 will be used. It\u0027s important to note that: Since"},{"line_number":166,"context_line":"[1] on kernel the load-balancing on CPU isolated from the kernel"},{"line_number":167,"context_line":"command line using \u0027isolcpus\u003d\u0027 has been removed. It means that the"},{"line_number":168,"context_line":"emulator threads are not going to float on the union of pCPUs"},{"line_number":169,"context_line":"dedicated to the guest but instead be constrained to the pCPU running"},{"line_number":170,"context_line":"vCPU 0."},{"line_number":171,"context_line":""},{"line_number":172,"context_line":"The InstanceNUMACell object will be extended to have a new field where"},{"line_number":173,"context_line":"physical CPUs ID will be stored and used by the driver layer to pin"}],"source_content_type":"text/x-rst","patch_set":1,"id":"3f4b6375_91b1e292","line":170,"range":{"start_line":168,"start_character":0,"end_line":170,"end_character":6},"in_reply_to":"5f4e5783_e3a9b38f","updated":"2017-10-19 07:32:18.000000000","message":"I\u0027m not going to remove this the use-cases was clear. Also it\u0027s already approved and give some details, the reader can just pass on them.","commit_id":"b9604184a9c79353a57d3ac5250d7be9989350c4"},{"author":{"_account_id":11604,"name":"sean mooney","email":"smooney@redhat.com","username":"sean-k-mooney"},"change_message_id":"7b69fedbd6fac8d0ba9668cd7614ce12c491c53d","unresolved":false,"context_lines":[{"line_number":173,"context_line":"physical CPUs ID will be stored and used by the driver layer to pin"},{"line_number":174,"context_line":"emulator threads"},{"line_number":175,"context_line":""},{"line_number":176,"context_line":"* cpuset_reserved\u003dSetOfIntegersField(nullable\u003dTrue)"},{"line_number":177,"context_line":""},{"line_number":178,"context_line":""},{"line_number":179,"context_line":"[1] https://kernel.googlesource.com/pub/scm/linux/kernel/git/stable/linux-stable/+/47b8ea7186aae7f474ec4c98f43eaa8da719cd83%5E%21/#F0"}],"source_content_type":"text/x-rst","patch_set":1,"id":"5f4e5783_e3e533dd","line":176,"range":{"start_line":176,"start_character":2,"end_line":176,"end_character":17},"updated":"2017-10-17 20:02:04.000000000","message":"why cpuset_reserved?","commit_id":"b9604184a9c79353a57d3ac5250d7be9989350c4"},{"author":{"_account_id":11604,"name":"sean mooney","email":"smooney@redhat.com","username":"sean-k-mooney"},"change_message_id":"371177e0d488e48a2bc0c3dc3136b0fa72555d51","unresolved":false,"context_lines":[{"line_number":173,"context_line":"physical CPUs ID will be stored and used by the driver layer to pin"},{"line_number":174,"context_line":"emulator threads"},{"line_number":175,"context_line":""},{"line_number":176,"context_line":"* cpuset_reserved\u003dSetOfIntegersField(nullable\u003dTrue)"},{"line_number":177,"context_line":""},{"line_number":178,"context_line":""},{"line_number":179,"context_line":"[1] https://kernel.googlesource.com/pub/scm/linux/kernel/git/stable/linux-stable/+/47b8ea7186aae7f474ec4c98f43eaa8da719cd83%5E%21/#F0"}],"source_content_type":"text/x-rst","patch_set":1,"id":"3f4b6375_97ebb676","line":176,"in_reply_to":"","updated":"2017-11-02 04:12:56.000000000","message":"That is not an adequate response to the question when was it implemented and why was it merged before the spech was approved","commit_id":"b9604184a9c79353a57d3ac5250d7be9989350c4"},{"author":{"_account_id":7730,"name":"Sahid Orentino Ferdjaoui","email":"sahid.ferdjaoui@industrialdiscipline.com","username":"sahid"},"change_message_id":"e82791525d27ad41ed357aa3e496bf20998e840a","unresolved":false,"context_lines":[{"line_number":173,"context_line":"physical CPUs ID will be stored and used by the driver layer to pin"},{"line_number":174,"context_line":"emulator threads"},{"line_number":175,"context_line":""},{"line_number":176,"context_line":"* cpuset_reserved\u003dSetOfIntegersField(nullable\u003dTrue)"},{"line_number":177,"context_line":""},{"line_number":178,"context_line":""},{"line_number":179,"context_line":"[1] https://kernel.googlesource.com/pub/scm/linux/kernel/git/stable/linux-stable/+/47b8ea7186aae7f474ec4c98f43eaa8da719cd83%5E%21/#F0"}],"source_content_type":"text/x-rst","patch_set":1,"id":"3f4b6375_51800a8c","line":176,"range":{"start_line":176,"start_character":2,"end_line":176,"end_character":17},"in_reply_to":"5f4e5783_e3e533dd","updated":"2017-10-19 07:32:18.000000000","message":"Sean, please, it\u0027s already implemented :)","commit_id":"b9604184a9c79353a57d3ac5250d7be9989350c4"},{"author":{"_account_id":11604,"name":"sean mooney","email":"smooney@redhat.com","username":"sean-k-mooney"},"change_message_id":"7b69fedbd6fac8d0ba9668cd7614ce12c491c53d","unresolved":false,"context_lines":[{"line_number":197,"context_line":"Other end user impact"},{"line_number":198,"context_line":"---------------------"},{"line_number":199,"context_line":""},{"line_number":200,"context_line":"For end users, using the option \u0027cpu_emulator_threads\u0027 is going to"},{"line_number":201,"context_line":"consume an additional host CPU on the resources quota regarding the"},{"line_number":202,"context_line":"guest vCPUs allocated."},{"line_number":203,"context_line":""}],"source_content_type":"text/x-rst","patch_set":1,"id":"5f4e5783_43de47da","line":200,"range":{"start_line":200,"start_character":33,"end_line":200,"end_character":53},"updated":"2017-10-17 20:02:04.000000000","message":"cpu_emulator_threads\u003disolate\n\nthe other forms should not consume addtional quota","commit_id":"b9604184a9c79353a57d3ac5250d7be9989350c4"},{"author":{"_account_id":7730,"name":"Sahid Orentino Ferdjaoui","email":"sahid.ferdjaoui@industrialdiscipline.com","username":"sahid"},"change_message_id":"e82791525d27ad41ed357aa3e496bf20998e840a","unresolved":false,"context_lines":[{"line_number":197,"context_line":"Other end user impact"},{"line_number":198,"context_line":"---------------------"},{"line_number":199,"context_line":""},{"line_number":200,"context_line":"For end users, using the option \u0027cpu_emulator_threads\u0027 is going to"},{"line_number":201,"context_line":"consume an additional host CPU on the resources quota regarding the"},{"line_number":202,"context_line":"guest vCPUs allocated."},{"line_number":203,"context_line":""}],"source_content_type":"text/x-rst","patch_set":1,"id":"3f4b6375_915fa240","line":200,"range":{"start_line":200,"start_character":33,"end_line":200,"end_character":53},"in_reply_to":"5f4e5783_43de47da","updated":"2017-10-19 07:32:18.000000000","message":"Good point thanks.","commit_id":"b9604184a9c79353a57d3ac5250d7be9989350c4"},{"author":{"_account_id":9708,"name":"Balazs Gibizer","display_name":"gibi","email":"gibizer@gmail.com","username":"gibi"},"change_message_id":"69729adec243a3622946a4811d69b34493f95d8a","unresolved":false,"context_lines":[{"line_number":128,"context_line":"Would say the emulator threads of all the guests are going to be"},{"line_number":129,"context_line":"pinned on a global isolated host pCPUs according \u0027overhead_pin_set\u0027"},{"line_number":130,"context_line":""},{"line_number":131,"context_line":"NOTE: The \u0027overhead_pin_set\u0027 will be a subset of \u0027vcpu_pin_set\u0027 if"},{"line_number":132,"context_line":"defined or a subset of all host CPUs online."},{"line_number":133,"context_line":""},{"line_number":134,"context_line":""},{"line_number":135,"context_line":"Alternatives"}],"source_content_type":"text/x-rst","patch_set":3,"id":"3f4b6375_35d1dafc","line":132,"range":{"start_line":131,"start_character":0,"end_line":132,"end_character":44},"updated":"2017-10-20 07:41:36.000000000","message":"This is not clear to me. Does overhead_pin_set has to be a subset of vcpu_pin_set? And when the vcpu_pin_set is not defined then the vcpu_pin_set is all the online CPUs by default therefore the overhead_pin_set is a subset of all online CPUs? If yes then I think you should change \u0027will be\u0027 to \u0027has to be\u0027 and drop the second half of the sentence as that is just a consequence not a real option.","commit_id":"275325f393a706fdfc3cd55ff8eea0aa37e69d65"},{"author":{"_account_id":7730,"name":"Sahid Orentino Ferdjaoui","email":"sahid.ferdjaoui@industrialdiscipline.com","username":"sahid"},"change_message_id":"b8edfb621a05507a529e4518f2f582c606566cb0","unresolved":false,"context_lines":[{"line_number":128,"context_line":"Would say the emulator threads of all the guests are going to be"},{"line_number":129,"context_line":"pinned on a global isolated host pCPUs according \u0027overhead_pin_set\u0027"},{"line_number":130,"context_line":""},{"line_number":131,"context_line":"NOTE: The \u0027overhead_pin_set\u0027 will be a subset of \u0027vcpu_pin_set\u0027 if"},{"line_number":132,"context_line":"defined or a subset of all host CPUs online."},{"line_number":133,"context_line":""},{"line_number":134,"context_line":""},{"line_number":135,"context_line":"Alternatives"}],"source_content_type":"text/x-rst","patch_set":3,"id":"3f4b6375_86b2a6e5","line":132,"range":{"start_line":131,"start_character":0,"end_line":132,"end_character":44},"in_reply_to":"3f4b6375_35d1dafc","updated":"2017-10-20 13:21:36.000000000","message":"I will update the note according your suggestion.","commit_id":"275325f393a706fdfc3cd55ff8eea0aa37e69d65"},{"author":{"_account_id":9708,"name":"Balazs Gibizer","display_name":"gibi","email":"gibizer@gmail.com","username":"gibi"},"change_message_id":"69729adec243a3622946a4811d69b34493f95d8a","unresolved":false,"context_lines":[{"line_number":175,"context_line":"* host - The emulator threads are isolated on the pCPUs set configure"},{"line_number":176,"context_line":"         with \u0027overhead_pin_set\u0027"},{"line_number":177,"context_line":""},{"line_number":178,"context_line":"By default \u0027shared\u0027 will be used. It\u0027s important to note that: Since"},{"line_number":179,"context_line":"[1] was merged into the kernel source, any CPU isolated from the"},{"line_number":180,"context_line":"scheduler using \"isolcpus\u003d\" on the kernel boot args will not be"},{"line_number":181,"context_line":"load-balanced by the kernel scheduler. It means that the emulator"},{"line_number":182,"context_line":"threads are not going to float on the union of pCPUs dedicated to the"},{"line_number":183,"context_line":"guest but instead be constrained to the pCPU running vCPU 0."},{"line_number":184,"context_line":""},{"line_number":185,"context_line":"The InstanceNUMACell object will be extended to have a new field where"},{"line_number":186,"context_line":"physical CPUs ID will be stored and used by the driver layer to pin"}],"source_content_type":"text/x-rst","patch_set":3,"id":"3f4b6375_15681e1f","line":183,"range":{"start_line":178,"start_character":0,"end_line":183,"end_character":60},"updated":"2017-10-20 07:41:36.000000000","message":"Does it also means that when the \u0027host\u0027 option is used and the overhead_pin_set is configured to contain more than one pCPUs then those pCPUs should not be part of the isolcpus kernel boot arg as that would prevent the emulator threads to float across the overhead_pin_set pCPUs? Does it contradict with the requirement that the overhead_pin_set has to be a subset of the vcpu_pin_set? (At least we have such deployment where the the pCPUs in the vcpu_pin_set are put into the isolcpu param for performance reasons)","commit_id":"275325f393a706fdfc3cd55ff8eea0aa37e69d65"},{"author":{"_account_id":7730,"name":"Sahid Orentino Ferdjaoui","email":"sahid.ferdjaoui@industrialdiscipline.com","username":"sahid"},"change_message_id":"b8edfb621a05507a529e4518f2f582c606566cb0","unresolved":false,"context_lines":[{"line_number":175,"context_line":"* host - The emulator threads are isolated on the pCPUs set configure"},{"line_number":176,"context_line":"         with \u0027overhead_pin_set\u0027"},{"line_number":177,"context_line":""},{"line_number":178,"context_line":"By default \u0027shared\u0027 will be used. It\u0027s important to note that: Since"},{"line_number":179,"context_line":"[1] was merged into the kernel source, any CPU isolated from the"},{"line_number":180,"context_line":"scheduler using \"isolcpus\u003d\" on the kernel boot args will not be"},{"line_number":181,"context_line":"load-balanced by the kernel scheduler. It means that the emulator"},{"line_number":182,"context_line":"threads are not going to float on the union of pCPUs dedicated to the"},{"line_number":183,"context_line":"guest but instead be constrained to the pCPU running vCPU 0."},{"line_number":184,"context_line":""},{"line_number":185,"context_line":"The InstanceNUMACell object will be extended to have a new field where"},{"line_number":186,"context_line":"physical CPUs ID will be stored and used by the driver layer to pin"}],"source_content_type":"text/x-rst","patch_set":3,"id":"3f4b6375_6ce0a8fb","line":183,"range":{"start_line":178,"start_character":0,"end_line":183,"end_character":60},"in_reply_to":"3f4b6375_15681e1f","updated":"2017-10-20 13:21:36.000000000","message":"It means that if you use a set of pCPUs which are isolated by isolcpus to run emulator threads, the emulthreads are never going to be load balanced.\n\nIf you really want to isolate the pCPUs running emulthreads so you what you want is to use CPUAffinity. That is said for RT It should not be a problem to have emulthreads to contend with other host OS processes.","commit_id":"275325f393a706fdfc3cd55ff8eea0aa37e69d65"},{"author":{"_account_id":9708,"name":"Balazs Gibizer","display_name":"gibi","email":"gibizer@gmail.com","username":"gibi"},"change_message_id":"6c9e23bc8116575278d77110212e0175e6068bea","unresolved":false,"context_lines":[{"line_number":175,"context_line":"* host - The emulator threads are isolated on the pCPUs set configure"},{"line_number":176,"context_line":"         with \u0027overhead_pin_set\u0027"},{"line_number":177,"context_line":""},{"line_number":178,"context_line":"By default \u0027shared\u0027 will be used. It\u0027s important to note that: Since"},{"line_number":179,"context_line":"[1] was merged into the kernel source, any CPU isolated from the"},{"line_number":180,"context_line":"scheduler using \"isolcpus\u003d\" on the kernel boot args will not be"},{"line_number":181,"context_line":"load-balanced by the kernel scheduler. It means that the emulator"},{"line_number":182,"context_line":"threads are not going to float on the union of pCPUs dedicated to the"},{"line_number":183,"context_line":"guest but instead be constrained to the pCPU running vCPU 0."},{"line_number":184,"context_line":""},{"line_number":185,"context_line":"The InstanceNUMACell object will be extended to have a new field where"},{"line_number":186,"context_line":"physical CPUs ID will be stored and used by the driver layer to pin"}],"source_content_type":"text/x-rst","patch_set":3,"id":"3f4b6375_2cd630c5","line":183,"range":{"start_line":178,"start_character":0,"end_line":183,"end_character":60},"in_reply_to":"3f4b6375_6ce0a8fb","updated":"2017-10-20 13:35:59.000000000","message":"Thanks.","commit_id":"275325f393a706fdfc3cd55ff8eea0aa37e69d65"},{"author":{"_account_id":9708,"name":"Balazs Gibizer","display_name":"gibi","email":"gibizer@gmail.com","username":"gibi"},"change_message_id":"69729adec243a3622946a4811d69b34493f95d8a","unresolved":false,"context_lines":[{"line_number":210,"context_line":"Other end user impact"},{"line_number":211,"context_line":"---------------------"},{"line_number":212,"context_line":""},{"line_number":213,"context_line":"For end users, using the option \u0027cpu_emulator_threads\u003disolate\u0027 is"},{"line_number":214,"context_line":"going to consume an additional host CPU on the resources quota"},{"line_number":215,"context_line":"regarding the guest vCPUs allocated."},{"line_number":216,"context_line":""},{"line_number":217,"context_line":"Performance Impact"},{"line_number":218,"context_line":"------------------"}],"source_content_type":"text/x-rst","patch_set":3,"id":"3f4b6375_15513e63","line":215,"range":{"start_line":213,"start_character":0,"end_line":215,"end_character":36},"updated":"2017-10-20 07:41:36.000000000","message":"L107-109 states that the quota will not be updated.","commit_id":"275325f393a706fdfc3cd55ff8eea0aa37e69d65"},{"author":{"_account_id":7730,"name":"Sahid Orentino Ferdjaoui","email":"sahid.ferdjaoui@industrialdiscipline.com","username":"sahid"},"change_message_id":"b8edfb621a05507a529e4518f2f582c606566cb0","unresolved":false,"context_lines":[{"line_number":210,"context_line":"Other end user impact"},{"line_number":211,"context_line":"---------------------"},{"line_number":212,"context_line":""},{"line_number":213,"context_line":"For end users, using the option \u0027cpu_emulator_threads\u003disolate\u0027 is"},{"line_number":214,"context_line":"going to consume an additional host CPU on the resources quota"},{"line_number":215,"context_line":"regarding the guest vCPUs allocated."},{"line_number":216,"context_line":""},{"line_number":217,"context_line":"Performance Impact"},{"line_number":218,"context_line":"------------------"}],"source_content_type":"text/x-rst","patch_set":3,"id":"3f4b6375_4ce72c14","line":215,"range":{"start_line":213,"start_character":0,"end_line":215,"end_character":36},"in_reply_to":"3f4b6375_15513e63","updated":"2017-10-20 13:21:36.000000000","message":"Yes makes sense.","commit_id":"275325f393a706fdfc3cd55ff8eea0aa37e69d65"},{"author":{"_account_id":25625,"name":"Tetsuro Nakamura","email":"tetsuro.nakamura.bc@hco.ntt.co.jp","username":"tetsuro0907"},"change_message_id":"f464363131fce705536c0c572a65a9ec62ff1f19","unresolved":false,"context_lines":[{"line_number":92,"context_line":"isolate emulator threads must use a flavor configured to accept that"},{"line_number":93,"context_line":"specification as:"},{"line_number":94,"context_line":""},{"line_number":95,"context_line":"* hw:cpu_emulator_threads\u003disolate"},{"line_number":96,"context_line":""},{"line_number":97,"context_line":"Would say that this instance is to be considered to consume 1"},{"line_number":98,"context_line":"additional host CPU. That pCPU used to make running emulator threads"}],"source_content_type":"text/x-rst","patch_set":4,"id":"df87a7cf_bbe15401","line":95,"range":{"start_line":95,"start_character":0,"end_line":95,"end_character":33},"updated":"2017-12-03 23:46:48.000000000","message":"We already have this option implemented, so this is not proposed change. We should put description of this feature, the reason why this feature, having 2 options of shared and isolate, is not enough in the previous section of problem description.\n[1]https://github.com/openstack/nova/blob/master/nova/virt/hardware.py#L1274","commit_id":"69d47b455d50e0bd51bc0e6efa0b315374f06470"},{"author":{"_account_id":25625,"name":"Tetsuro Nakamura","email":"tetsuro.nakamura.bc@hco.ntt.co.jp","username":"tetsuro0907"},"change_message_id":"f464363131fce705536c0c572a65a9ec62ff1f19","unresolved":false,"context_lines":[{"line_number":165,"context_line":"The InstanceNUMATopology object will be extended to have a new field"},{"line_number":166,"context_line":"used to store requested policy"},{"line_number":167,"context_line":""},{"line_number":168,"context_line":"* emulator_threads_policy\u003dCPUEmulatorThreadsPolicy()"},{"line_number":169,"context_line":""},{"line_number":170,"context_line":"This field will be implemented as an enum with three options:"},{"line_number":171,"context_line":""}],"source_content_type":"text/x-rst","patch_set":4,"id":"df87a7cf_bbbab4db","line":168,"range":{"start_line":168,"start_character":0,"end_line":168,"end_character":52},"updated":"2017-12-03 23:46:48.000000000","message":"We already have this field implemented , so this new feature of \u0027hw:cpu_emulator_threads\u003dhost\u0027 supplies no such data model impact.\n[2]https://github.com/openstack/nova/blob/master/nova/objects/instance_numa_topology.py#L166","commit_id":"69d47b455d50e0bd51bc0e6efa0b315374f06470"},{"author":{"_account_id":25625,"name":"Tetsuro Nakamura","email":"tetsuro.nakamura.bc@hco.ntt.co.jp","username":"tetsuro0907"},"change_message_id":"f464363131fce705536c0c572a65a9ec62ff1f19","unresolved":false,"context_lines":[{"line_number":178,"context_line":"By default \u0027shared\u0027 will be used. It\u0027s important to note that: Since"},{"line_number":179,"context_line":"[1] was merged into the kernel source, any CPU isolated from the"},{"line_number":180,"context_line":"scheduler using \"isolcpus\u003d\" on the kernel boot args will not be"},{"line_number":181,"context_line":"load-balanced by the kernel scheduler. It means that the emulator"},{"line_number":182,"context_line":"threads are not going to float on the union of pCPUs dedicated to the"},{"line_number":183,"context_line":"guest but instead be constrained to the pCPU running vCPU 0."},{"line_number":184,"context_line":""},{"line_number":185,"context_line":"The InstanceNUMACell object will be extended to have a new field where"},{"line_number":186,"context_line":"physical CPUs ID will be stored and used by the driver layer to pin"}],"source_content_type":"text/x-rst","patch_set":4,"id":"df87a7cf_db78d8f9","line":183,"range":{"start_line":181,"start_character":39,"end_line":183,"end_character":60},"updated":"2017-12-03 23:46:48.000000000","message":"Great information. \n\nFirst, we can be more verbose for clarity; it means that when we set isolcpus to cpus that we are specified in vcpu_pin_set, even if multiple cpus are provided in share option or host option, threads are not... . \n\nSecond, is there anything we should consider about isolcpus when adding this new option of \u0027cpu_emulator_threads\u003dhost\u0027?\n\nIn my opinion, this info is a general aspect of the extra_spec of cpu_emulator_threads, rather than a specific aspect of the new feature of \u0027host\u0027 option. In that case, we can just put this information not here in spec but in flavor\u0027s document as a Note.","commit_id":"69d47b455d50e0bd51bc0e6efa0b315374f06470"},{"author":{"_account_id":25625,"name":"Tetsuro Nakamura","email":"tetsuro.nakamura.bc@hco.ntt.co.jp","username":"tetsuro0907"},"change_message_id":"f464363131fce705536c0c572a65a9ec62ff1f19","unresolved":false,"context_lines":[{"line_number":186,"context_line":"physical CPUs ID will be stored and used by the driver layer to pin"},{"line_number":187,"context_line":"emulator threads"},{"line_number":188,"context_line":""},{"line_number":189,"context_line":"* cpuset_reserved\u003dSetOfIntegersField(nullable\u003dTrue)"},{"line_number":190,"context_line":""},{"line_number":191,"context_line":""},{"line_number":192,"context_line":"[1] https://kernel.googlesource.com/pub/scm/linux/kernel/git/stable/linux-stable/+/47b8ea7186aae7f474ec4c98f43eaa8da719cd83%5E%21/#F0"}],"source_content_type":"text/x-rst","patch_set":4,"id":"df87a7cf_7bcc2c89","line":189,"range":{"start_line":189,"start_character":0,"end_line":189,"end_character":51},"updated":"2017-12-03 23:46:48.000000000","message":"We already have this field implemented , so this new feature of \u0027hw:cpu_emulator_threads\u003dhost\u0027 supplies no such data model impact.\n[3]https://github.com/openstack/nova/blob/master/nova/objects/instance_numa_topology.py#L59","commit_id":"69d47b455d50e0bd51bc0e6efa0b315374f06470"}],"specs/rocky/approved/libvirt-emulator-threads-policy.rst":[{"author":{"_account_id":15334,"name":"Stephen Finucane","display_name":"stephenfin","email":"stephenfin@redhat.com","username":"sfinucan"},"change_message_id":"1349bde7b8548791c4f651aa288cce967287eff4","unresolved":false,"context_lines":[{"line_number":29,"context_line":"feature is in use, there is an entire domain used to provide I/O"},{"line_number":30,"context_line":"backends for the main domain."},{"line_number":31,"context_line":""},{"line_number":32,"context_line":"Nova does not have any current mechanism to either track this extra"},{"line_number":33,"context_line":"guest instance compute requirement in order to measure utilization,"},{"line_number":34,"context_line":"nor to place any control over its execution policy."},{"line_number":35,"context_line":""},{"line_number":36,"context_line":"The libvirt driver has implemented a generic placement policy for KVM"},{"line_number":37,"context_line":"whereby the QEMU emulator threads are allowed to float across the same"}],"source_content_type":"text/x-rst","patch_set":6,"id":"5f93b717_f3b0abdf","line":34,"range":{"start_line":32,"start_character":0,"end_line":34,"end_character":51},"updated":"2018-02-01 11:49:44.000000000","message":"But it does - you implemented it in Pike. This still seems to be most the original spec from there with a couple of changes. Could you strip out the stuff that\u0027s already done so we can focus on what you want changed this cycle, please.\n\n[1] https://review.openstack.org/#/c/427066/","commit_id":"66898e8a0166378884d19cbdd94c6960baae6a00"},{"author":{"_account_id":15334,"name":"Stephen Finucane","display_name":"stephenfin","email":"stephenfin@redhat.com","username":"sfinucan"},"change_message_id":"adc74e27ed2717c1f7b67ecd37808417d1d33e71","unresolved":false,"context_lines":[{"line_number":29,"context_line":"feature is in use, there is an entire domain used to provide I/O"},{"line_number":30,"context_line":"backends for the main domain."},{"line_number":31,"context_line":""},{"line_number":32,"context_line":"Nova does not have any current mechanism to either track this extra"},{"line_number":33,"context_line":"guest instance compute requirement in order to measure utilization,"},{"line_number":34,"context_line":"nor to place any control over its execution policy."},{"line_number":35,"context_line":""},{"line_number":36,"context_line":"The libvirt driver has implemented a generic placement policy for KVM"},{"line_number":37,"context_line":"whereby the QEMU emulator threads are allowed to float across the same"}],"source_content_type":"text/x-rst","patch_set":6,"id":"5f93b717_62f7de8c","line":34,"range":{"start_line":32,"start_character":0,"end_line":34,"end_character":51},"in_reply_to":"5f93b717_53d2be29","updated":"2018-02-01 15:28:05.000000000","message":"It\u0027s not a history section, it\u0027s a problem description section. This problem seems already solved so I\u0027ve to diff patches to figure out what you\u0027re attempting to solve here. Based on that, it seems you\u0027re adding a new option, \u0027host\u0027, to the values for \u0027hw:cpu_emulator_threads\u0027. Why not just call that out and link to the original, implemented spec?\n\nThere are a couple of other questions I have here, but there\u0027s no point commenting until the above is resolved.","commit_id":"66898e8a0166378884d19cbdd94c6960baae6a00"},{"author":{"_account_id":7730,"name":"Sahid Orentino Ferdjaoui","email":"sahid.ferdjaoui@industrialdiscipline.com","username":"sahid"},"change_message_id":"53bd2eaecbf7b27a26f3ba11cdea283cfe1f4270","unresolved":false,"context_lines":[{"line_number":29,"context_line":"feature is in use, there is an entire domain used to provide I/O"},{"line_number":30,"context_line":"backends for the main domain."},{"line_number":31,"context_line":""},{"line_number":32,"context_line":"Nova does not have any current mechanism to either track this extra"},{"line_number":33,"context_line":"guest instance compute requirement in order to measure utilization,"},{"line_number":34,"context_line":"nor to place any control over its execution policy."},{"line_number":35,"context_line":""},{"line_number":36,"context_line":"The libvirt driver has implemented a generic placement policy for KVM"},{"line_number":37,"context_line":"whereby the QEMU emulator threads are allowed to float across the same"}],"source_content_type":"text/x-rst","patch_set":6,"id":"5f93b717_53d2be29","line":34,"range":{"start_line":32,"start_character":0,"end_line":34,"end_character":51},"in_reply_to":"5f93b717_f3b0abdf","updated":"2018-02-01 12:27:05.000000000","message":"Not really, this is a \"history\" section.","commit_id":"66898e8a0166378884d19cbdd94c6960baae6a00"},{"author":{"_account_id":15334,"name":"Stephen Finucane","display_name":"stephenfin","email":"stephenfin@redhat.com","username":"sfinucan"},"change_message_id":"ac1a2d8898706c47daf14c158dbce7449d623583","unresolved":false,"context_lines":[{"line_number":129,"context_line":"Alternatives"},{"line_number":130,"context_line":"------------"},{"line_number":131,"context_line":""},{"line_number":132,"context_line":"We could use a host level tunable to just reserve a set of host pCPUs"},{"line_number":133,"context_line":"for running emulator threads globally, instead of trying to account"},{"line_number":134,"context_line":"for it per instance. This would work in the simple case, but when NUMA"},{"line_number":135,"context_line":"is used, it is highly desirable to have more fine grained config to"},{"line_number":136,"context_line":"control emulator thread placement. When real-time or dedicated CPUs"},{"line_number":137,"context_line":"are used, it will be critical to separate emulator threads for"},{"line_number":138,"context_line":"different KVM instances."},{"line_number":139,"context_line":""},{"line_number":140,"context_line":"Another option is to hardcode an assumption that the vCPUs number set"},{"line_number":141,"context_line":"against the flavour implicitly includes 1 vCPUs for emulator. eg a"}],"source_content_type":"text/x-rst","patch_set":6,"id":"5f93b717_e219aed6","line":138,"range":{"start_line":132,"start_character":0,"end_line":138,"end_character":24},"updated":"2018-02-01 15:35:22.000000000","message":"This is _exactly_ what you\u0027re describing in this spec. This is what I\u0027m saying","commit_id":"66898e8a0166378884d19cbdd94c6960baae6a00"},{"author":{"_account_id":15334,"name":"Stephen Finucane","display_name":"stephenfin","email":"stephenfin@redhat.com","username":"sfinucan"},"change_message_id":"ac1a2d8898706c47daf14c158dbce7449d623583","unresolved":false,"context_lines":[{"line_number":143,"context_line":"This would likely be extremely confusing to tenant users, and"},{"line_number":144,"context_line":"developers alike."},{"line_number":145,"context_line":""},{"line_number":146,"context_line":"Do nothing is always an option. If we did nothing, then it would limit"},{"line_number":147,"context_line":"the types of workload that can be run on Nova. This would have a"},{"line_number":148,"context_line":"negative impact inparticular on users making use of the dedicated vCPU"},{"line_number":149,"context_line":"feature, as there would be no way to guarantee their vCPUs are not"},{"line_number":150,"context_line":"pre-empted by emulator threads. It can be worked around to some degree"},{"line_number":151,"context_line":"with realtime by setting a fixed policy that the emulator threads only"},{"line_number":152,"context_line":"run on the vCPUs that have non-realtime policy. This requires that all"},{"line_number":153,"context_line":"guest OS using realtime are SMP, but some guest OS want realtime, but"},{"line_number":154,"context_line":"are only UP."},{"line_number":155,"context_line":""},{"line_number":156,"context_line":"Data model impact"},{"line_number":157,"context_line":"-----------------"}],"source_content_type":"text/x-rst","patch_set":6,"id":"5f93b717_820c6a94","line":154,"range":{"start_line":146,"start_character":0,"end_line":154,"end_character":12},"updated":"2018-02-01 15:35:22.000000000","message":"We\u0027ve already done something. You get where I\u0027m going here :)","commit_id":"66898e8a0166378884d19cbdd94c6960baae6a00"},{"author":{"_account_id":15334,"name":"Stephen Finucane","display_name":"stephenfin","email":"stephenfin@redhat.com","username":"sfinucan"},"change_message_id":"ac1a2d8898706c47daf14c158dbce7449d623583","unresolved":false,"context_lines":[{"line_number":152,"context_line":"run on the vCPUs that have non-realtime policy. This requires that all"},{"line_number":153,"context_line":"guest OS using realtime are SMP, but some guest OS want realtime, but"},{"line_number":154,"context_line":"are only UP."},{"line_number":155,"context_line":""},{"line_number":156,"context_line":"Data model impact"},{"line_number":157,"context_line":"-----------------"},{"line_number":158,"context_line":""}],"source_content_type":"text/x-rst","patch_set":6,"id":"5f93b717_825a2a80","line":155,"updated":"2018-02-01 15:35:22.000000000","message":"Another alternative: instead of adding a new option to the flavor key, we just add the global option. I\u0027ve seen no data suggesting that one emulator thread per instance yields any performance improvement over the pool that this spec proposes. As a result, why do we need to retain the one emulator thread-per-instance option? Could we not make this the one and only option?","commit_id":"66898e8a0166378884d19cbdd94c6960baae6a00"},{"author":{"_account_id":15334,"name":"Stephen Finucane","display_name":"stephenfin","email":"stephenfin@redhat.com","username":"sfinucan"},"change_message_id":"ac1a2d8898706c47daf14c158dbce7449d623583","unresolved":false,"context_lines":[{"line_number":295,"context_line":".. list-table:: Revisions"},{"line_number":296,"context_line":"   :header-rows: 1"},{"line_number":297,"context_line":""},{"line_number":298,"context_line":"   * - Release Name"},{"line_number":299,"context_line":"     - Description"},{"line_number":300,"context_line":"   * - Mitaka"},{"line_number":301,"context_line":"     - Proposed"},{"line_number":302,"context_line":"   * - Ocata"},{"line_number":303,"context_line":"     - Re-proposed"},{"line_number":304,"context_line":"   * - Pike"},{"line_number":305,"context_line":"     - Re-proposed"},{"line_number":306,"context_line":"   * - Queen"},{"line_number":307,"context_line":"     - Introducing \u0027host\u0027 option"},{"line_number":308,"context_line":"   * - Rocky"},{"line_number":309,"context_line":"     - Re-proposed"}],"source_content_type":"text/x-rst","patch_set":6,"id":"5f93b717_42b42280","line":307,"range":{"start_line":298,"start_character":0,"end_line":307,"end_character":32},"updated":"2018-02-01 15:35:22.000000000","message":"This is a new spec. None of this should be here","commit_id":"66898e8a0166378884d19cbdd94c6960baae6a00"},{"author":{"_account_id":15334,"name":"Stephen Finucane","display_name":"stephenfin","email":"stephenfin@redhat.com","username":"sfinucan"},"change_message_id":"ac1a2d8898706c47daf14c158dbce7449d623583","unresolved":false,"context_lines":[{"line_number":306,"context_line":"   * - Queen"},{"line_number":307,"context_line":"     - Introducing \u0027host\u0027 option"},{"line_number":308,"context_line":"   * - Rocky"},{"line_number":309,"context_line":"     - Re-proposed"}],"source_content_type":"text/x-rst","patch_set":6,"id":"5f93b717_02aa9a17","line":309,"range":{"start_line":309,"start_character":7,"end_line":309,"end_character":18},"updated":"2018-02-01 15:35:22.000000000","message":"Proposed","commit_id":"66898e8a0166378884d19cbdd94c6960baae6a00"},{"author":{"_account_id":8768,"name":"Chris Friesen","email":"chris.friesen@windriver.com","username":"cbf123"},"change_message_id":"317f717f73e7089d5264ac454517efe0d5d5d572","unresolved":false,"context_lines":[{"line_number":17,"context_line":"Previously was introduced option \u0027hw:emulator_threads_policy\u0027 which"},{"line_number":18,"context_line":"adds additional pCPU per guest to run emulator threads."},{"line_number":19,"context_line":""},{"line_number":20,"context_line":"While it resolves issues related to splike latency caused by emulator"},{"line_number":21,"context_line":"threads running on same pCPUs than vCPUs are pinned on. Some Operators"},{"line_number":22,"context_line":"have desire to pack all emulator threads on a specific and dedicated"},{"line_number":23,"context_line":"set of pCPUs that to allow more pCPUs running vCPUs."}],"source_content_type":"text/x-rst","patch_set":8,"id":"1f9dbf25_5e05b179","line":20,"range":{"start_line":20,"start_character":36,"end_line":20,"end_character":42},"updated":"2018-02-27 11:56:12.000000000","message":"typo","commit_id":"7bc6da7aed544516d77b59be2420464bbb9e36a3"},{"author":{"_account_id":8768,"name":"Chris Friesen","email":"chris.friesen@windriver.com","username":"cbf123"},"change_message_id":"317f717f73e7089d5264ac454517efe0d5d5d572","unresolved":false,"context_lines":[{"line_number":18,"context_line":"adds additional pCPU per guest to run emulator threads."},{"line_number":19,"context_line":""},{"line_number":20,"context_line":"While it resolves issues related to splike latency caused by emulator"},{"line_number":21,"context_line":"threads running on same pCPUs than vCPUs are pinned on. Some Operators"},{"line_number":22,"context_line":"have desire to pack all emulator threads on a specific and dedicated"},{"line_number":23,"context_line":"set of pCPUs that to allow more pCPUs running vCPUs."},{"line_number":24,"context_line":""}],"source_content_type":"text/x-rst","patch_set":8,"id":"1f9dbf25_5e2a91e7","line":21,"range":{"start_line":21,"start_character":54,"end_line":21,"end_character":70},"updated":"2018-02-27 11:56:12.000000000","message":"grammar nit, this should be a continuation of the same sentence.","commit_id":"7bc6da7aed544516d77b59be2420464bbb9e36a3"},{"author":{"_account_id":8768,"name":"Chris Friesen","email":"chris.friesen@windriver.com","username":"cbf123"},"change_message_id":"317f717f73e7089d5264ac454517efe0d5d5d572","unresolved":false,"context_lines":[{"line_number":20,"context_line":"While it resolves issues related to splike latency caused by emulator"},{"line_number":21,"context_line":"threads running on same pCPUs than vCPUs are pinned on. Some Operators"},{"line_number":22,"context_line":"have desire to pack all emulator threads on a specific and dedicated"},{"line_number":23,"context_line":"set of pCPUs that to allow more pCPUs running vCPUs."},{"line_number":24,"context_line":""},{"line_number":25,"context_line":"Use Cases"},{"line_number":26,"context_line":"---------"}],"source_content_type":"text/x-rst","patch_set":8,"id":"1f9dbf25_3e39c540","line":23,"range":{"start_line":23,"start_character":13,"end_line":23,"end_character":21},"updated":"2018-02-27 11:56:12.000000000","message":"nit, should be \"in order to\"","commit_id":"7bc6da7aed544516d77b59be2420464bbb9e36a3"},{"author":{"_account_id":8768,"name":"Chris Friesen","email":"chris.friesen@windriver.com","username":"cbf123"},"change_message_id":"317f717f73e7089d5264ac454517efe0d5d5d572","unresolved":false,"context_lines":[{"line_number":25,"context_line":"Use Cases"},{"line_number":26,"context_line":"---------"},{"line_number":27,"context_line":""},{"line_number":28,"context_line":"As an operator I want all the emulator threads running in a set of"},{"line_number":29,"context_line":"pCPUs dedicated."},{"line_number":30,"context_line":""},{"line_number":31,"context_line":"Project Priority"},{"line_number":32,"context_line":"----------------"}],"source_content_type":"text/x-rst","patch_set":8,"id":"1f9dbf25_fe235d04","line":29,"range":{"start_line":28,"start_character":60,"end_line":29,"end_character":15},"updated":"2018-02-27 11:56:12.000000000","message":"grammar: would be clearer as \"in a specific set of pCPUs\"","commit_id":"7bc6da7aed544516d77b59be2420464bbb9e36a3"},{"author":{"_account_id":8768,"name":"Chris Friesen","email":"chris.friesen@windriver.com","username":"cbf123"},"change_message_id":"317f717f73e7089d5264ac454517efe0d5d5d572","unresolved":false,"context_lines":[{"line_number":36,"context_line":"Proposed change"},{"line_number":37,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"},{"line_number":38,"context_line":""},{"line_number":39,"context_line":"A pre-requisite for enabling the emulator threads placement policy"},{"line_number":40,"context_line":"feature on a flavor is that it must also have ‘hw:cpu_policy’ set to"},{"line_number":41,"context_line":"‘dedicated’."},{"line_number":42,"context_line":""},{"line_number":43,"context_line":"To extend the flexibility and address use-cases where resources on"},{"line_number":44,"context_line":"hosts are limited, we are adding the ability for operators to reserve"}],"source_content_type":"text/x-rst","patch_set":8,"id":"1f9dbf25_fed6fdc3","line":41,"range":{"start_line":39,"start_character":0,"end_line":41,"end_character":12},"updated":"2018-02-27 11:56:12.000000000","message":"is this relevent to this spec?","commit_id":"7bc6da7aed544516d77b59be2420464bbb9e36a3"},{"author":{"_account_id":7730,"name":"Sahid Orentino Ferdjaoui","email":"sahid.ferdjaoui@industrialdiscipline.com","username":"sahid"},"change_message_id":"3f748391555dd480c4675b7408c385daf8b1f330","unresolved":false,"context_lines":[{"line_number":36,"context_line":"Proposed change"},{"line_number":37,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"},{"line_number":38,"context_line":""},{"line_number":39,"context_line":"A pre-requisite for enabling the emulator threads placement policy"},{"line_number":40,"context_line":"feature on a flavor is that it must also have ‘hw:cpu_policy’ set to"},{"line_number":41,"context_line":"‘dedicated’."},{"line_number":42,"context_line":""},{"line_number":43,"context_line":"To extend the flexibility and address use-cases where resources on"},{"line_number":44,"context_line":"hosts are limited, we are adding the ability for operators to reserve"}],"source_content_type":"text/x-rst","patch_set":8,"id":"ff6b8bd7_01402935","line":41,"range":{"start_line":39,"start_character":0,"end_line":41,"end_character":12},"in_reply_to":"1f9dbf25_fed6fdc3","updated":"2018-03-05 13:56:33.000000000","message":"I think so since it\u0027s a requirement","commit_id":"7bc6da7aed544516d77b59be2420464bbb9e36a3"},{"author":{"_account_id":8768,"name":"Chris Friesen","email":"chris.friesen@windriver.com","username":"cbf123"},"change_message_id":"317f717f73e7089d5264ac454517efe0d5d5d572","unresolved":false,"context_lines":[{"line_number":47,"context_line":"configured to use them (as opposed to reserving a whole extra CPU per"},{"line_number":48,"context_line":"instance with the \"isolate\" option.)"},{"line_number":49,"context_line":""},{"line_number":50,"context_line":"The reserved host CPUs are specified via the \u0027overhead_pin_set\u0027 config"},{"line_number":51,"context_line":"option. This is analogous to the existing \"vcpu_pin_set\" and is"},{"line_number":52,"context_line":"formatted the same way."},{"line_number":53,"context_line":""}],"source_content_type":"text/x-rst","patch_set":8,"id":"1f9dbf25_7e998d1d","line":50,"range":{"start_line":50,"start_character":46,"end_line":50,"end_character":62},"updated":"2018-02-27 11:56:12.000000000","message":"\"overhead\" is really vague.  would it make sense to change this to \"emulator_pin_set\" or something similar?","commit_id":"7bc6da7aed544516d77b59be2420464bbb9e36a3"},{"author":{"_account_id":7730,"name":"Sahid Orentino Ferdjaoui","email":"sahid.ferdjaoui@industrialdiscipline.com","username":"sahid"},"change_message_id":"3f748391555dd480c4675b7408c385daf8b1f330","unresolved":false,"context_lines":[{"line_number":47,"context_line":"configured to use them (as opposed to reserving a whole extra CPU per"},{"line_number":48,"context_line":"instance with the \"isolate\" option.)"},{"line_number":49,"context_line":""},{"line_number":50,"context_line":"The reserved host CPUs are specified via the \u0027overhead_pin_set\u0027 config"},{"line_number":51,"context_line":"option. This is analogous to the existing \"vcpu_pin_set\" and is"},{"line_number":52,"context_line":"formatted the same way."},{"line_number":53,"context_line":""}],"source_content_type":"text/x-rst","patch_set":8,"id":"ff6b8bd7_c16231d9","line":50,"range":{"start_line":50,"start_character":46,"end_line":50,"end_character":62},"in_reply_to":"1f9dbf25_7e998d1d","updated":"2018-03-05 13:56:33.000000000","message":"I wanted to have something generic, we already have this term in the virt drivers. The point is that other drivers may want to use this option and emulator threads is kind of a specific term used by libvirt. Makes sense?","commit_id":"7bc6da7aed544516d77b59be2420464bbb9e36a3"},{"author":{"_account_id":8768,"name":"Chris Friesen","email":"chris.friesen@windriver.com","username":"cbf123"},"change_message_id":"317f717f73e7089d5264ac454517efe0d5d5d572","unresolved":false,"context_lines":[{"line_number":51,"context_line":"option. This is analogous to the existing \"vcpu_pin_set\" and is"},{"line_number":52,"context_line":"formatted the same way."},{"line_number":53,"context_line":""},{"line_number":54,"context_line":"In order to run the hypervisor work on these reserved host CPUs, the"},{"line_number":55,"context_line":"admin must set \"hw:cpu_emulator_threads\u003dhost\" in the flavor"},{"line_number":56,"context_line":"extra-spec."},{"line_number":57,"context_line":""},{"line_number":58,"context_line":"* hw:cpu_emulator_threads\u003dhost"},{"line_number":59,"context_line":""},{"line_number":60,"context_line":"Would say the emulator threads of all the guests are going to be"},{"line_number":61,"context_line":"pinned on a global isolated host pCPUs according \u0027overhead_pin_set\u0027"},{"line_number":62,"context_line":""},{"line_number":63,"context_line":"NOTE: The \u0027overhead_pin_set\u0027 has to be subset of \u0027vcpu_pin_set\u0027 if"},{"line_number":64,"context_line":"defined."}],"source_content_type":"text/x-rst","patch_set":8,"id":"1f9dbf25_21568a08","line":61,"range":{"start_line":54,"start_character":0,"end_line":61,"end_character":67},"updated":"2018-02-27 11:56:12.000000000","message":"As Stephen Finucane asked earlier, do we really need both the existing \"hw:emulator_threads_policy\u003disolate\" option (where the emulator threads for each instance get a whole pCPU) as well as this option (where the emulator threads from all instances share a pool of pCPUs)?\n\nIf we don\u0027t need both, maybe we could just keep the existing flavor extra spec, and if the compute node specifies a pool of pCPUs for the emulator threads then they run there instead of allocating a whole pCPU per instance.\n\n(It would probably result in the simplest code if we said that you *must* specify the pool of pCPUs in order to run with \"hw:emulator_threads_policy\u003disolate\", but that would introduce upgrade complexities.)","commit_id":"7bc6da7aed544516d77b59be2420464bbb9e36a3"},{"author":{"_account_id":7730,"name":"Sahid Orentino Ferdjaoui","email":"sahid.ferdjaoui@industrialdiscipline.com","username":"sahid"},"change_message_id":"3f748391555dd480c4675b7408c385daf8b1f330","unresolved":false,"context_lines":[{"line_number":51,"context_line":"option. This is analogous to the existing \"vcpu_pin_set\" and is"},{"line_number":52,"context_line":"formatted the same way."},{"line_number":53,"context_line":""},{"line_number":54,"context_line":"In order to run the hypervisor work on these reserved host CPUs, the"},{"line_number":55,"context_line":"admin must set \"hw:cpu_emulator_threads\u003dhost\" in the flavor"},{"line_number":56,"context_line":"extra-spec."},{"line_number":57,"context_line":""},{"line_number":58,"context_line":"* hw:cpu_emulator_threads\u003dhost"},{"line_number":59,"context_line":""},{"line_number":60,"context_line":"Would say the emulator threads of all the guests are going to be"},{"line_number":61,"context_line":"pinned on a global isolated host pCPUs according \u0027overhead_pin_set\u0027"},{"line_number":62,"context_line":""},{"line_number":63,"context_line":"NOTE: The \u0027overhead_pin_set\u0027 has to be subset of \u0027vcpu_pin_set\u0027 if"},{"line_number":64,"context_line":"defined."}],"source_content_type":"text/x-rst","patch_set":8,"id":"ff6b8bd7_81e8b916","line":61,"range":{"start_line":54,"start_character":0,"end_line":61,"end_character":67},"in_reply_to":"1f9dbf25_21568a08","updated":"2018-03-05 13:56:33.000000000","message":"I can\u0027t say we don\u0027t need both I can say we need to add one with more flexibility.","commit_id":"7bc6da7aed544516d77b59be2420464bbb9e36a3"},{"author":{"_account_id":8768,"name":"Chris Friesen","email":"chris.friesen@windriver.com","username":"cbf123"},"change_message_id":"317f717f73e7089d5264ac454517efe0d5d5d572","unresolved":false,"context_lines":[{"line_number":60,"context_line":"Would say the emulator threads of all the guests are going to be"},{"line_number":61,"context_line":"pinned on a global isolated host pCPUs according \u0027overhead_pin_set\u0027"},{"line_number":62,"context_line":""},{"line_number":63,"context_line":"NOTE: The \u0027overhead_pin_set\u0027 has to be subset of \u0027vcpu_pin_set\u0027 if"},{"line_number":64,"context_line":"defined."},{"line_number":65,"context_line":""},{"line_number":66,"context_line":""},{"line_number":67,"context_line":"Alternatives"}],"source_content_type":"text/x-rst","patch_set":8,"id":"1f9dbf25_61757298","line":64,"range":{"start_line":63,"start_character":1,"end_line":64,"end_character":8},"updated":"2018-02-27 11:56:12.000000000","message":"Why?  Wouldn\u0027t the resource tracking be simpler if they were disjoint sets?","commit_id":"7bc6da7aed544516d77b59be2420464bbb9e36a3"},{"author":{"_account_id":7730,"name":"Sahid Orentino Ferdjaoui","email":"sahid.ferdjaoui@industrialdiscipline.com","username":"sahid"},"change_message_id":"3f748391555dd480c4675b7408c385daf8b1f330","unresolved":false,"context_lines":[{"line_number":60,"context_line":"Would say the emulator threads of all the guests are going to be"},{"line_number":61,"context_line":"pinned on a global isolated host pCPUs according \u0027overhead_pin_set\u0027"},{"line_number":62,"context_line":""},{"line_number":63,"context_line":"NOTE: The \u0027overhead_pin_set\u0027 has to be subset of \u0027vcpu_pin_set\u0027 if"},{"line_number":64,"context_line":"defined."},{"line_number":65,"context_line":""},{"line_number":66,"context_line":""},{"line_number":67,"context_line":"Alternatives"}],"source_content_type":"text/x-rst","patch_set":8,"id":"ff6b8bd7_81d1995a","line":64,"range":{"start_line":63,"start_character":1,"end_line":64,"end_character":8},"in_reply_to":"1f9dbf25_61757298","updated":"2018-03-05 13:56:33.000000000","message":"I think the whole concept behind vcpu_pin_set is to say that it\u0027s the generic option used by Nova to limit the pCPUs so we guarantee to operators everything running for Nova will be under that range.","commit_id":"7bc6da7aed544516d77b59be2420464bbb9e36a3"},{"author":{"_account_id":7730,"name":"Sahid Orentino Ferdjaoui","email":"sahid.ferdjaoui@industrialdiscipline.com","username":"sahid"},"change_message_id":"4db71b0831d56813afcba015211bdfa84b0d3180","unresolved":false,"context_lines":[{"line_number":60,"context_line":"Would say the emulator threads of all the guests are going to be"},{"line_number":61,"context_line":"pinned on a global isolated host pCPUs according \u0027overhead_pin_set\u0027"},{"line_number":62,"context_line":""},{"line_number":63,"context_line":"NOTE: The \u0027overhead_pin_set\u0027 has to be subset of \u0027vcpu_pin_set\u0027 if"},{"line_number":64,"context_line":"defined."},{"line_number":65,"context_line":""},{"line_number":66,"context_line":""},{"line_number":67,"context_line":"Alternatives"}],"source_content_type":"text/x-rst","patch_set":8,"id":"ff6b8bd7_02f01c90","line":64,"range":{"start_line":63,"start_character":1,"end_line":64,"end_character":8},"in_reply_to":"ff6b8bd7_1724eb0c","updated":"2018-03-06 08:34:51.000000000","message":"I\u0027m not agreed with that we need a global config to restraint the set of pCPUs used per Nova. Example for isolate we pick a free pCPU on that range. All the new options introduced to create range of pCPUs for certain workflow should be picked on that \"main\" range.\n\nI\u0027m going to keep it like that since I think it\u0027s how we should do it and if I get more reviews against my solution so I will change it.\n\nThanks.","commit_id":"7bc6da7aed544516d77b59be2420464bbb9e36a3"},{"author":{"_account_id":8768,"name":"Chris Friesen","email":"chris.friesen@windriver.com","username":"cbf123"},"change_message_id":"e12767569c1f697539f21b0c69d0508d50a5ce08","unresolved":false,"context_lines":[{"line_number":60,"context_line":"Would say the emulator threads of all the guests are going to be"},{"line_number":61,"context_line":"pinned on a global isolated host pCPUs according \u0027overhead_pin_set\u0027"},{"line_number":62,"context_line":""},{"line_number":63,"context_line":"NOTE: The \u0027overhead_pin_set\u0027 has to be subset of \u0027vcpu_pin_set\u0027 if"},{"line_number":64,"context_line":"defined."},{"line_number":65,"context_line":""},{"line_number":66,"context_line":""},{"line_number":67,"context_line":"Alternatives"}],"source_content_type":"text/x-rst","patch_set":8,"id":"ff6b8bd7_1724eb0c","line":64,"range":{"start_line":63,"start_character":1,"end_line":64,"end_character":8},"in_reply_to":"ff6b8bd7_81d1995a","updated":"2018-03-05 16:04:25.000000000","message":"At the PTG when talking about the \"support shared/dedicated instances on the same host\" spec, Dan Smith supported having disjoint sets for \"pCPUs to be used for shared instances\" (the existing \"vcpu_pin_set\") and \"pCPUs to be used for dedicated instances\".\n\nI think this would fall under similar logic, since it\u0027s basically \"pCPUs to be used for emulator overhead\".","commit_id":"7bc6da7aed544516d77b59be2420464bbb9e36a3"},{"author":{"_account_id":15334,"name":"Stephen Finucane","display_name":"stephenfin","email":"stephenfin@redhat.com","username":"sfinucan"},"change_message_id":"07be78fc1f4654bd08fe51598984a99606e09114","unresolved":false,"context_lines":[{"line_number":35,"context_line":""},{"line_number":36,"context_line":"Proposed change"},{"line_number":37,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"},{"line_number":38,"context_line":""},{"line_number":39,"context_line":"A pre-requisite for enabling the emulator threads placement policy"},{"line_number":40,"context_line":"feature on a flavor is that it must also have ‘hw:cpu_policy’ set to"},{"line_number":41,"context_line":"‘dedicated’."},{"line_number":42,"context_line":""},{"line_number":43,"context_line":"To extend the flexibility and address use-cases where resources on"},{"line_number":44,"context_line":"hosts are limited, we are adding the ability for operators to reserve"}],"source_content_type":"text/x-rst","patch_set":9,"id":"df7087c5_763f69d6","line":41,"range":{"start_line":38,"start_character":0,"end_line":41,"end_character":12},"updated":"2018-03-07 11:34:20.000000000","message":"You already call this out in the deployer impact section. Is it relevant to the functional change you\u0027re going to make? If not, perhaps you could omit this","commit_id":"4b670656dc163f48a491c33df40401b463f257af"},{"author":{"_account_id":15334,"name":"Stephen Finucane","display_name":"stephenfin","email":"stephenfin@redhat.com","username":"sfinucan"},"change_message_id":"07be78fc1f4654bd08fe51598984a99606e09114","unresolved":false,"context_lines":[{"line_number":51,"context_line":"option. This is analogous to the existing \"vcpu_pin_set\" and is"},{"line_number":52,"context_line":"formatted the same way."},{"line_number":53,"context_line":""},{"line_number":54,"context_line":"In order to run the hypervisor work on these reserved host CPUs, the"},{"line_number":55,"context_line":"admin must set \"hw:cpu_emulator_threads\u003dhost\" in the flavor"},{"line_number":56,"context_line":"extra-spec."},{"line_number":57,"context_line":""},{"line_number":58,"context_line":"* hw:cpu_emulator_threads\u003dhost"},{"line_number":59,"context_line":""},{"line_number":60,"context_line":"Would say the emulator threads of all the guests are going to be"},{"line_number":61,"context_line":"pinned on a global isolated host pCPUs according \u0027overhead_pin_set\u0027"}],"source_content_type":"text/x-rst","patch_set":9,"id":"df7087c5_b17aeb85","line":58,"range":{"start_line":54,"start_character":0,"end_line":58,"end_character":30},"updated":"2018-03-07 11:34:20.000000000","message":"I raised this before and didn\u0027t get an answer: why are we adding a new extra spec rather than extending the current one - hw:emulator_threads_policy? We could simply add \u0027host\u0027 to this list:\n\n- shared - threads floating across guest pCPUs\n- isolate - threads isolated on a single guest pCPU\n- host - threads isolated on pool of host pCPUs","commit_id":"4b670656dc163f48a491c33df40401b463f257af"},{"author":{"_account_id":15334,"name":"Stephen Finucane","display_name":"stephenfin","email":"stephenfin@redhat.com","username":"sfinucan"},"change_message_id":"07be78fc1f4654bd08fe51598984a99606e09114","unresolved":false,"context_lines":[{"line_number":60,"context_line":"Would say the emulator threads of all the guests are going to be"},{"line_number":61,"context_line":"pinned on a global isolated host pCPUs according \u0027overhead_pin_set\u0027"},{"line_number":62,"context_line":""},{"line_number":63,"context_line":"NOTE: The \u0027overhead_pin_set\u0027 has to be subset of \u0027vcpu_pin_set\u0027 if"},{"line_number":64,"context_line":"defined."},{"line_number":65,"context_line":""},{"line_number":66,"context_line":""},{"line_number":67,"context_line":"Alternatives"}],"source_content_type":"text/x-rst","patch_set":9,"id":"df7087c5_b12c2b9a","line":64,"range":{"start_line":63,"start_character":0,"end_line":64,"end_character":8},"updated":"2018-03-07 11:34:20.000000000","message":"I think other reviewers have raised this already, but there was a clear preference for disjoint set for the dedicated CPU set flag that is proposed in another spec. The same reasoning applied there really applies here (easier to comprehend and easier to configure)","commit_id":"4b670656dc163f48a491c33df40401b463f257af"},{"author":{"_account_id":15334,"name":"Stephen Finucane","display_name":"stephenfin","email":"stephenfin@redhat.com","username":"sfinucan"},"change_message_id":"07be78fc1f4654bd08fe51598984a99606e09114","unresolved":false,"context_lines":[{"line_number":72,"context_line":"vCPU value of 5 would imply 4 actual vCPUs and 1 system pseudo-vCPU."},{"line_number":73,"context_line":"This would likely be extremely confusing to tenant users, and"},{"line_number":74,"context_line":"developers alike."},{"line_number":75,"context_line":""},{"line_number":76,"context_line":"Data model impact"},{"line_number":77,"context_line":"-----------------"},{"line_number":78,"context_line":""}],"source_content_type":"text/x-rst","patch_set":9,"id":"df7087c5_51402f08","line":75,"updated":"2018-03-07 11:34:20.000000000","message":"Another alternative is to add this configuration option and, if it\u0027s defined and the current \u0027hw:emulator_threads_policy\u003disolate\u0027 extra spec is set, use this pool instead of the one extra CPU per instance. This could allow us to phase out the current \"one extra CPU for every instance\" pattern over time in favor of the shared pool, as there appears to be a strong preference for the latter.","commit_id":"4b670656dc163f48a491c33df40401b463f257af"},{"author":{"_account_id":15334,"name":"Stephen Finucane","display_name":"stephenfin","email":"stephenfin@redhat.com","username":"sfinucan"},"change_message_id":"07be78fc1f4654bd08fe51598984a99606e09114","unresolved":false,"context_lines":[{"line_number":77,"context_line":"-----------------"},{"line_number":78,"context_line":""},{"line_number":79,"context_line":"The CPUEmulatorThreadsPolicy() will be extended to admint a new option"},{"line_number":80,"context_line":"`host`, meaning that the emulator threads are isolated on the pCPUs"},{"line_number":81,"context_line":"set configure with \u0027overhead_pin_set\u0027"},{"line_number":82,"context_line":""},{"line_number":83,"context_line":""}],"source_content_type":"text/x-rst","patch_set":9,"id":"df7087c5_11971793","line":80,"range":{"start_line":80,"start_character":0,"end_line":80,"end_character":6},"updated":"2018-03-07 11:34:20.000000000","message":"nit: double backticks are necessary indicate literal or code blocks, which is what I think you intend here. Single backticks indicate the default role which, per the Sphinx developers, can be absolutely anything.","commit_id":"4b670656dc163f48a491c33df40401b463f257af"},{"author":{"_account_id":15334,"name":"Stephen Finucane","display_name":"stephenfin","email":"stephenfin@redhat.com","username":"sfinucan"},"change_message_id":"07be78fc1f4654bd08fe51598984a99606e09114","unresolved":false,"context_lines":[{"line_number":176,"context_line":""},{"line_number":177,"context_line":"   * - Release Name"},{"line_number":178,"context_line":"     - Description"},{"line_number":179,"context_line":"   * - Mitaka"},{"line_number":180,"context_line":"     - Proposed"},{"line_number":181,"context_line":"   * - Ocata"},{"line_number":182,"context_line":"     - Re-proposed"},{"line_number":183,"context_line":"   * - Pike"},{"line_number":184,"context_line":"     - Re-proposed"},{"line_number":185,"context_line":"   * - Queen"},{"line_number":186,"context_line":"     - Introducing \u0027host\u0027 option"},{"line_number":187,"context_line":"   * - Rocky"},{"line_number":188,"context_line":"     - Re-proposed"}],"source_content_type":"text/x-rst","patch_set":9,"id":"df7087c5_114ab726","line":186,"range":{"start_line":179,"start_character":0,"end_line":186,"end_character":32},"updated":"2018-03-07 11:34:20.000000000","message":"This is a new spec so none of this is true. You should remove it.","commit_id":"4b670656dc163f48a491c33df40401b463f257af"},{"author":{"_account_id":7,"name":"Jay Pipes","email":"jaypipes@gmail.com","username":"jaypipes"},"change_message_id":"ee241386308202fccec68c6b01e102799bc13339","unresolved":false,"context_lines":[{"line_number":14,"context_line":"Problem description"},{"line_number":15,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"},{"line_number":16,"context_line":""},{"line_number":17,"context_line":"Previously was introduced option \u0027hw:emulator_threads_policy\u0027 which"},{"line_number":18,"context_line":"adds additional pCPU per guest to run emulator threads."},{"line_number":19,"context_line":""},{"line_number":20,"context_line":"While it resolves issues related to spike latency caused by emulator"}],"source_content_type":"text/x-rst","patch_set":10,"id":"df7087c5_cf79b4fc","line":17,"range":{"start_line":17,"start_character":0,"end_line":17,"end_character":15},"updated":"2018-03-09 19:57:44.000000000","message":"Previously, we introduced the option","commit_id":"59ebaee15c470cfbd845d359f57a9c20bb26295f"},{"author":{"_account_id":7,"name":"Jay Pipes","email":"jaypipes@gmail.com","username":"jaypipes"},"change_message_id":"ee241386308202fccec68c6b01e102799bc13339","unresolved":false,"context_lines":[{"line_number":18,"context_line":"adds additional pCPU per guest to run emulator threads."},{"line_number":19,"context_line":""},{"line_number":20,"context_line":"While it resolves issues related to spike latency caused by emulator"},{"line_number":21,"context_line":"threads running on same pCPUs than vCPUs are pinned on some Operators"},{"line_number":22,"context_line":"have desire to pack all emulator threads on a specific and dedicated"},{"line_number":23,"context_line":"set of pCPUs in order to allow more pCPUs running vCPUs."},{"line_number":24,"context_line":""}],"source_content_type":"text/x-rst","patch_set":10,"id":"df7087c5_cf62143f","line":21,"range":{"start_line":21,"start_character":42,"end_line":21,"end_character":61},"updated":"2018-03-09 19:57:44.000000000","message":"\"threads running on the same pCPUs that vCPUs are pinned to, some operators have the desire\"","commit_id":"59ebaee15c470cfbd845d359f57a9c20bb26295f"},{"author":{"_account_id":7,"name":"Jay Pipes","email":"jaypipes@gmail.com","username":"jaypipes"},"change_message_id":"ee241386308202fccec68c6b01e102799bc13339","unresolved":false,"context_lines":[{"line_number":26,"context_line":"---------"},{"line_number":27,"context_line":""},{"line_number":28,"context_line":"As an operator I want all the emulator threads of all the instances"},{"line_number":29,"context_line":"running in a specific set pCPUs."},{"line_number":30,"context_line":""},{"line_number":31,"context_line":"Project Priority"},{"line_number":32,"context_line":"----------------"}],"source_content_type":"text/x-rst","patch_set":10,"id":"df7087c5_2f4b90b0","line":29,"range":{"start_line":29,"start_character":25,"end_line":29,"end_character":26},"updated":"2018-03-09 19:57:44.000000000","message":"set of pCPUs","commit_id":"59ebaee15c470cfbd845d359f57a9c20bb26295f"},{"author":{"_account_id":7,"name":"Jay Pipes","email":"jaypipes@gmail.com","username":"jaypipes"},"change_message_id":"ee241386308202fccec68c6b01e102799bc13339","unresolved":false,"context_lines":[{"line_number":47,"context_line":"option. This is analogous to the existing \"vcpu_pin_set\" and is"},{"line_number":48,"context_line":"formatted the same way."},{"line_number":49,"context_line":""},{"line_number":50,"context_line":"In order to run the hypervisor work on these reserved host CPUs, the"},{"line_number":51,"context_line":"admin must set \"hw:cpu_emulator_threads_policy\u003dhost\" in the flavor"},{"line_number":52,"context_line":"extra-spec."},{"line_number":53,"context_line":""},{"line_number":54,"context_line":"* hw:cpu_emulator_threads_policy\u003dhost"},{"line_number":55,"context_line":""}],"source_content_type":"text/x-rst","patch_set":10,"id":"df7087c5_ef66f829","line":52,"range":{"start_line":50,"start_character":0,"end_line":52,"end_character":11},"updated":"2018-03-09 19:57:44.000000000","message":"here\u0027s an idea... what about just getting rid of the hw:emulator_threads_policy extra spec entirely and just using this new overhead_pin_set CONF option to determine which pCPUs to place the emulator threads on?\n\nLet the *host configuration* determine the policy of which pCPUs to place emulator threads on instead of having a flavor/image-specified policy...","commit_id":"59ebaee15c470cfbd845d359f57a9c20bb26295f"},{"author":{"_account_id":8768,"name":"Chris Friesen","email":"chris.friesen@windriver.com","username":"cbf123"},"change_message_id":"00758a042865e5ae02848d29b553f858b454ed13","unresolved":false,"context_lines":[{"line_number":47,"context_line":"option. This is analogous to the existing \"vcpu_pin_set\" and is"},{"line_number":48,"context_line":"formatted the same way."},{"line_number":49,"context_line":""},{"line_number":50,"context_line":"In order to run the hypervisor work on these reserved host CPUs, the"},{"line_number":51,"context_line":"admin must set \"hw:cpu_emulator_threads_policy\u003dhost\" in the flavor"},{"line_number":52,"context_line":"extra-spec."},{"line_number":53,"context_line":""},{"line_number":54,"context_line":"* hw:cpu_emulator_threads_policy\u003dhost"},{"line_number":55,"context_line":""}],"source_content_type":"text/x-rst","patch_set":10,"id":"df7087c5_37dbc11b","line":52,"range":{"start_line":50,"start_character":0,"end_line":52,"end_character":11},"in_reply_to":"df7087c5_209ffab4","updated":"2018-03-12 18:13:31.000000000","message":"sahid: the main benefit of allowing shared/dedicated instances on one host is if you have a small number of hosts (possibly even a single host) and want to run a mix of high-performance workloads and low-performance workloads (stats gathering, management, etc.)\n\njay:  First, I thought we had agreed at the PTG that disjoint sets were the way to go. :)   Second, if you remove the extra-spec \"isolate\" option, how would you handle the case where all the pCPUs were allocated to \"dedicated\" vCPUs?  Currently the extra-spec flag is a signal that the user cares so much about performance that they want to allocate a whole extra pCPU for their instance.\n\nI agree that if there are \"shared\" pCPUs then it might make sense to just always run the emulator threads there.  I\u0027m just not sure how to handle the case where there aren\u0027t.","commit_id":"59ebaee15c470cfbd845d359f57a9c20bb26295f"},{"author":{"_account_id":7730,"name":"Sahid Orentino Ferdjaoui","email":"sahid.ferdjaoui@industrialdiscipline.com","username":"sahid"},"change_message_id":"9986244be288faa1278a2203ce8b7c64a6d90ec5","unresolved":false,"context_lines":[{"line_number":47,"context_line":"option. This is analogous to the existing \"vcpu_pin_set\" and is"},{"line_number":48,"context_line":"formatted the same way."},{"line_number":49,"context_line":""},{"line_number":50,"context_line":"In order to run the hypervisor work on these reserved host CPUs, the"},{"line_number":51,"context_line":"admin must set \"hw:cpu_emulator_threads_policy\u003dhost\" in the flavor"},{"line_number":52,"context_line":"extra-spec."},{"line_number":53,"context_line":""},{"line_number":54,"context_line":"* hw:cpu_emulator_threads_policy\u003dhost"},{"line_number":55,"context_line":""}],"source_content_type":"text/x-rst","patch_set":10,"id":"df7087c5_34c65f4a","line":52,"range":{"start_line":50,"start_character":0,"end_line":52,"end_character":11},"in_reply_to":"df7087c5_20d61a51","updated":"2018-03-12 17:26:40.000000000","message":"Yes you are right I see what you mean now.","commit_id":"59ebaee15c470cfbd845d359f57a9c20bb26295f"},{"author":{"_account_id":25625,"name":"Tetsuro Nakamura","email":"tetsuro.nakamura.bc@hco.ntt.co.jp","username":"tetsuro0907"},"change_message_id":"0da4bd6532454cc2bb0e7bb169235ac44a93dc11","unresolved":false,"context_lines":[{"line_number":47,"context_line":"option. This is analogous to the existing \"vcpu_pin_set\" and is"},{"line_number":48,"context_line":"formatted the same way."},{"line_number":49,"context_line":""},{"line_number":50,"context_line":"In order to run the hypervisor work on these reserved host CPUs, the"},{"line_number":51,"context_line":"admin must set \"hw:cpu_emulator_threads_policy\u003dhost\" in the flavor"},{"line_number":52,"context_line":"extra-spec."},{"line_number":53,"context_line":""},{"line_number":54,"context_line":"* hw:cpu_emulator_threads_policy\u003dhost"},{"line_number":55,"context_line":""}],"source_content_type":"text/x-rst","patch_set":10,"id":"df7087c5_d856005d","line":52,"range":{"start_line":50,"start_character":0,"end_line":52,"end_character":11},"in_reply_to":"df7087c5_2f86e267","updated":"2018-03-14 09:14:01.000000000","message":"Yes. I intentionally didn\u0027t change emulator threads behavior within the \"shared/dedicated on same host\" spec, but my preference is to guarantee there is at least one shared cpu (by checking if vcpu_pin_set is not empty) and change \"emulator_threads_policy\u003dshared\" option to let emulator threads flow on the shared cpus.\n\nBy doing this we can deprecate \"emulator_threads_policy\u003disolate\" option and in the future remove it with codes related to emulator_threads_policy. This would provide us simpler code, which would provide easier life to NFV related nova maintainers.","commit_id":"59ebaee15c470cfbd845d359f57a9c20bb26295f"},{"author":{"_account_id":7730,"name":"Sahid Orentino Ferdjaoui","email":"sahid.ferdjaoui@industrialdiscipline.com","username":"sahid"},"change_message_id":"411e76f386f9e048312037987a6f7872c03668a1","unresolved":false,"context_lines":[{"line_number":47,"context_line":"option. This is analogous to the existing \"vcpu_pin_set\" and is"},{"line_number":48,"context_line":"formatted the same way."},{"line_number":49,"context_line":""},{"line_number":50,"context_line":"In order to run the hypervisor work on these reserved host CPUs, the"},{"line_number":51,"context_line":"admin must set \"hw:cpu_emulator_threads_policy\u003dhost\" in the flavor"},{"line_number":52,"context_line":"extra-spec."},{"line_number":53,"context_line":""},{"line_number":54,"context_line":"* hw:cpu_emulator_threads_policy\u003dhost"},{"line_number":55,"context_line":""}],"source_content_type":"text/x-rst","patch_set":10,"id":"df7087c5_f577739f","line":52,"range":{"start_line":50,"start_character":0,"end_line":52,"end_character":11},"in_reply_to":"df7087c5_37dbc11b","updated":"2018-03-13 07:16:02.000000000","message":"Chris: Yes I really understand that but even I feel like it\u0027s a bit premature to think about such use-cases.\n\nThe funny point that I did not have noticed... The Jay idea is a bit different than the example. I think if we want make Jay example working, the current state of the spec is necessary since we need that tunable to distinguish the guests which will have their emulthreads running on overhead_pin_set and the other, or do I missed something?","commit_id":"59ebaee15c470cfbd845d359f57a9c20bb26295f"},{"author":{"_account_id":7,"name":"Jay Pipes","email":"jaypipes@gmail.com","username":"jaypipes"},"change_message_id":"da87d9981754f85a5aa5bcc22f5992e4381650c6","unresolved":false,"context_lines":[{"line_number":47,"context_line":"option. This is analogous to the existing \"vcpu_pin_set\" and is"},{"line_number":48,"context_line":"formatted the same way."},{"line_number":49,"context_line":""},{"line_number":50,"context_line":"In order to run the hypervisor work on these reserved host CPUs, the"},{"line_number":51,"context_line":"admin must set \"hw:cpu_emulator_threads_policy\u003dhost\" in the flavor"},{"line_number":52,"context_line":"extra-spec."},{"line_number":53,"context_line":""},{"line_number":54,"context_line":"* hw:cpu_emulator_threads_policy\u003dhost"},{"line_number":55,"context_line":""}],"source_content_type":"text/x-rst","patch_set":10,"id":"df7087c5_20d61a51","line":52,"range":{"start_line":50,"start_character":0,"end_line":52,"end_character":11},"in_reply_to":"df7087c5_9aec40f4","updated":"2018-03-12 16:48:48.000000000","message":"\u003e I don\u0027t know, if we remove that extra-spec button, we can\u0027t have\n \u003e some kind of high-perf instances and low-perf instances running on\n \u003e the same host.\n\nWell, nothing about my suggestion would prevent high-perf instances and low-perf instances running on the same host.\n\nFor example, let\u0027s say the compute host was a 2 NUMA node system with 12 cores on each node. The admin might set vcpu_pin_set to, say, 0-23 and set overhead_pin_set to 12-23, leaving cores 0-11 as the cores to use for dedicated/RT vCPUs.","commit_id":"59ebaee15c470cfbd845d359f57a9c20bb26295f"},{"author":{"_account_id":8768,"name":"Chris Friesen","email":"chris.friesen@windriver.com","username":"cbf123"},"change_message_id":"ca8c40455b6cbb473a4648bda4e19e0a81b1ee3d","unresolved":false,"context_lines":[{"line_number":47,"context_line":"option. This is analogous to the existing \"vcpu_pin_set\" and is"},{"line_number":48,"context_line":"formatted the same way."},{"line_number":49,"context_line":""},{"line_number":50,"context_line":"In order to run the hypervisor work on these reserved host CPUs, the"},{"line_number":51,"context_line":"admin must set \"hw:cpu_emulator_threads_policy\u003dhost\" in the flavor"},{"line_number":52,"context_line":"extra-spec."},{"line_number":53,"context_line":""},{"line_number":54,"context_line":"* hw:cpu_emulator_threads_policy\u003dhost"},{"line_number":55,"context_line":""}],"source_content_type":"text/x-rst","patch_set":10,"id":"df7087c5_bae7bbe4","line":52,"range":{"start_line":50,"start_character":0,"end_line":52,"end_character":11},"in_reply_to":"df7087c5_9aec40f4","updated":"2018-03-12 15:51:55.000000000","message":"Jay, for instances with shared vCPUs on a node then you\u0027ll likely get the best results just letting everything (emulator threads, I/O threads, vCPU threads) float over the same pool of pCPUs.  (I suspect this is why  hw:emulator_threads_policy\u003dISOLATE requires \"dedicated\" vCPUs.)\n\nOnce you have each vCPU thread running on a dedicated pCPU then you want to make sure the emulator threads aren\u0027t running on those dedicated pCPUs (and this is critical for vCPU threads running with a realtime scheduler policy).\n\nIf we had a mix of shared/dedicated vCPUs on this host NUMA node then we could potentially always run emulator threads on the same pCPUs as the \"shared\" vCPUs.  This breaks down if the node is purely intended for \"dedicated\" vCPUs though, in that case you\u0027d need to either allocate from a pool, allocate a separate pCPU for the emulator threads, or run the emulator threads alongside the vCPU threads.\n\nOn possibility would be for us to approve the \"shared/dedicated instances on same host\" spec with the caveat that there needs to be at least one \"shared\" pCPU on each host NUMA node, and we could just always use that pCPU to run the emulator threads.","commit_id":"59ebaee15c470cfbd845d359f57a9c20bb26295f"},{"author":{"_account_id":7730,"name":"Sahid Orentino Ferdjaoui","email":"sahid.ferdjaoui@industrialdiscipline.com","username":"sahid"},"change_message_id":"b8b736819fc1fa95bf860574006a8dd8ca6653dc","unresolved":false,"context_lines":[{"line_number":47,"context_line":"option. This is analogous to the existing \"vcpu_pin_set\" and is"},{"line_number":48,"context_line":"formatted the same way."},{"line_number":49,"context_line":""},{"line_number":50,"context_line":"In order to run the hypervisor work on these reserved host CPUs, the"},{"line_number":51,"context_line":"admin must set \"hw:cpu_emulator_threads_policy\u003dhost\" in the flavor"},{"line_number":52,"context_line":"extra-spec."},{"line_number":53,"context_line":""},{"line_number":54,"context_line":"* hw:cpu_emulator_threads_policy\u003dhost"},{"line_number":55,"context_line":""}],"source_content_type":"text/x-rst","patch_set":10,"id":"df7087c5_209ffab4","line":52,"range":{"start_line":50,"start_character":0,"end_line":52,"end_character":11},"in_reply_to":"df7087c5_bae7bbe4","updated":"2018-03-12 16:39:57.000000000","message":"\u003e Jay, for instances with shared vCPUs on a node then you\u0027ll likely get the best results just letting everything (emulator threads, I/O threads, vCPU threads) float over the same pool of pCPUs. (I suspect this is why hw:emulator_threads_policy\u003dISOLATE requires \"dedicated\" vCPUs.)\n\nEverytime we have an emulator threads sharing pCPU with vCPUs threads we are subject to suffer a latency issue. But that is only a problem for realtime vCPUs usecase.\n\n\u003e On possibility would be for us to approve the \"shared/dedicated instances on same host\" spec with the caveat that there needs to be at least one \"shared\" pCPU on each host NUMA node, and we could just always use that pCPU to run the emulator threads.\n\nOr we should not approve that spec at all. trying to have a host to handle performance and best effort instances does not seem right. That is adding a lot of complexity where we are still trying to design how to make host running performance instances.\n\nBasically I\u0027m agree with the solution proposed by Jay.","commit_id":"59ebaee15c470cfbd845d359f57a9c20bb26295f"},{"author":{"_account_id":7730,"name":"Sahid Orentino Ferdjaoui","email":"sahid.ferdjaoui@industrialdiscipline.com","username":"sahid"},"change_message_id":"62ffb01b73bd989f368fa493c4822803f5bdd768","unresolved":false,"context_lines":[{"line_number":47,"context_line":"option. This is analogous to the existing \"vcpu_pin_set\" and is"},{"line_number":48,"context_line":"formatted the same way."},{"line_number":49,"context_line":""},{"line_number":50,"context_line":"In order to run the hypervisor work on these reserved host CPUs, the"},{"line_number":51,"context_line":"admin must set \"hw:cpu_emulator_threads_policy\u003dhost\" in the flavor"},{"line_number":52,"context_line":"extra-spec."},{"line_number":53,"context_line":""},{"line_number":54,"context_line":"* hw:cpu_emulator_threads_policy\u003dhost"},{"line_number":55,"context_line":""}],"source_content_type":"text/x-rst","patch_set":10,"id":"df7087c5_9aec40f4","line":52,"range":{"start_line":50,"start_character":0,"end_line":52,"end_character":11},"in_reply_to":"df7087c5_ef66f829","updated":"2018-03-12 10:27:58.000000000","message":"I don\u0027t know, if we remove that extra-spec button, we can\u0027t have some kind of high-perf instances and low-perf instances running on the same host. But perhaps it\u0027s not an use case. I remember me been against that spec to allow shared/dedicated instances on the same host :)\n\nI know that Chris was not completely happy too with my solution. Chris can you give to us your feeling?\n\nIf we can be all agree to have a path to follow I will update the spec.","commit_id":"59ebaee15c470cfbd845d359f57a9c20bb26295f"},{"author":{"_account_id":8768,"name":"Chris Friesen","email":"chris.friesen@windriver.com","username":"cbf123"},"change_message_id":"3c1c1387193828fbbb839dd44b85b067407a3add","unresolved":false,"context_lines":[{"line_number":47,"context_line":"option. This is analogous to the existing \"vcpu_pin_set\" and is"},{"line_number":48,"context_line":"formatted the same way."},{"line_number":49,"context_line":""},{"line_number":50,"context_line":"In order to run the hypervisor work on these reserved host CPUs, the"},{"line_number":51,"context_line":"admin must set \"hw:cpu_emulator_threads_policy\u003dhost\" in the flavor"},{"line_number":52,"context_line":"extra-spec."},{"line_number":53,"context_line":""},{"line_number":54,"context_line":"* hw:cpu_emulator_threads_policy\u003dhost"},{"line_number":55,"context_line":""}],"source_content_type":"text/x-rst","patch_set":10,"id":"df7087c5_2f86e267","line":52,"range":{"start_line":50,"start_character":0,"end_line":52,"end_character":11},"in_reply_to":"df7087c5_f577739f","updated":"2018-03-13 17:49:33.000000000","message":"sahid: there are fundamentally two things.  The hw:emulator_threads_policy extra spec is a signal from the flavor that it wants the emulator threads to run elsewhere from the vCPU threads even if it means consuming additional host resources.\n\nThe proposed \"overhead_pin_set\" is a config option on the host to say \"here are some pCPUs to run emulator threads on\".   If we approve the \"shared/dedicated on same host\" spec then the pCPUs for \"shared\" instances could also be used to run emulator threads and we wouldn\u0027t need the \"overhead_pin_set\" config option.\n\nFinally, the question is whether we *require* there to be a separate place for emulator threads on all compute nodes.  If we don\u0027t, then the flavor extra spec is still needed to determine whether an extra pCPU needs to be allocated for the instance.\n\nPersonally I think an argument could be made for hypervisor drivers to ensure vCPU threads can always run at full speed (for libvirt we could require at least one pCPU be reserved for running emulator threads (and possibly \"shared\" vCPU threads)) and then we could drop the flavor extra-spec since all \"dedicated\" vCPUs would run at full speed.  I think this would simplify the code as well, since we wouldn\u0027t have to special-case hw:emulator_threads_policy\u003dISOLATE.","commit_id":"59ebaee15c470cfbd845d359f57a9c20bb26295f"},{"author":{"_account_id":15334,"name":"Stephen Finucane","display_name":"stephenfin","email":"stephenfin@redhat.com","username":"sfinucan"},"change_message_id":"a3d78544d177bcafa66a29cb3076b5cf17ef426e","unresolved":false,"context_lines":[{"line_number":51,"context_line":"admin must set \"hw:cpu_emulator_threads_policy\u003dhost\" in the flavor"},{"line_number":52,"context_line":"extra-spec."},{"line_number":53,"context_line":""},{"line_number":54,"context_line":"* hw:cpu_emulator_threads_policy\u003dhost"},{"line_number":55,"context_line":""},{"line_number":56,"context_line":"Would say the emulator threads of all the guests are going to be"},{"line_number":57,"context_line":"pinned on a global isolated host pCPUs according \u0027overhead_pin_set\u0027"}],"source_content_type":"text/x-rst","patch_set":10,"id":"df7087c5_fb4b59ad","line":54,"range":{"start_line":54,"start_character":5,"end_line":54,"end_character":32},"updated":"2018-03-07 16:21:52.000000000","message":"Did you mean \u0027emulator_threads_policy\u0027?","commit_id":"59ebaee15c470cfbd845d359f57a9c20bb26295f"},{"author":{"_account_id":7,"name":"Jay Pipes","email":"jaypipes@gmail.com","username":"jaypipes"},"change_message_id":"ee241386308202fccec68c6b01e102799bc13339","unresolved":false,"context_lines":[{"line_number":69,"context_line":"Data model impact"},{"line_number":70,"context_line":"-----------------"},{"line_number":71,"context_line":""},{"line_number":72,"context_line":"The CPUEmulatorThreadsPolicy() will be extended to admint a new option"},{"line_number":73,"context_line":"`host`, meaning that the emulator threads are isolated on the pCPUs"},{"line_number":74,"context_line":"set configure with \u0027overhead_pin_set\u0027"},{"line_number":75,"context_line":""}],"source_content_type":"text/x-rst","patch_set":10,"id":"df7087c5_2f7cf0b3","line":72,"range":{"start_line":72,"start_character":51,"end_line":72,"end_character":57},"updated":"2018-03-09 19:57:44.000000000","message":"allow?","commit_id":"59ebaee15c470cfbd845d359f57a9c20bb26295f"},{"author":{"_account_id":7,"name":"Jay Pipes","email":"jaypipes@gmail.com","username":"jaypipes"},"change_message_id":"ee241386308202fccec68c6b01e102799bc13339","unresolved":false,"context_lines":[{"line_number":92,"context_line":"Other end user impact"},{"line_number":93,"context_line":"---------------------"},{"line_number":94,"context_line":""},{"line_number":95,"context_line":"For end users, using the option \u0027cpu_emulator_threads_policy\u003dhost\u0027 is"},{"line_number":96,"context_line":"going to improve the latency of guests running sensitive workloads."},{"line_number":97,"context_line":""},{"line_number":98,"context_line":"Performance Impact"},{"line_number":99,"context_line":"------------------"}],"source_content_type":"text/x-rst","patch_set":10,"id":"df7087c5_2f55d01d","line":96,"range":{"start_line":95,"start_character":0,"end_line":96,"end_character":67},"updated":"2018-03-09 19:57:44.000000000","message":"It won\u0027t improve the latency compared to guests with the emulator_threads_policy\u003disolate spec, though, right? I mean, this is more about allowing the host to specify the pool of pCPUs to use for emulator threads than it is about the performance of a guest (compared with the isolate policy).","commit_id":"59ebaee15c470cfbd845d359f57a9c20bb26295f"},{"author":{"_account_id":8768,"name":"Chris Friesen","email":"chris.friesen@windriver.com","username":"cbf123"},"change_message_id":"ca8c40455b6cbb473a4648bda4e19e0a81b1ee3d","unresolved":false,"context_lines":[{"line_number":92,"context_line":"Other end user impact"},{"line_number":93,"context_line":"---------------------"},{"line_number":94,"context_line":""},{"line_number":95,"context_line":"For end users, using the option \u0027cpu_emulator_threads_policy\u003dhost\u0027 is"},{"line_number":96,"context_line":"going to improve the latency of guests running sensitive workloads."},{"line_number":97,"context_line":""},{"line_number":98,"context_line":"Performance Impact"},{"line_number":99,"context_line":"------------------"}],"source_content_type":"text/x-rst","patch_set":10,"id":"df7087c5_f470ecde","line":96,"range":{"start_line":95,"start_character":0,"end_line":96,"end_character":67},"in_reply_to":"df7087c5_2f55d01d","updated":"2018-03-12 15:51:55.000000000","message":"Right, compared to the existing \"emulator_threads_policy\u003disolate\" the performance might actually be slightly worse in the worst-case but operators will be able to pack more instances onto the same host.","commit_id":"59ebaee15c470cfbd845d359f57a9c20bb26295f"},{"author":{"_account_id":7,"name":"Jay Pipes","email":"jaypipes@gmail.com","username":"jaypipes"},"change_message_id":"ee241386308202fccec68c6b01e102799bc13339","unresolved":false,"context_lines":[{"line_number":105,"context_line":""},{"line_number":106,"context_line":"Deployers who want to use that new feature will have to configure"},{"line_number":107,"context_line":"their flavors to use a dedicated cpu policy (hw:cpu_policy\u003ddedicated),"},{"line_number":108,"context_line":"in a same time set \u0027hw:cpu_emulator_threads\u0027 to \u0027host\u0027."},{"line_number":109,"context_line":""},{"line_number":110,"context_line":"Developer impact"},{"line_number":111,"context_line":"----------------"}],"source_content_type":"text/x-rst","patch_set":10,"id":"df7087c5_6f20a8b1","line":108,"updated":"2018-03-09 19:57:44.000000000","message":"And deployers will also need to set the overhead_pin_set CONF option.\n\nOr, if you went with my idea of getting rid of the whole emulator_threads_policy extra spec, you could just have deployers set the overhead_pin_set CONF option and that\u0027s that.","commit_id":"59ebaee15c470cfbd845d359f57a9c20bb26295f"},{"author":{"_account_id":15334,"name":"Stephen Finucane","display_name":"stephenfin","email":"stephenfin@redhat.com","username":"sfinucan"},"change_message_id":"94959b9fb51524ef7d925fa04d3c35636ceb5335","unresolved":false,"context_lines":[{"line_number":14,"context_line":"Problem description"},{"line_number":15,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"},{"line_number":16,"context_line":""},{"line_number":17,"context_line":"Previously was introduced option \u0027hw:emulator_threads_policy\u0027 which"},{"line_number":18,"context_line":"adds additional pCPU per guest to run emulator threads."},{"line_number":19,"context_line":""},{"line_number":20,"context_line":"While it resolves issues related to spike latency caused by emulator"}],"source_content_type":"text/x-rst","patch_set":11,"id":"bf659307_0a0a60d1","line":17,"range":{"start_line":17,"start_character":33,"end_line":17,"end_character":61},"updated":"2018-03-26 13:16:18.000000000","message":"nit: ``literal``","commit_id":"340298c22c0ed0a1b8d980f010c3264c7853abe1"},{"author":{"_account_id":15334,"name":"Stephen Finucane","display_name":"stephenfin","email":"stephenfin@redhat.com","username":"sfinucan"},"change_message_id":"94959b9fb51524ef7d925fa04d3c35636ceb5335","unresolved":false,"context_lines":[{"line_number":47,"context_line":"requests."},{"line_number":48,"context_line":""},{"line_number":49,"context_line":"The proposed change is to run the emulator threads work on these"},{"line_number":50,"context_line":"shared host CPUs. The admin who would like take benefice of such"},{"line_number":51,"context_line":"improvement for its flavors will have to configure the flavor"},{"line_number":52,"context_line":"extra-specs ``hw:cpu_emulator_threads_policy\u003dshare``."},{"line_number":53,"context_line":""}],"source_content_type":"text/x-rst","patch_set":11,"id":"bf659307_0a36e004","line":50,"range":{"start_line":50,"start_character":48,"end_line":50,"end_character":56},"updated":"2018-03-26 13:16:18.000000000","message":"advantage","commit_id":"340298c22c0ed0a1b8d980f010c3264c7853abe1"},{"author":{"_account_id":15334,"name":"Stephen Finucane","display_name":"stephenfin","email":"stephenfin@redhat.com","username":"sfinucan"},"change_message_id":"94959b9fb51524ef7d925fa04d3c35636ceb5335","unresolved":false,"context_lines":[{"line_number":49,"context_line":"The proposed change is to run the emulator threads work on these"},{"line_number":50,"context_line":"shared host CPUs. The admin who would like take benefice of such"},{"line_number":51,"context_line":"improvement for its flavors will have to configure the flavor"},{"line_number":52,"context_line":"extra-specs ``hw:cpu_emulator_threads_policy\u003dshare``."},{"line_number":53,"context_line":""},{"line_number":54,"context_line":"It is as noted that, the ``hw:cpu_emulator_threads_policy\u003dshare``"},{"line_number":55,"context_line":"already exists but its default behavior where ``CONF.cpu_shared_set``"}],"source_content_type":"text/x-rst","patch_set":11,"id":"bf659307_ca18e882","line":52,"range":{"start_line":52,"start_character":45,"end_line":52,"end_character":50},"updated":"2018-03-26 13:16:18.000000000","message":"IMO, it would make far more sense to use the \u0027isolate\u0027 key here. This would change semantics, but we\u0027re going to be changing semantics for _one_ of the options.\n\nAlternatively, couldn\u0027t we just ignore this key entirely and *always* pin emulator threads to the pool CPUs on \u0027cpu_shared_set\u0027 if it exists and the user requested dedicated CPU or realtime policies?","commit_id":"340298c22c0ed0a1b8d980f010c3264c7853abe1"},{"author":{"_account_id":15334,"name":"Stephen Finucane","display_name":"stephenfin","email":"stephenfin@redhat.com","username":"sfinucan"},"change_message_id":"94959b9fb51524ef7d925fa04d3c35636ceb5335","unresolved":false,"context_lines":[{"line_number":49,"context_line":"The proposed change is to run the emulator threads work on these"},{"line_number":50,"context_line":"shared host CPUs. The admin who would like take benefice of such"},{"line_number":51,"context_line":"improvement for its flavors will have to configure the flavor"},{"line_number":52,"context_line":"extra-specs ``hw:cpu_emulator_threads_policy\u003dshare``."},{"line_number":53,"context_line":""},{"line_number":54,"context_line":"It is as noted that, the ``hw:cpu_emulator_threads_policy\u003dshare``"},{"line_number":55,"context_line":"already exists but its default behavior where ``CONF.cpu_shared_set``"}],"source_content_type":"text/x-rst","patch_set":11,"id":"bf659307_65122d5b","line":52,"range":{"start_line":52,"start_character":17,"end_line":52,"end_character":44},"updated":"2018-03-26 13:16:18.000000000","message":"This doesn\u0027t exist. I think you meant \u0027emulator_threads_policy\u0027","commit_id":"340298c22c0ed0a1b8d980f010c3264c7853abe1"},{"author":{"_account_id":7730,"name":"Sahid Orentino Ferdjaoui","email":"sahid.ferdjaoui@industrialdiscipline.com","username":"sahid"},"change_message_id":"e09a68365e614c5cdf67cdccee34787d7e3b2a39","unresolved":false,"context_lines":[{"line_number":49,"context_line":"The proposed change is to run the emulator threads work on these"},{"line_number":50,"context_line":"shared host CPUs. The admin who would like take benefice of such"},{"line_number":51,"context_line":"improvement for its flavors will have to configure the flavor"},{"line_number":52,"context_line":"extra-specs ``hw:cpu_emulator_threads_policy\u003dshare``."},{"line_number":53,"context_line":""},{"line_number":54,"context_line":"It is as noted that, the ``hw:cpu_emulator_threads_policy\u003dshare``"},{"line_number":55,"context_line":"already exists but its default behavior where ``CONF.cpu_shared_set``"}],"source_content_type":"text/x-rst","patch_set":11,"id":"bf659307_60c53b8e","line":52,"range":{"start_line":52,"start_character":45,"end_line":52,"end_character":50},"in_reply_to":"bf659307_ca18e882","updated":"2018-03-26 13:51:34.000000000","message":"\u003e IMO, it would make far more sense to use the \u0027isolate\u0027 key here. This would change semantics, but we\u0027re going to be changing semantics for _one_ of the options.\n\nCurrently I prefer to keep it like that. I don\u0027t want to remove an option. We could have some users who want to have an addition and dedicated pCPU to make running guest emulator threads and that even if we CONF.cpu_shared_set is configured. \n\n\u003e Alternatively, couldn\u0027t we just ignore this key entirely and *always* pin emulator threads to the pool CPUs on \u0027cpu_shared_set\u0027 if it exists and the user requested dedicated CPU or realtime policies?\n\nWhy would you do that? when configured in dedicated policy the emulator threads float across the guest pCPUs dedicated which are enough to address a lot of use-cases. I don\u0027t see any reason to remove it.\n\n\nThe aim of the spec is to add flexibility, not remove some. I can understand that you are not happy which what has been done previously but that does not seem right to remove the options we have provided and which are addressing use-cases.","commit_id":"340298c22c0ed0a1b8d980f010c3264c7853abe1"},{"author":{"_account_id":15334,"name":"Stephen Finucane","display_name":"stephenfin","email":"stephenfin@redhat.com","username":"sfinucan"},"change_message_id":"94959b9fb51524ef7d925fa04d3c35636ceb5335","unresolved":false,"context_lines":[{"line_number":51,"context_line":"improvement for its flavors will have to configure the flavor"},{"line_number":52,"context_line":"extra-specs ``hw:cpu_emulator_threads_policy\u003dshare``."},{"line_number":53,"context_line":""},{"line_number":54,"context_line":"It is as noted that, the ``hw:cpu_emulator_threads_policy\u003dshare``"},{"line_number":55,"context_line":"already exists but its default behavior where ``CONF.cpu_shared_set``"},{"line_number":56,"context_line":"is not configured on host will remain the same, meaning that the"},{"line_number":57,"context_line":"emulator theads will be pinned on the set of pCPUs dedicated for the"}],"source_content_type":"text/x-rst","patch_set":11,"id":"bf659307_e5e87d17","line":54,"range":{"start_line":54,"start_character":30,"end_line":54,"end_character":57},"updated":"2018-03-26 13:16:18.000000000","message":"drop \u0027cpu_\u0027","commit_id":"340298c22c0ed0a1b8d980f010c3264c7853abe1"},{"author":{"_account_id":15334,"name":"Stephen Finucane","display_name":"stephenfin","email":"stephenfin@redhat.com","username":"sfinucan"},"change_message_id":"94959b9fb51524ef7d925fa04d3c35636ceb5335","unresolved":false,"context_lines":[{"line_number":63,"context_line":"Alternatives"},{"line_number":64,"context_line":"------------"},{"line_number":65,"context_line":""},{"line_number":66,"context_line":"Another option is to hardcode an assumption that the vCPUs number set"},{"line_number":67,"context_line":"against the flavour implicitly includes 1 vCPUs for emulator. eg a"},{"line_number":68,"context_line":"vCPU value of 5 would imply 4 actual vCPUs and 1 system pseudo-vCPU."},{"line_number":69,"context_line":"This would likely be extremely confusing to tenant users, and"},{"line_number":70,"context_line":"developers a like."},{"line_number":71,"context_line":""},{"line_number":72,"context_line":"Data model impact"},{"line_number":73,"context_line":"-----------------"}],"source_content_type":"text/x-rst","patch_set":11,"id":"bf659307_255435ca","line":70,"range":{"start_line":66,"start_character":0,"end_line":70,"end_character":18},"updated":"2018-03-26 13:16:18.000000000","message":"This was an alternative to the original spec, but we already decided against it. What I\u0027m suggesting above sound more like alternatives to _this_ spec.","commit_id":"340298c22c0ed0a1b8d980f010c3264c7853abe1"},{"author":{"_account_id":15334,"name":"Stephen Finucane","display_name":"stephenfin","email":"stephenfin@redhat.com","username":"sfinucan"},"change_message_id":"94959b9fb51524ef7d925fa04d3c35636ceb5335","unresolved":false,"context_lines":[{"line_number":93,"context_line":"Other end user impact"},{"line_number":94,"context_line":"---------------------"},{"line_number":95,"context_line":""},{"line_number":96,"context_line":"For end users, using option ``cpu_emulator_threads_policy\u003dshare`` with"},{"line_number":97,"context_line":"hosts have ``CONF.cpu_shared_set`` configured is going to improve the"},{"line_number":98,"context_line":"latency of guests running sensitive workloads."},{"line_number":99,"context_line":""}],"source_content_type":"text/x-rst","patch_set":11,"id":"bf659307_a55925ff","line":96,"range":{"start_line":96,"start_character":30,"end_line":96,"end_character":34},"updated":"2018-03-26 13:16:18.000000000","message":"drop","commit_id":"340298c22c0ed0a1b8d980f010c3264c7853abe1"},{"author":{"_account_id":15334,"name":"Stephen Finucane","display_name":"stephenfin","email":"stephenfin@redhat.com","username":"sfinucan"},"change_message_id":"94959b9fb51524ef7d925fa04d3c35636ceb5335","unresolved":false,"context_lines":[{"line_number":105,"context_line":"Other deployer impact"},{"line_number":106,"context_line":"---------------------"},{"line_number":107,"context_line":""},{"line_number":108,"context_line":"Deployers who want to use that new feature will have to configure"},{"line_number":109,"context_line":"``CONF.cpu_shared_set`` and then their flavors to use a dedicated CPU"},{"line_number":110,"context_line":"policy ``hw:cpu_policy\u003ddedicated`` in a same time set"},{"line_number":111,"context_line":"``hw:cpu_emulator_threads\u003dshare``."},{"line_number":112,"context_line":""},{"line_number":113,"context_line":"Developer impact"},{"line_number":114,"context_line":"----------------"}],"source_content_type":"text/x-rst","patch_set":11,"id":"bf659307_2502f5b5","line":111,"range":{"start_line":108,"start_character":0,"end_line":111,"end_character":34},"updated":"2018-03-26 13:16:18.000000000","message":"What happens if this is not defined? Do we want to deprecate the ability to configure a dedicated CPU per guest at some point in the future (i.e. make \u0027cpu_shared_set\u0027 a requirement if you want this stuff offloaded?). This seems like something that would simplify accounting significantly going forward.","commit_id":"340298c22c0ed0a1b8d980f010c3264c7853abe1"},{"author":{"_account_id":7730,"name":"Sahid Orentino Ferdjaoui","email":"sahid.ferdjaoui@industrialdiscipline.com","username":"sahid"},"change_message_id":"e09a68365e614c5cdf67cdccee34787d7e3b2a39","unresolved":false,"context_lines":[{"line_number":105,"context_line":"Other deployer impact"},{"line_number":106,"context_line":"---------------------"},{"line_number":107,"context_line":""},{"line_number":108,"context_line":"Deployers who want to use that new feature will have to configure"},{"line_number":109,"context_line":"``CONF.cpu_shared_set`` and then their flavors to use a dedicated CPU"},{"line_number":110,"context_line":"policy ``hw:cpu_policy\u003ddedicated`` in a same time set"},{"line_number":111,"context_line":"``hw:cpu_emulator_threads\u003dshare``."},{"line_number":112,"context_line":""},{"line_number":113,"context_line":"Developer impact"},{"line_number":114,"context_line":"----------------"}],"source_content_type":"text/x-rst","patch_set":11,"id":"bf659307_e08e2b26","line":111,"range":{"start_line":108,"start_character":0,"end_line":111,"end_character":34},"in_reply_to":"bf659307_2502f5b5","updated":"2018-03-26 13:51:34.000000000","message":"As indicated previously if not defined the hw:emulator_threads_policy\u003dshare keep the same and original behavior.\n\nThat does not look right to deprecate this option. There are more and more pCPUs on host and the guests created for NFV purpose are more and more larger. It\u0027s not impossible that somewhere in future we will require to have the emulator threads of guests to run on their own and dedicated pCPUs.\n\nMore of that with the I/O threads future coming soon (I hope) having one addition pCPU reserved for guest seems to be a requirement.","commit_id":"340298c22c0ed0a1b8d980f010c3264c7853abe1"},{"author":{"_account_id":15334,"name":"Stephen Finucane","display_name":"stephenfin","email":"stephenfin@redhat.com","username":"sfinucan"},"change_message_id":"94959b9fb51524ef7d925fa04d3c35636ceb5335","unresolved":false,"context_lines":[{"line_number":118,"context_line":"  will be of particular interest to the Xen hypervisor, if using the"},{"line_number":119,"context_line":"  stub domain feature."},{"line_number":120,"context_line":""},{"line_number":121,"context_line":"* Developers of metric or GUI systems have to take into account that"},{"line_number":122,"context_line":"  host CPU overhead which are going to be consumed by instances with a"},{"line_number":123,"context_line":"  `cpu_emulator_threads` set as `isolate`."},{"line_number":124,"context_line":""},{"line_number":125,"context_line":"Upgrade impact"},{"line_number":126,"context_line":"--------------"}],"source_content_type":"text/x-rst","patch_set":11,"id":"bf659307_25c99577","line":123,"range":{"start_line":121,"start_character":0,"end_line":123,"end_character":42},"updated":"2018-03-26 13:16:18.000000000","message":"Don\u0027t think this is relevant to this particular spec. Rather, it applies to the predecessor spec. Drop it?","commit_id":"340298c22c0ed0a1b8d980f010c3264c7853abe1"},{"author":{"_account_id":7730,"name":"Sahid Orentino Ferdjaoui","email":"sahid.ferdjaoui@industrialdiscipline.com","username":"sahid"},"change_message_id":"e09a68365e614c5cdf67cdccee34787d7e3b2a39","unresolved":false,"context_lines":[{"line_number":118,"context_line":"  will be of particular interest to the Xen hypervisor, if using the"},{"line_number":119,"context_line":"  stub domain feature."},{"line_number":120,"context_line":""},{"line_number":121,"context_line":"* Developers of metric or GUI systems have to take into account that"},{"line_number":122,"context_line":"  host CPU overhead which are going to be consumed by instances with a"},{"line_number":123,"context_line":"  `cpu_emulator_threads` set as `isolate`."},{"line_number":124,"context_line":""},{"line_number":125,"context_line":"Upgrade impact"},{"line_number":126,"context_line":"--------------"}],"source_content_type":"text/x-rst","patch_set":11,"id":"bf659307_60a21b9d","line":123,"range":{"start_line":121,"start_character":0,"end_line":123,"end_character":42},"in_reply_to":"bf659307_25c99577","updated":"2018-03-26 13:51:34.000000000","message":"Yes right.","commit_id":"340298c22c0ed0a1b8d980f010c3264c7853abe1"},{"author":{"_account_id":15334,"name":"Stephen Finucane","display_name":"stephenfin","email":"stephenfin@redhat.com","username":"sfinucan"},"change_message_id":"94959b9fb51524ef7d925fa04d3c35636ceb5335","unresolved":false,"context_lines":[{"line_number":139,"context_line":"Work Items"},{"line_number":140,"context_line":"----------"},{"line_number":141,"context_line":""},{"line_number":142,"context_line":"* Introducing ``CONF.share_cpu_set`` option for compute nodes"},{"line_number":143,"context_line":"* Configuring guest to pin its emulator threads on the"},{"line_number":144,"context_line":"  ``CONF.cpu_shared_set`` when ``emulator_threads_polic\u003dshare``"},{"line_number":145,"context_line":""}],"source_content_type":"text/x-rst","patch_set":11,"id":"bf659307_a5836504","line":142,"range":{"start_line":142,"start_character":21,"end_line":142,"end_character":34},"updated":"2018-03-26 13:16:18.000000000","message":"Isn\u0027t this going to go in the \u0027compute\u0027 group? If so, I think you meant ``CONF.compute.shared_cpu_set``","commit_id":"340298c22c0ed0a1b8d980f010c3264c7853abe1"},{"author":{"_account_id":25625,"name":"Tetsuro Nakamura","email":"tetsuro.nakamura.bc@hco.ntt.co.jp","username":"tetsuro0907"},"change_message_id":"596f3593c07442efffbe840a5029240424053259","unresolved":false,"context_lines":[{"line_number":18,"context_line":"adds additional pCPU per guest to run emulator threads."},{"line_number":19,"context_line":""},{"line_number":20,"context_line":"While it resolves issues related to spike latency caused by emulator"},{"line_number":21,"context_line":"threads running on same pCPUs than vCPUs are pinned on, some operators"},{"line_number":22,"context_line":"have desire to pack all emulator threads on a specific set of pCPUs in"},{"line_number":23,"context_line":"order to allow more pCPUs running vCPUs."},{"line_number":24,"context_line":""}],"source_content_type":"text/x-rst","patch_set":12,"id":"bf659307_4f9371a4","line":21,"range":{"start_line":21,"start_character":30,"end_line":21,"end_character":34},"updated":"2018-03-27 06:29:17.000000000","message":"that?","commit_id":"ef0f697eba0ce387a89b268edaa1b42404b7439f"},{"author":{"_account_id":25625,"name":"Tetsuro Nakamura","email":"tetsuro.nakamura.bc@hco.ntt.co.jp","username":"tetsuro0907"},"change_message_id":"596f3593c07442efffbe840a5029240424053259","unresolved":false,"context_lines":[{"line_number":47,"context_line":"requests."},{"line_number":48,"context_line":""},{"line_number":49,"context_line":"The proposed change is to run the emulator threads work on these"},{"line_number":50,"context_line":"shared host CPUs. The admin who would like take advantage of such"},{"line_number":51,"context_line":"improvement for its flavors will have to configure the flavor"},{"line_number":52,"context_line":"extra-specs ``hw:emulator_threads_policy\u003dshare``."},{"line_number":53,"context_line":""}],"source_content_type":"text/x-rst","patch_set":12,"id":"bf659307_4fcb7171","line":50,"range":{"start_line":50,"start_character":38,"end_line":50,"end_character":42},"updated":"2018-03-27 06:29:17.000000000","message":"like to?","commit_id":"ef0f697eba0ce387a89b268edaa1b42404b7439f"},{"author":{"_account_id":8768,"name":"Chris Friesen","email":"chris.friesen@windriver.com","username":"cbf123"},"change_message_id":"f38477d3f41099180f5da0a2150f9b8f39216a20","unresolved":false,"context_lines":[{"line_number":51,"context_line":"improvement for its flavors will have to configure the flavor"},{"line_number":52,"context_line":"extra-specs ``hw:emulator_threads_policy\u003dshare``."},{"line_number":53,"context_line":""},{"line_number":54,"context_line":"It is as noted that, the ``hw:emulator_threads_policy\u003dshare`` already"},{"line_number":55,"context_line":"exists but its default behavior where ``CONF.cpu_shared_set`` is not"},{"line_number":56,"context_line":"configured on host will remain the same, meaning that the emulator"},{"line_number":57,"context_line":"theads will float on the set of pCPUs dedicated for the guest."},{"line_number":58,"context_line":""},{"line_number":59,"context_line":".. _spec: https://review.openstack.org/#/c/555081/"},{"line_number":60,"context_line":""}],"source_content_type":"text/x-rst","patch_set":12,"id":"bf659307_507ebbe4","line":57,"range":{"start_line":54,"start_character":0,"end_line":57,"end_character":62},"updated":"2018-03-26 19:59:41.000000000","message":"I\u0027m of two minds about this.  One could argue that the original \"hw:emulator_threads_policy\u003disolate\" really just meant \"isolate the emulator threads from the vCPU threads\", in which case we could legitimately just run them on CONF.cpu_shared_set.\n\nHowever, since we currently have the \"isolate the emulator threads from everything else\" behaviour it might make sense to preserve it for extremely sensitive workloads while improving everyone else by default (as your proposal does).","commit_id":"ef0f697eba0ce387a89b268edaa1b42404b7439f"},{"author":{"_account_id":25625,"name":"Tetsuro Nakamura","email":"tetsuro.nakamura.bc@hco.ntt.co.jp","username":"tetsuro0907"},"change_message_id":"596f3593c07442efffbe840a5029240424053259","unresolved":false,"context_lines":[{"line_number":67,"context_line":"flexibility provided to users to isolate guest emulator threads on an"},{"line_number":68,"context_line":"dedicated pCPU or let the guest emulator threads floating across the"},{"line_number":69,"context_line":"whole set of pCPUs dedicated for guest are still valid option and we"},{"line_number":70,"context_line":"should not remove such flexibility."},{"line_number":71,"context_line":""},{"line_number":72,"context_line":"Data model impact"},{"line_number":73,"context_line":"-----------------"}],"source_content_type":"text/x-rst","patch_set":12,"id":"bf659307_0f17b9ef","line":70,"updated":"2018-03-27 06:29:17.000000000","message":"I see. Thanks for putting this in the section!","commit_id":"ef0f697eba0ce387a89b268edaa1b42404b7439f"},{"author":{"_account_id":8768,"name":"Chris Friesen","email":"chris.friesen@windriver.com","username":"cbf123"},"change_message_id":"f38477d3f41099180f5da0a2150f9b8f39216a20","unresolved":false,"context_lines":[{"line_number":106,"context_line":"---------------------"},{"line_number":107,"context_line":""},{"line_number":108,"context_line":"Deployers who want to use that new feature will have to configure"},{"line_number":109,"context_line":"``CONF.cpu_shared_set`` and then their flavors to use a dedicated CPU"},{"line_number":110,"context_line":"policy ``hw:cpu_policy\u003ddedicated`` in a same time set"},{"line_number":111,"context_line":"``hw:emulator_threads\u003dshare``."},{"line_number":112,"context_line":""},{"line_number":113,"context_line":"Developer impact"},{"line_number":114,"context_line":"----------------"}],"source_content_type":"text/x-rst","patch_set":12,"id":"bf659307_70e41fad","line":111,"range":{"start_line":109,"start_character":0,"end_line":111,"end_character":30},"updated":"2018-03-26 19:59:41.000000000","message":"grammar","commit_id":"ef0f697eba0ce387a89b268edaa1b42404b7439f"},{"author":{"_account_id":8768,"name":"Chris Friesen","email":"chris.friesen@windriver.com","username":"cbf123"},"change_message_id":"f38477d3f41099180f5da0a2150f9b8f39216a20","unresolved":false,"context_lines":[{"line_number":135,"context_line":"Work Items"},{"line_number":136,"context_line":"----------"},{"line_number":137,"context_line":""},{"line_number":138,"context_line":"* Introducing ``CONF.compute.share_cpu_set`` option for compute nodes"},{"line_number":139,"context_line":"* Configuring guest to pin its emulator threads on the"},{"line_number":140,"context_line":"  ``CONF.compute.cpu_shared_set`` when"},{"line_number":141,"context_line":"  ``emulator_threads_polic\u003dshare``"}],"source_content_type":"text/x-rst","patch_set":12,"id":"bf659307_f09eef05","line":138,"range":{"start_line":138,"start_character":29,"end_line":138,"end_character":42},"updated":"2018-03-26 19:59:41.000000000","message":"\"cpu_shared_set\", no?","commit_id":"ef0f697eba0ce387a89b268edaa1b42404b7439f"},{"author":{"_account_id":25625,"name":"Tetsuro Nakamura","email":"tetsuro.nakamura.bc@hco.ntt.co.jp","username":"tetsuro0907"},"change_message_id":"596f3593c07442efffbe840a5029240424053259","unresolved":false,"context_lines":[{"line_number":138,"context_line":"* Introducing ``CONF.compute.share_cpu_set`` option for compute nodes"},{"line_number":139,"context_line":"* Configuring guest to pin its emulator threads on the"},{"line_number":140,"context_line":"  ``CONF.compute.cpu_shared_set`` when"},{"line_number":141,"context_line":"  ``emulator_threads_polic\u003dshare``"},{"line_number":142,"context_line":""},{"line_number":143,"context_line":"Dependencies"},{"line_number":144,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"}],"source_content_type":"text/x-rst","patch_set":12,"id":"bf659307_6fabd588","line":141,"range":{"start_line":141,"start_character":21,"end_line":141,"end_character":26},"updated":"2018-03-27 06:29:17.000000000","message":"policy","commit_id":"ef0f697eba0ce387a89b268edaa1b42404b7439f"},{"author":{"_account_id":25625,"name":"Tetsuro Nakamura","email":"tetsuro.nakamura.bc@hco.ntt.co.jp","username":"tetsuro0907"},"change_message_id":"596f3593c07442efffbe840a5029240424053259","unresolved":false,"context_lines":[{"line_number":150,"context_line":""},{"line_number":151,"context_line":"This can be tested in any CI system that is capable of testing the"},{"line_number":152,"context_line":"current NUMA and dedicated CPUs policy. i.e. It requires ability to"},{"line_number":153,"context_line":"use KVM and not merely QEMU. Functionnal tests for the scheduling and"},{"line_number":154,"context_line":"driver bits (libvirt) are going to be added."},{"line_number":155,"context_line":""},{"line_number":156,"context_line":"Documentation Impact"}],"source_content_type":"text/x-rst","patch_set":12,"id":"bf659307_cf8d4132","line":153,"range":{"start_line":153,"start_character":29,"end_line":153,"end_character":40},"updated":"2018-03-27 06:29:17.000000000","message":"Functional","commit_id":"ef0f697eba0ce387a89b268edaa1b42404b7439f"},{"author":{"_account_id":7166,"name":"Sylvain Bauza","email":"sbauza@redhat.com","username":"sbauza"},"change_message_id":"5999d387f905f752731f55f5b5d79b2820a431c6","unresolved":false,"context_lines":[{"line_number":4,"context_line":""},{"line_number":5,"context_line":"https://blueprints.launchpad.net/nova/+spec/libvirt-emulator-threads-policy"},{"line_number":6,"context_line":""},{"line_number":7,"context_line":"The Nova scheduler determines CPU resource utilization and instance"},{"line_number":8,"context_line":"CPU placement based on the number of vCPUs in the flavor. A number of"},{"line_number":9,"context_line":"hypervisors have operations that are being performed on behalf of the"},{"line_number":10,"context_line":"guest instance in the host OS. These operations should be accounted"}],"source_content_type":"text/x-rst","patch_set":13,"id":"bf659307_b8443c75","line":7,"range":{"start_line":7,"start_character":4,"end_line":7,"end_character":19},"updated":"2018-04-03 12:12:07.000000000","message":"... and the Placement API too ;)","commit_id":"d23c724a31ba3c6d4ac74f56ce2e08bdee9b2748"},{"author":{"_account_id":11604,"name":"sean mooney","email":"smooney@redhat.com","username":"sean-k-mooney"},"change_message_id":"7d4afd7b813293ba64bde545a0cfb61cdb026fa6","unresolved":false,"context_lines":[{"line_number":7,"context_line":"The Nova scheduler determines CPU resource utilization and instance"},{"line_number":8,"context_line":"CPU placement based on the number of vCPUs in the flavor. A number of"},{"line_number":9,"context_line":"hypervisors have operations that are being performed on behalf of the"},{"line_number":10,"context_line":"guest instance in the host OS. These operations should be accounted"},{"line_number":11,"context_line":"and scheduled separately, as well as have their own placement policy"},{"line_number":12,"context_line":"controls applied."},{"line_number":13,"context_line":""}],"source_content_type":"text/x-rst","patch_set":13,"id":"bf659307_627ca371","line":10,"in_reply_to":"","updated":"2018-04-03 22:58:25.000000000","message":"Should is a strong word here but that is just a nit. I have talked to operators in the past that believe it should not be acouted for separately or allocated form cores othere then the core used for the gust un less you pay more for the privilege of having the hypervisors not steal cpu cycles form the guest. So there are two facets to this, 1 the technical requirement of the workload and 2 the business policy the operators has chosen.","commit_id":"d23c724a31ba3c6d4ac74f56ce2e08bdee9b2748"},{"author":{"_account_id":8768,"name":"Chris Friesen","email":"chris.friesen@windriver.com","username":"cbf123"},"change_message_id":"591052fce1e6521a8d97bc4cea15a84f66e6ee59","unresolved":false,"context_lines":[{"line_number":18,"context_line":"adds additional pCPU per guest to run emulator threads."},{"line_number":19,"context_line":""},{"line_number":20,"context_line":"While it resolves issues related to spike latency caused by emulator"},{"line_number":21,"context_line":"threads running on same pCPUs than vCPUs are pinned on, some operators"},{"line_number":22,"context_line":"have desire to pack all emulator threads on a specific set of pCPUs in"},{"line_number":23,"context_line":"order to allow more pCPUs running vCPUs."},{"line_number":24,"context_line":""}],"source_content_type":"text/x-rst","patch_set":13,"id":"bf659307_0cc7b547","line":21,"range":{"start_line":21,"start_character":30,"end_line":21,"end_character":34},"updated":"2018-03-27 16:15:25.000000000","message":"nit: \"...on the same pCPUs that...\"","commit_id":"d23c724a31ba3c6d4ac74f56ce2e08bdee9b2748"},{"author":{"_account_id":7166,"name":"Sylvain Bauza","email":"sbauza@redhat.com","username":"sbauza"},"change_message_id":"5999d387f905f752731f55f5b5d79b2820a431c6","unresolved":false,"context_lines":[{"line_number":37,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"},{"line_number":38,"context_line":""},{"line_number":39,"context_line":"To extend flexibility and address use-cases where resources on hosts"},{"line_number":40,"context_line":"are limited a separate spec_ that discusses a change to how we would"},{"line_number":41,"context_line":"like to both simplify the configuration of a compute node with regards"},{"line_number":42,"context_line":"to CPU resource inventory as well as make the quantitative tracking of"},{"line_number":43,"context_line":"dedicated CPU resources consistent with the tracking of shared CPU"}],"source_content_type":"text/x-rst","patch_set":13,"id":"bf659307_982298b8","line":40,"range":{"start_line":40,"start_character":14,"end_line":40,"end_character":28},"updated":"2018-04-03 12:12:07.000000000","message":"maybe saying which spec without needing to click the link.","commit_id":"d23c724a31ba3c6d4ac74f56ce2e08bdee9b2748"},{"author":{"_account_id":7166,"name":"Sylvain Bauza","email":"sbauza@redhat.com","username":"sbauza"},"change_message_id":"5999d387f905f752731f55f5b5d79b2820a431c6","unresolved":false,"context_lines":[{"line_number":44,"context_line":"resources via the placement API, introduces a CONF option"},{"line_number":45,"context_line":"``cpu_shared_set`` which stores a pinset string that indicates the"},{"line_number":46,"context_line":"physical processors that should be used for the ``VCPU`` resource"},{"line_number":47,"context_line":"requests."},{"line_number":48,"context_line":""},{"line_number":49,"context_line":"The proposed change is to run the emulator threads work on these"},{"line_number":50,"context_line":"shared host CPUs. The admin who would like to take advantage of such"}],"source_content_type":"text/x-rst","patch_set":13,"id":"bf659307_13050911","line":47,"updated":"2018-04-03 12:12:07.000000000","message":"AFAIU, you\u0027re telling us that https://review.openstack.org/#/c/555081/ would then be a dependency for this spec, right?","commit_id":"d23c724a31ba3c6d4ac74f56ce2e08bdee9b2748"},{"author":{"_account_id":7,"name":"Jay Pipes","email":"jaypipes@gmail.com","username":"jaypipes"},"change_message_id":"f302d0196d5612211b8140dae220be09848b4c18","unresolved":false,"context_lines":[{"line_number":54,"context_line":"It is as noted that, the ``hw:emulator_threads_policy\u003dshare`` already"},{"line_number":55,"context_line":"exists but its default behavior where ``CONF.cpu_shared_set`` is not"},{"line_number":56,"context_line":"configured on host will remain the same, meaning that the emulator"},{"line_number":57,"context_line":"theads will float on the set of pCPUs dedicated for the guest."},{"line_number":58,"context_line":""},{"line_number":59,"context_line":".. _spec: https://review.openstack.org/#/c/555081/"},{"line_number":60,"context_line":""}],"source_content_type":"text/x-rst","patch_set":13,"id":"bf659307_d2070f1c","line":57,"range":{"start_line":57,"start_character":0,"end_line":57,"end_character":6},"updated":"2018-03-27 18:38:32.000000000","message":"threads","commit_id":"d23c724a31ba3c6d4ac74f56ce2e08bdee9b2748"},{"author":{"_account_id":11604,"name":"sean mooney","email":"smooney@redhat.com","username":"sean-k-mooney"},"change_message_id":"7d4afd7b813293ba64bde545a0cfb61cdb026fa6","unresolved":false,"context_lines":[{"line_number":54,"context_line":"It is as noted that, the ``hw:emulator_threads_policy\u003dshare`` already"},{"line_number":55,"context_line":"exists but its default behavior where ``CONF.cpu_shared_set`` is not"},{"line_number":56,"context_line":"configured on host will remain the same, meaning that the emulator"},{"line_number":57,"context_line":"theads will float on the set of pCPUs dedicated for the guest."},{"line_number":58,"context_line":""},{"line_number":59,"context_line":".. _spec: https://review.openstack.org/#/c/555081/"},{"line_number":60,"context_line":""}],"source_content_type":"text/x-rst","patch_set":13,"id":"bf659307_2276ab71","line":57,"in_reply_to":"bf659307_d2070f1c","updated":"2018-04-03 22:58:25.000000000","message":"+1\nThe use of pCPUs and vCPUs in this spec is somewhat ambiguous given they are not defined and do not corralte directly to the definition jay provided in his spec where they refer to resources classes rather specific hardware elements  or virtualised prcessors as they are used in this spec.","commit_id":"d23c724a31ba3c6d4ac74f56ce2e08bdee9b2748"},{"author":{"_account_id":7166,"name":"Sylvain Bauza","email":"sbauza@redhat.com","username":"sbauza"},"change_message_id":"5999d387f905f752731f55f5b5d79b2820a431c6","unresolved":false,"context_lines":[{"line_number":55,"context_line":"exists but its default behavior where ``CONF.cpu_shared_set`` is not"},{"line_number":56,"context_line":"configured on host will remain the same, meaning that the emulator"},{"line_number":57,"context_line":"theads will float on the set of pCPUs dedicated for the guest."},{"line_number":58,"context_line":""},{"line_number":59,"context_line":".. _spec: https://review.openstack.org/#/c/555081/"},{"line_number":60,"context_line":""},{"line_number":61,"context_line":""}],"source_content_type":"text/x-rst","patch_set":13,"id":"bf659307_132e6991","line":58,"updated":"2018-04-03 12:12:07.000000000","message":"What would be the all values for hw:emulator_threads_policy configuration option ?\nAlso, in case we have other values than just \u0027share\u0027, what would then be the behaviour ?","commit_id":"d23c724a31ba3c6d4ac74f56ce2e08bdee9b2748"},{"author":{"_account_id":11604,"name":"sean mooney","email":"smooney@redhat.com","username":"sean-k-mooney"},"change_message_id":"7d4afd7b813293ba64bde545a0cfb61cdb026fa6","unresolved":false,"context_lines":[{"line_number":55,"context_line":"exists but its default behavior where ``CONF.cpu_shared_set`` is not"},{"line_number":56,"context_line":"configured on host will remain the same, meaning that the emulator"},{"line_number":57,"context_line":"theads will float on the set of pCPUs dedicated for the guest."},{"line_number":58,"context_line":""},{"line_number":59,"context_line":".. _spec: https://review.openstack.org/#/c/555081/"},{"line_number":60,"context_line":""},{"line_number":61,"context_line":""}],"source_content_type":"text/x-rst","patch_set":13,"id":"bf659307_8210d701","line":58,"in_reply_to":"bf659307_132e6991","updated":"2018-04-03 22:58:25.000000000","message":"The other option is isolate where an additional allocation is make in placement/resource tracker for a dedicated/shared cpu to run the emulator thread on.","commit_id":"d23c724a31ba3c6d4ac74f56ce2e08bdee9b2748"},{"author":{"_account_id":9708,"name":"Balazs Gibizer","display_name":"gibi","email":"gibizer@gmail.com","username":"gibi"},"change_message_id":"2a3822fcbfcbc58c4d76829e4e755296de5809f4","unresolved":false,"context_lines":[{"line_number":134,"context_line":""},{"line_number":135,"context_line":"Work Items"},{"line_number":136,"context_line":"----------"},{"line_number":137,"context_line":""},{"line_number":138,"context_line":"* Introducing ``CONF.compute.cpu_shared_set`` option for compute nodes"},{"line_number":139,"context_line":"* Configuring guest to pin its emulator threads on the"},{"line_number":140,"context_line":"  ``CONF.compute.cpu_shared_set`` when"},{"line_number":141,"context_line":"  ``emulator_threads_policy\u003dshare``"}],"source_content_type":"text/x-rst","patch_set":13,"id":"bf659307_7b062989","line":138,"range":{"start_line":137,"start_character":0,"end_line":138,"end_character":70},"updated":"2018-03-27 14:55:34.000000000","message":"Isn\u0027t it the other _spec: https://review.openstack.org/#/c/555081/ the one that introduces this option?","commit_id":"d23c724a31ba3c6d4ac74f56ce2e08bdee9b2748"},{"author":{"_account_id":11604,"name":"sean mooney","email":"smooney@redhat.com","username":"sean-k-mooney"},"change_message_id":"7d4afd7b813293ba64bde545a0cfb61cdb026fa6","unresolved":false,"context_lines":[{"line_number":135,"context_line":"Work Items"},{"line_number":136,"context_line":"----------"},{"line_number":137,"context_line":""},{"line_number":138,"context_line":"* Introducing ``CONF.compute.cpu_shared_set`` option for compute nodes"},{"line_number":139,"context_line":"* Configuring guest to pin its emulator threads on the"},{"line_number":140,"context_line":"  ``CONF.compute.cpu_shared_set`` when"},{"line_number":141,"context_line":"  ``emulator_threads_policy\u003dshare``"}],"source_content_type":"text/x-rst","patch_set":13,"id":"bf659307_a2307ba0","line":138,"in_reply_to":"bf659307_2d3710e6","updated":"2018-04-03 22:58:25.000000000","message":"If jay\u0027s spec is a depdecy of this it makes sense to remove this but if not i guess it can remain","commit_id":"d23c724a31ba3c6d4ac74f56ce2e08bdee9b2748"},{"author":{"_account_id":7730,"name":"Sahid Orentino Ferdjaoui","email":"sahid.ferdjaoui@industrialdiscipline.com","username":"sahid"},"change_message_id":"3441fdf49b501c18fb27aeaf76915707d4ca26c0","unresolved":false,"context_lines":[{"line_number":134,"context_line":""},{"line_number":135,"context_line":"Work Items"},{"line_number":136,"context_line":"----------"},{"line_number":137,"context_line":""},{"line_number":138,"context_line":"* Introducing ``CONF.compute.cpu_shared_set`` option for compute nodes"},{"line_number":139,"context_line":"* Configuring guest to pin its emulator threads on the"},{"line_number":140,"context_line":"  ``CONF.compute.cpu_shared_set`` when"},{"line_number":141,"context_line":"  ``emulator_threads_policy\u003dshare``"}],"source_content_type":"text/x-rst","patch_set":13,"id":"bf659307_86872462","line":138,"range":{"start_line":137,"start_character":0,"end_line":138,"end_character":70},"in_reply_to":"bf659307_7b062989","updated":"2018-03-27 15:08:48.000000000","message":"The option could be introduced by this spec or the other, I don\u0027t think that is going to be an issue.","commit_id":"d23c724a31ba3c6d4ac74f56ce2e08bdee9b2748"},{"author":{"_account_id":7,"name":"Jay Pipes","email":"jaypipes@gmail.com","username":"jaypipes"},"change_message_id":"f302d0196d5612211b8140dae220be09848b4c18","unresolved":false,"context_lines":[{"line_number":134,"context_line":""},{"line_number":135,"context_line":"Work Items"},{"line_number":136,"context_line":"----------"},{"line_number":137,"context_line":""},{"line_number":138,"context_line":"* Introducing ``CONF.compute.cpu_shared_set`` option for compute nodes"},{"line_number":139,"context_line":"* Configuring guest to pin its emulator threads on the"},{"line_number":140,"context_line":"  ``CONF.compute.cpu_shared_set`` when"},{"line_number":141,"context_line":"  ``emulator_threads_policy\u003dshare``"}],"source_content_type":"text/x-rst","patch_set":13,"id":"bf659307_2d3710e6","line":138,"range":{"start_line":137,"start_character":0,"end_line":138,"end_character":70},"in_reply_to":"bf659307_86872462","updated":"2018-03-27 18:38:32.000000000","message":"I\u0027m fine with this spec having the work item to actually introduce CONF.cpu_shared_set.","commit_id":"d23c724a31ba3c6d4ac74f56ce2e08bdee9b2748"},{"author":{"_account_id":9708,"name":"Balazs Gibizer","display_name":"gibi","email":"gibizer@gmail.com","username":"gibi"},"change_message_id":"2a3822fcbfcbc58c4d76829e4e755296de5809f4","unresolved":false,"context_lines":[{"line_number":143,"context_line":"Dependencies"},{"line_number":144,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"},{"line_number":145,"context_line":""},{"line_number":146,"context_line":"None"},{"line_number":147,"context_line":""},{"line_number":148,"context_line":"Testing"},{"line_number":149,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d"}],"source_content_type":"text/x-rst","patch_set":13,"id":"bf659307_bbacc18c","line":146,"updated":"2018-03-27 14:55:34.000000000","message":"At least the _spec: https://review.openstack.org/#/c/555081/ needs to be implemented first as that spec introduces \u0027cpu_shared_set\u0027","commit_id":"d23c724a31ba3c6d4ac74f56ce2e08bdee9b2748"},{"author":{"_account_id":11604,"name":"sean mooney","email":"smooney@redhat.com","username":"sean-k-mooney"},"change_message_id":"7d4afd7b813293ba64bde545a0cfb61cdb026fa6","unresolved":false,"context_lines":[{"line_number":143,"context_line":"Dependencies"},{"line_number":144,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"},{"line_number":145,"context_line":""},{"line_number":146,"context_line":"None"},{"line_number":147,"context_line":""},{"line_number":148,"context_line":"Testing"},{"line_number":149,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d"}],"source_content_type":"text/x-rst","patch_set":13,"id":"bf659307_a2d5db1d","line":146,"in_reply_to":"bf659307_1353c918","updated":"2018-04-03 22:58:25.000000000","message":"I think it makes more sense for this spec to depend on jays standardised cpu tracking spec then the other way around as without intoducing both cpu sets and the new pCPU resource class this will lead to more technical debt to resolve when we make that change in the future.","commit_id":"d23c724a31ba3c6d4ac74f56ce2e08bdee9b2748"},{"author":{"_account_id":7166,"name":"Sylvain Bauza","email":"sbauza@redhat.com","username":"sbauza"},"change_message_id":"5999d387f905f752731f55f5b5d79b2820a431c6","unresolved":false,"context_lines":[{"line_number":143,"context_line":"Dependencies"},{"line_number":144,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"},{"line_number":145,"context_line":""},{"line_number":146,"context_line":"None"},{"line_number":147,"context_line":""},{"line_number":148,"context_line":"Testing"},{"line_number":149,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d"}],"source_content_type":"text/x-rst","patch_set":13,"id":"bf659307_1353c918","line":146,"in_reply_to":"bf659307_87795e94","updated":"2018-04-03 12:12:07.000000000","message":"TBC, you should explain *why* https://review.openstack.org/#/c/555081/ doesn\u0027t need to be a dependency, at least because you want to add CONF.compute.cpu_shared_set by your own spec.","commit_id":"d23c724a31ba3c6d4ac74f56ce2e08bdee9b2748"},{"author":{"_account_id":7730,"name":"Sahid Orentino Ferdjaoui","email":"sahid.ferdjaoui@industrialdiscipline.com","username":"sahid"},"change_message_id":"3441fdf49b501c18fb27aeaf76915707d4ca26c0","unresolved":false,"context_lines":[{"line_number":143,"context_line":"Dependencies"},{"line_number":144,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"},{"line_number":145,"context_line":""},{"line_number":146,"context_line":"None"},{"line_number":147,"context_line":""},{"line_number":148,"context_line":"Testing"},{"line_number":149,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d"}],"source_content_type":"text/x-rst","patch_set":13,"id":"bf659307_e64aa025","line":146,"in_reply_to":"bf659307_bbacc18c","updated":"2018-03-27 15:08:48.000000000","message":"The inverse could also be true. Not sure what we should do in such case. I don\"t want create a dependency where actually it\u0027s not a real one.","commit_id":"d23c724a31ba3c6d4ac74f56ce2e08bdee9b2748"},{"author":{"_account_id":9708,"name":"Balazs Gibizer","display_name":"gibi","email":"gibizer@gmail.com","username":"gibi"},"change_message_id":"12be4534c20bff395216c05b122ef92b464329c8","unresolved":false,"context_lines":[{"line_number":143,"context_line":"Dependencies"},{"line_number":144,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"},{"line_number":145,"context_line":""},{"line_number":146,"context_line":"None"},{"line_number":147,"context_line":""},{"line_number":148,"context_line":"Testing"},{"line_number":149,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d"}],"source_content_type":"text/x-rst","patch_set":13,"id":"bf659307_87795e94","line":146,"in_reply_to":"bf659307_e64aa025","updated":"2018-03-27 16:28:42.000000000","message":"True. Do we want to select the dependency order now, or do it during the implementation?","commit_id":"d23c724a31ba3c6d4ac74f56ce2e08bdee9b2748"},{"author":{"_account_id":7634,"name":"Takashi Natsume","email":"takanattie@gmail.com","username":"natsumet"},"change_message_id":"f8936b269a4343110f80fdaf0e67c9a379832bd6","unresolved":false,"context_lines":[{"line_number":146,"context_line":"Dependencies"},{"line_number":147,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"},{"line_number":148,"context_line":""},{"line_number":149,"context_line":"* The ``CONF.compute.cpu_shared_set` is also defined in \"Standardize"},{"line_number":150,"context_line":"  CPU resource tracking\" spec_. This option could be introduced by"},{"line_number":151,"context_line":"  both of the spec."},{"line_number":152,"context_line":""}],"source_content_type":"text/x-rst","patch_set":15,"id":"bf659307_2d5a1782","line":149,"range":{"start_line":149,"start_character":36,"end_line":149,"end_character":37},"updated":"2018-04-04 07:20:04.000000000","message":"missing \u0027`\u0027","commit_id":"a246d788d6f39a66198110b122c4e28d462b2436"},{"author":{"_account_id":7730,"name":"Sahid Orentino Ferdjaoui","email":"sahid.ferdjaoui@industrialdiscipline.com","username":"sahid"},"change_message_id":"b3a2222e2c4a9621443d8af56e70aafe4b246aaa","unresolved":false,"context_lines":[{"line_number":146,"context_line":"Dependencies"},{"line_number":147,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"},{"line_number":148,"context_line":""},{"line_number":149,"context_line":"* The ``CONF.compute.cpu_shared_set` is also defined in \"Standardize"},{"line_number":150,"context_line":"  CPU resource tracking\" spec_. This option could be introduced by"},{"line_number":151,"context_line":"  both of the spec."},{"line_number":152,"context_line":""}],"source_content_type":"text/x-rst","patch_set":15,"id":"bf659307_68f23a72","line":149,"range":{"start_line":149,"start_character":36,"end_line":149,"end_character":37},"in_reply_to":"bf659307_2d5a1782","updated":"2018-04-04 09:47:05.000000000","message":"Thanks.","commit_id":"a246d788d6f39a66198110b122c4e28d462b2436"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"84185f2ca71c8abd43d923fab0b7a52083212fae","unresolved":false,"context_lines":[{"line_number":2,"context_line":"Libvirt driver emulator threads placement policy"},{"line_number":3,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"},{"line_number":4,"context_line":""},{"line_number":5,"context_line":"https://blueprints.launchpad.net/nova/+spec/libvirt-emulator-threads-policy"},{"line_number":6,"context_line":""},{"line_number":7,"context_line":"The Nova scheduler and the placement API determine CPU resource"},{"line_number":8,"context_line":"utilization and instance CPU placement based on the number of vCPUs in"}],"source_content_type":"text/x-rst","patch_set":16,"id":"bf659307_ecada1ea","line":5,"range":{"start_line":5,"start_character":0,"end_line":5,"end_character":75},"updated":"2018-04-09 14:46:33.000000000","message":"This is already marked complete, why isn\u0027t this tracked for Rocky against blueprint overhead-pin-set?","commit_id":"215782881f7d92a616feb495a85e1a0747ccd9bb"}]}