)]}'
{"/COMMIT_MSG":[{"author":{"_account_id":9708,"name":"Balazs Gibizer","display_name":"gibi","email":"gibizer@gmail.com","username":"gibi"},"change_message_id":"3a98037a5d4db138b8710ecf37c05cc0091841b3","unresolved":true,"context_lines":[{"line_number":8,"context_line":""},{"line_number":9,"context_line":"Previously, the libvirt driver defaulted to 1024 * (# of CPUs) for the"},{"line_number":10,"context_line":"value of domain/cputune/shares in the libvirt XML. This value is then"},{"line_number":11,"context_line":"passed directly by libvirt to the cgroups API. With cgroups v2, there"},{"line_number":12,"context_line":"is now a maximum value of 10000 that can be passed in. This makes Nova"},{"line_number":13,"context_line":"unable to launch instances with more than 9 CPUs on hosts that run"},{"line_number":14,"context_line":"cgroups v2, like Ubuntu Jammy or RHEL 9."},{"line_number":15,"context_line":""}],"source_content_type":"text/x-gerrit-commit-message","patch_set":2,"id":"6d356c0f_d9b7dfd2","line":12,"range":{"start_line":11,"start_character":47,"end_line":12,"end_character":54},"updated":"2022-06-14 07:51:47.000000000","message":"Do we have a libvirt regression then? Or is this change tight to a major libvirt version? As far as I see the libvirt doc[1] still states: \n\u003e  The value should be in range [2, 262144]\n\nSo 10000 should work.\n\n[1] https://libvirt.org/formatdomain.html#cpu-tuning","commit_id":"9556180bc391659fdfdf58ddaf7ff8fcf5ae51a2"},{"author":{"_account_id":8864,"name":"Artom Lifshitz","email":"notartom@gmail.com","username":"artom"},"change_message_id":"939f423da456c5371c667ff94c834714069e3a17","unresolved":true,"context_lines":[{"line_number":8,"context_line":""},{"line_number":9,"context_line":"Previously, the libvirt driver defaulted to 1024 * (# of CPUs) for the"},{"line_number":10,"context_line":"value of domain/cputune/shares in the libvirt XML. This value is then"},{"line_number":11,"context_line":"passed directly by libvirt to the cgroups API. With cgroups v2, there"},{"line_number":12,"context_line":"is now a maximum value of 10000 that can be passed in. This makes Nova"},{"line_number":13,"context_line":"unable to launch instances with more than 9 CPUs on hosts that run"},{"line_number":14,"context_line":"cgroups v2, like Ubuntu Jammy or RHEL 9."},{"line_number":15,"context_line":""}],"source_content_type":"text/x-gerrit-commit-message","patch_set":2,"id":"8b8a7ea7_e62e3d1e","line":12,"range":{"start_line":11,"start_character":47,"end_line":12,"end_character":54},"in_reply_to":"6d356c0f_d9b7dfd2","updated":"2022-06-14 15:03:46.000000000","message":"The max is in cgroups, not libvirt. So libvirt will handle 10000 just fine, pass it straight through to cgroups, and the latter will complain.","commit_id":"9556180bc391659fdfdf58ddaf7ff8fcf5ae51a2"},{"author":{"_account_id":11604,"name":"sean mooney","email":"smooney@redhat.com","username":"sean-k-mooney"},"change_message_id":"d62c04ffcdd7ec2a61468bb33fc97b30b412ff32","unresolved":true,"context_lines":[{"line_number":8,"context_line":""},{"line_number":9,"context_line":"Previously, the libvirt driver defaulted to 1024 * (# of CPUs) for the"},{"line_number":10,"context_line":"value of domain/cputune/shares in the libvirt XML. This value is then"},{"line_number":11,"context_line":"passed directly by libvirt to the cgroups API. With cgroups v2, there"},{"line_number":12,"context_line":"is now a maximum value of 10000 that can be passed in. This makes Nova"},{"line_number":13,"context_line":"unable to launch instances with more than 9 CPUs on hosts that run"},{"line_number":14,"context_line":"cgroups v2, like Ubuntu Jammy or RHEL 9."},{"line_number":15,"context_line":""}],"source_content_type":"text/x-gerrit-commit-message","patch_set":2,"id":"920bc5ba_ef447fa7","line":12,"range":{"start_line":11,"start_character":47,"end_line":12,"end_character":54},"in_reply_to":"6d356c0f_d9b7dfd2","updated":"2022-06-17 17:19:18.000000000","message":"this is not a libvirt regerssion persay.\n\nlibvirt was doing a direct passthough to the underlying cgroups api so that value changed between cgroups v1 and cgroups v2\n\nthere is an libvirt/qemu bug for change in behaivor within libvirt \nhttps://bugzilla.redhat.com/show_bug.cgi?id\u003d2037998\nhttps://gitlab.com/libvirt/libvirt/-/issues/324\n\nif you doploy the same libvirt on a differnt os or revert to croups v1 then we would not hit this issue.","commit_id":"9556180bc391659fdfdf58ddaf7ff8fcf5ae51a2"},{"author":{"_account_id":9708,"name":"Balazs Gibizer","display_name":"gibi","email":"gibizer@gmail.com","username":"gibi"},"change_message_id":"524ee602296af2629ad783fff68e756b5a7148e9","unresolved":true,"context_lines":[{"line_number":8,"context_line":""},{"line_number":9,"context_line":"Previously, the libvirt driver defaulted to 1024 * (# of CPUs) for the"},{"line_number":10,"context_line":"value of domain/cputune/shares in the libvirt XML. This value is then"},{"line_number":11,"context_line":"passed directly by libvirt to the cgroups API. With cgroups v2, there"},{"line_number":12,"context_line":"is now a maximum value of 10000 that can be passed in. This makes Nova"},{"line_number":13,"context_line":"unable to launch instances with more than 9 CPUs on hosts that run"},{"line_number":14,"context_line":"cgroups v2, like Ubuntu Jammy or RHEL 9."},{"line_number":15,"context_line":""}],"source_content_type":"text/x-gerrit-commit-message","patch_set":2,"id":"a222d075_85d6e580","line":12,"range":{"start_line":11,"start_character":47,"end_line":12,"end_character":54},"in_reply_to":"8b8a7ea7_e62e3d1e","updated":"2022-06-17 07:25:58.000000000","message":"Still feels like a failure in libvirt. A failure of abstracting out the hypervisor layer. As a libvirt user what I see that what worked before now stopped working. :/","commit_id":"9556180bc391659fdfdf58ddaf7ff8fcf5ae51a2"},{"author":{"_account_id":9708,"name":"Balazs Gibizer","display_name":"gibi","email":"gibizer@gmail.com","username":"gibi"},"change_message_id":"80452664a895581fde03f63d35ec231043f873f7","unresolved":false,"context_lines":[{"line_number":8,"context_line":""},{"line_number":9,"context_line":"Previously, the libvirt driver defaulted to 1024 * (# of CPUs) for the"},{"line_number":10,"context_line":"value of domain/cputune/shares in the libvirt XML. This value is then"},{"line_number":11,"context_line":"passed directly by libvirt to the cgroups API. With cgroups v2, there"},{"line_number":12,"context_line":"is now a maximum value of 10000 that can be passed in. This makes Nova"},{"line_number":13,"context_line":"unable to launch instances with more than 9 CPUs on hosts that run"},{"line_number":14,"context_line":"cgroups v2, like Ubuntu Jammy or RHEL 9."},{"line_number":15,"context_line":""}],"source_content_type":"text/x-gerrit-commit-message","patch_set":2,"id":"62811154_60f57886","line":12,"range":{"start_line":11,"start_character":47,"end_line":12,"end_character":54},"in_reply_to":"920bc5ba_ef447fa7","updated":"2022-06-23 16:37:18.000000000","message":"OK. so this nova patch is more like a WA until (if ever) libvirt fixes issues/324. This is fine by me. But In general I don\u0027t think that nova is the right place to create a cgroups abstraction just because libvirt failed to do it. :)","commit_id":"9556180bc391659fdfdf58ddaf7ff8fcf5ae51a2"},{"author":{"_account_id":8864,"name":"Artom Lifshitz","email":"notartom@gmail.com","username":"artom"},"change_message_id":"e93e6a9934837cf46b93760f8739ed1942878d0a","unresolved":true,"context_lines":[{"line_number":8,"context_line":""},{"line_number":9,"context_line":"Previously, the libvirt driver defaulted to 1024 * (# of CPUs) for the"},{"line_number":10,"context_line":"value of domain/cputune/shares in the libvirt XML. This value is then"},{"line_number":11,"context_line":"passed directly by libvirt to the cgroups API. With cgroups v2, there"},{"line_number":12,"context_line":"is now a maximum value of 10000 that can be passed in. This makes Nova"},{"line_number":13,"context_line":"unable to launch instances with more than 9 CPUs on hosts that run"},{"line_number":14,"context_line":"cgroups v2, like Ubuntu Jammy or RHEL 9."},{"line_number":15,"context_line":""}],"source_content_type":"text/x-gerrit-commit-message","patch_set":2,"id":"12552abc_f9e3e084","line":12,"range":{"start_line":11,"start_character":47,"end_line":12,"end_character":54},"in_reply_to":"a222d075_85d6e580","updated":"2022-06-17 16:31:35.000000000","message":"I don\u0027t disagree, but I think libvirt has always lived in a weird no man\u0027s land between full hypervisor (so something like Xen or HyperV, which provider a higher level of abstraction), and thin wrapper around bare qemu+kvm, so while I agree that libvirt should fix this (and there\u0027s a libvirt issue tracking this [1]), I\u0027m not so sure that it\u0027s outside of Nova\u0027s scope to try and mitigate this as well.\n\n[1] https://gitlab.com/libvirt/libvirt/-/issues/324","commit_id":"9556180bc391659fdfdf58ddaf7ff8fcf5ae51a2"},{"author":{"_account_id":11604,"name":"sean mooney","email":"smooney@redhat.com","username":"sean-k-mooney"},"change_message_id":"2d9a609f90a4a85b57091dc9e566081c809075f4","unresolved":true,"context_lines":[{"line_number":23,"context_line":"than 10000, their flavors will have to get updates, and their"},{"line_number":24,"context_line":"instances resized."},{"line_number":25,"context_line":""},{"line_number":26,"context_line":"Closes-bug: 1978489"},{"line_number":27,"context_line":"Change-Id: I49d757f5f261b3562ada27e6cf57284f615ca395"}],"source_content_type":"text/x-gerrit-commit-message","patch_set":2,"id":"d5786fb4_60cba849","line":26,"range":{"start_line":26,"start_character":0,"end_line":26,"end_character":6},"updated":"2022-06-13 17:36:50.000000000","message":"im not sure this is close bug i think Partial-Bug: woudl be better.","commit_id":"9556180bc391659fdfdf58ddaf7ff8fcf5ae51a2"},{"author":{"_account_id":8864,"name":"Artom Lifshitz","email":"notartom@gmail.com","username":"artom"},"change_message_id":"939f423da456c5371c667ff94c834714069e3a17","unresolved":false,"context_lines":[{"line_number":23,"context_line":"than 10000, their flavors will have to get updates, and their"},{"line_number":24,"context_line":"instances resized."},{"line_number":25,"context_line":""},{"line_number":26,"context_line":"Closes-bug: 1978489"},{"line_number":27,"context_line":"Change-Id: I49d757f5f261b3562ada27e6cf57284f615ca395"}],"source_content_type":"text/x-gerrit-commit-message","patch_set":2,"id":"19a10ff0_dc7ae107","line":26,"range":{"start_line":26,"start_character":0,"end_line":26,"end_character":6},"in_reply_to":"d5786fb4_60cba849","updated":"2022-06-14 15:03:46.000000000","message":"Done","commit_id":"9556180bc391659fdfdf58ddaf7ff8fcf5ae51a2"}],"/PATCHSET_LEVEL":[{"author":{"_account_id":8864,"name":"Artom Lifshitz","email":"notartom@gmail.com","username":"artom"},"change_message_id":"cf4b8fd586c575533ae96937b5da187d36ff5dcd","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":1,"id":"3326fd8d_ece6c45f","updated":"2022-06-13 17:07:40.000000000","message":"For the upgrade impact, we (Red Hat) discussed this internally. We think upstream it\u0027s fair to ask operators who have set this value to update their flavors and resize, as we consider that they own the risk of setting a value that is so coupled to the underlying host OS.","commit_id":"6571ff43ee8219fa3b6a3353f5cc166b9ec0ebef"},{"author":{"_account_id":6926,"name":"Bogdan Dobrelya","email":"bdobreli@redhat.com","username":"bogdando"},"change_message_id":"01b82fa80184c844455c31e9ecab38f64b8c92ee","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":1,"id":"9c71c3f3_7771ecd4","updated":"2022-01-17 16:06:28.000000000","message":"what would be potential upgrade impact for resize/migrate? Maybe existing VMs with \u003c10k should retain the value instead of switching it to unpredictable defaults coming from libvirt?","commit_id":"6571ff43ee8219fa3b6a3353f5cc166b9ec0ebef"},{"author":{"_account_id":25089,"name":"MarkMielke","email":"mark.mielke@gmail.com","username":"MarkMielke"},"change_message_id":"57c002a94e1637f45b18b65bea87f1663044cb0a","unresolved":true,"context_lines":[],"source_content_type":"","patch_set":1,"id":"fb9a12b5_625dbc6e","in_reply_to":"9c71c3f3_7771ecd4","updated":"2022-06-05 18:02:52.000000000","message":"The bug introduced into libvirt is that it passes through \"shares\" as \"weight\", and because the range is no longer invalid, OpenStack can exhibit failure to launch machines, or failures to migrate machines. I believe this is a bug, as \"shares\" and \"weight\" are different configurations according to cgroups v1 and cgroups v2, but libvirt is treating them as if they are synonymous with no mapping.\n\nI have been patching libvirt for the last year to work around this, by having it pass CPUShares through to systemd instead of CPUWeight, and letting systemd scale it \"properly\" (it maps the linear scale as proportional to the range). I had posted a patch to the libvirt developers, and they seemed to not be fans of the approach - however, it remains broken:\n\nhttps://gitlab.com/libvirt/libvirt/-/issues/161\n\nSo, I\u0027m interesting in seeing this patch, or one like it applied to OpenStack. It may not be perfect - but unless you scale it the same way systemd does with CPUShares, your attempt to scale will also be imperfect.\n\nTechnically the CPUShares or CPUWeight may not be applicable unless you oversubscribe the machines, so to some degree - even if it is wrong, it won\u0027t create breakage. However, if you have an SLA of some sort with a customer who is paying for CPU time, and you give them less time per CPU thread with larger instances because of this problem, they may not be so forgiving.\n\nI waiting to see what the libvirt people do next. So far - my patch to libvirt has worked just fine. OpenStack continues to work, passing \"shares\" through, which gets passed through to systemd as \"CPUShares\" instead of \"CPUWeight\", and systemd scales it properly. So, time until the situation is disrupted once again.","commit_id":"6571ff43ee8219fa3b6a3353f5cc166b9ec0ebef"},{"author":{"_account_id":19234,"name":"Alexey Stupnikov","email":"aleksey.stupnikov@gmail.com","username":"astupnikov"},"change_message_id":"a64eaef5f73e82f9047c3dd0d70613bb7d62ea5b","unresolved":true,"context_lines":[],"source_content_type":"","patch_set":1,"id":"d63a5198_0aa64f50","in_reply_to":"fb9a12b5_625dbc6e","updated":"2022-06-08 10:42:27.000000000","message":"If we are talking about workaround for libvirt bug, then it may make sense to introduce a simpler solution like denominator to force shares into smaller interval?","commit_id":"6571ff43ee8219fa3b6a3353f5cc166b9ec0ebef"},{"author":{"_account_id":11604,"name":"sean mooney","email":"smooney@redhat.com","username":"sean-k-mooney"},"change_message_id":"2d9a609f90a4a85b57091dc9e566081c809075f4","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":2,"id":"fb925fac_722b887c","updated":"2022-06-13 17:36:50.000000000","message":"+0 for now because i cant decide between -1 or +1 really.\n\nthis is a partial fix not a full fix for the bug hence Partial-Bug woudl be more correct.\n\nyou hage noted the limiation with regards to the fact this is platform depent and nova cannot\ngureentee the range that is valid so +1 for that.\n\ni wondering if we need to look a the other quota:* extra specs for other example of this and if we shoudl consider updating the extra spec validation or not.\n\nso +0 to let you and other respond \n\nthe code looks like it should work in general and fix the 90% case so over all im infavor of this approch and you have added the livemigration case which is very nice to see.","commit_id":"9556180bc391659fdfdf58ddaf7ff8fcf5ae51a2"},{"author":{"_account_id":9708,"name":"Balazs Gibizer","display_name":"gibi","email":"gibizer@gmail.com","username":"gibi"},"change_message_id":"378e420bccf3e2ebeb4a10d09e37be79e1ef5f55","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":2,"id":"a4f57759_159b3d0f","updated":"2022-06-14 14:30:49.000000000","message":"I forgot the -1 but this probably worth a -1 for visibility","commit_id":"9556180bc391659fdfdf58ddaf7ff8fcf5ae51a2"},{"author":{"_account_id":9708,"name":"Balazs Gibizer","display_name":"gibi","email":"gibizer@gmail.com","username":"gibi"},"change_message_id":"3a98037a5d4db138b8710ecf37c05cc0091841b3","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":2,"id":"2064bf41_a7f4015b","updated":"2022-06-14 07:51:47.000000000","message":"I have some questions","commit_id":"9556180bc391659fdfdf58ddaf7ff8fcf5ae51a2"},{"author":{"_account_id":8864,"name":"Artom Lifshitz","email":"notartom@gmail.com","username":"artom"},"change_message_id":"c8e2d684e52437c4fd28e5fe4e7a5f24db32e055","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":4,"id":"0266b58d_db018796","updated":"2022-07-14 15:02:33.000000000","message":"So how do we move this forward? I think we all agree that the starvation risk is real, but I don\u0027t think any of us have a better idea than live migration. Or at any rate, we agree that the advantages of removing shares during live migration is enough of an advantage for operators to outweigh the risk of starvation. Should I just add something to the release note?","commit_id":"447e1c2169e16967b365ced3b11b0c9b18691cde"},{"author":{"_account_id":11604,"name":"sean mooney","email":"smooney@redhat.com","username":"sean-k-mooney"},"change_message_id":"d62c04ffcdd7ec2a61468bb33fc97b30b412ff32","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":4,"id":"2908ef44_a17a0e44","updated":"2022-06-17 17:19:18.000000000","message":"just pushing the draftcomment i had from 3 days ago","commit_id":"447e1c2169e16967b365ced3b11b0c9b18691cde"},{"author":{"_account_id":9708,"name":"Balazs Gibizer","display_name":"gibi","email":"gibizer@gmail.com","username":"gibi"},"change_message_id":"63521d43e481bbe58606a7a5796e576924380999","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":5,"id":"4e6e9fb3_7c4cad53","updated":"2022-07-14 18:04:26.000000000","message":"We agreed to use the live migration. Reno is updated to state the possible issue. So I\u0027m OK with this.","commit_id":"f77a9fee5b736899ecc39d33e4f4e4012cee751c"}],"nova/tests/unit/virt/libvirt/test_migration.py":[{"author":{"_account_id":11604,"name":"sean mooney","email":"smooney@redhat.com","username":"sean-k-mooney"},"change_message_id":"2d9a609f90a4a85b57091dc9e566081c809075f4","unresolved":true,"context_lines":[{"line_number":98,"context_line":"        new_xml \u003d migration._update_quota_xml(instance,"},{"line_number":99,"context_line":"                                              etree.fromstring(old_xml))"},{"line_number":100,"context_line":"        new_xml \u003d etree.tostring(new_xml, encoding\u003d\u0027unicode\u0027)"},{"line_number":101,"context_line":"        self.assertEqual(\u0027\u003cdomain\u003e\u003ccputune/\u003e\u003c/domain\u003e\u0027, new_xml)"},{"line_number":102,"context_line":""},{"line_number":103,"context_line":"    def test_update_device_resources_xml_vpmem(self):"},{"line_number":104,"context_line":"        # original xml for vpmems, /dev/dax0.1 and /dev/dax0.2 here"}],"source_content_type":"text/x-python","patch_set":2,"id":"a385393e_34dd5bd5","line":101,"updated":"2022-06-13 17:36:50.000000000","message":"i woudl expect the cputune element to not be present at all in this case.","commit_id":"9556180bc391659fdfdf58ddaf7ff8fcf5ae51a2"},{"author":{"_account_id":8864,"name":"Artom Lifshitz","email":"notartom@gmail.com","username":"artom"},"change_message_id":"939f423da456c5371c667ff94c834714069e3a17","unresolved":false,"context_lines":[{"line_number":98,"context_line":"        new_xml \u003d migration._update_quota_xml(instance,"},{"line_number":99,"context_line":"                                              etree.fromstring(old_xml))"},{"line_number":100,"context_line":"        new_xml \u003d etree.tostring(new_xml, encoding\u003d\u0027unicode\u0027)"},{"line_number":101,"context_line":"        self.assertEqual(\u0027\u003cdomain\u003e\u003ccputune/\u003e\u003c/domain\u003e\u0027, new_xml)"},{"line_number":102,"context_line":""},{"line_number":103,"context_line":"    def test_update_device_resources_xml_vpmem(self):"},{"line_number":104,"context_line":"        # original xml for vpmems, /dev/dax0.1 and /dev/dax0.2 here"}],"source_content_type":"text/x-python","patch_set":2,"id":"e3f9ad77_c441e625","line":101,"in_reply_to":"a385393e_34dd5bd5","updated":"2022-06-14 15:03:46.000000000","message":"Done","commit_id":"9556180bc391659fdfdf58ddaf7ff8fcf5ae51a2"}],"nova/virt/libvirt/driver.py":[{"author":{"_account_id":11604,"name":"sean mooney","email":"smooney@redhat.com","username":"sean-k-mooney"},"change_message_id":"2d9a609f90a4a85b57091dc9e566081c809075f4","unresolved":true,"context_lines":[{"line_number":5514,"context_line":"        if not is_able or CONF.libvirt.virt_type not in (\u0027lxc\u0027, \u0027kvm\u0027, \u0027qemu\u0027):"},{"line_number":5515,"context_line":"            return"},{"line_number":5516,"context_line":""},{"line_number":5517,"context_line":"        if guest.cputune is None:"},{"line_number":5518,"context_line":"            guest.cputune \u003d vconfig.LibvirtConfigGuestCPUTune()"},{"line_number":5519,"context_line":""},{"line_number":5520,"context_line":"        for name in cputuning:"},{"line_number":5521,"context_line":"            key \u003d \"quota:cpu_\" + name"}],"source_content_type":"text/x-python","patch_set":2,"id":"c6797470_ea7e08f8","line":5518,"range":{"start_line":5517,"start_character":0,"end_line":5518,"end_character":63},"updated":"2022-06-13 17:36:50.000000000","message":"i would personally move this into the for loop.\nim not sure if its valid to generate this as an empty xml element.\n\nit look kind of messy to add it with no values set.","commit_id":"9556180bc391659fdfdf58ddaf7ff8fcf5ae51a2"},{"author":{"_account_id":8864,"name":"Artom Lifshitz","email":"notartom@gmail.com","username":"artom"},"change_message_id":"939f423da456c5371c667ff94c834714069e3a17","unresolved":false,"context_lines":[{"line_number":5514,"context_line":"        if not is_able or CONF.libvirt.virt_type not in (\u0027lxc\u0027, \u0027kvm\u0027, \u0027qemu\u0027):"},{"line_number":5515,"context_line":"            return"},{"line_number":5516,"context_line":""},{"line_number":5517,"context_line":"        if guest.cputune is None:"},{"line_number":5518,"context_line":"            guest.cputune \u003d vconfig.LibvirtConfigGuestCPUTune()"},{"line_number":5519,"context_line":""},{"line_number":5520,"context_line":"        for name in cputuning:"},{"line_number":5521,"context_line":"            key \u003d \"quota:cpu_\" + name"}],"source_content_type":"text/x-python","patch_set":2,"id":"7b6075b5_8ff5b647","line":5518,"range":{"start_line":5517,"start_character":0,"end_line":5518,"end_character":63},"in_reply_to":"c6797470_ea7e08f8","updated":"2022-06-14 15:03:46.000000000","message":"Done","commit_id":"9556180bc391659fdfdf58ddaf7ff8fcf5ae51a2"}],"releasenotes/notes/remove-default-cputune-shares-values-85d5ddf4b8e24eaa.yaml":[{"author":{"_account_id":19234,"name":"Alexey Stupnikov","email":"aleksey.stupnikov@gmail.com","username":"astupnikov"},"change_message_id":"a64eaef5f73e82f9047c3dd0d70613bb7d62ea5b","unresolved":true,"context_lines":[{"line_number":1,"context_line":"upgrade:"},{"line_number":2,"context_line":"  - |"},{"line_number":3,"context_line":"    In the libvirt driver, the default value of the ``\u003ccputune\u003e\u003cshares\u003e``"},{"line_number":4,"context_line":"    element has been remved, and is now left to libvirt to decide. This is"},{"line_number":5,"context_line":"    because allowed values are platform dependant, and the previous code was"},{"line_number":6,"context_line":"    not guarateend to be supported on all platforms. If any of your flavors are"},{"line_number":7,"context_line":"    using the quota:cpu_shares extra spec, you may need to resize to a"}],"source_content_type":"text/x-yaml","patch_set":1,"id":"f263b04c_fd7e61ca","line":4,"range":{"start_line":4,"start_character":21,"end_line":4,"end_character":27},"updated":"2022-06-08 10:42:27.000000000","message":"typo","commit_id":"6571ff43ee8219fa3b6a3353f5cc166b9ec0ebef"},{"author":{"_account_id":8864,"name":"Artom Lifshitz","email":"notartom@gmail.com","username":"artom"},"change_message_id":"cf4b8fd586c575533ae96937b5da187d36ff5dcd","unresolved":false,"context_lines":[{"line_number":1,"context_line":"upgrade:"},{"line_number":2,"context_line":"  - |"},{"line_number":3,"context_line":"    In the libvirt driver, the default value of the ``\u003ccputune\u003e\u003cshares\u003e``"},{"line_number":4,"context_line":"    element has been remved, and is now left to libvirt to decide. This is"},{"line_number":5,"context_line":"    because allowed values are platform dependant, and the previous code was"},{"line_number":6,"context_line":"    not guarateend to be supported on all platforms. If any of your flavors are"},{"line_number":7,"context_line":"    using the quota:cpu_shares extra spec, you may need to resize to a"}],"source_content_type":"text/x-yaml","patch_set":1,"id":"2edd3ca2_e1b58568","line":4,"range":{"start_line":4,"start_character":21,"end_line":4,"end_character":27},"in_reply_to":"f263b04c_fd7e61ca","updated":"2022-06-13 17:07:40.000000000","message":"Done","commit_id":"6571ff43ee8219fa3b6a3353f5cc166b9ec0ebef"},{"author":{"_account_id":19234,"name":"Alexey Stupnikov","email":"aleksey.stupnikov@gmail.com","username":"astupnikov"},"change_message_id":"a64eaef5f73e82f9047c3dd0d70613bb7d62ea5b","unresolved":true,"context_lines":[{"line_number":3,"context_line":"    In the libvirt driver, the default value of the ``\u003ccputune\u003e\u003cshares\u003e``"},{"line_number":4,"context_line":"    element has been remved, and is now left to libvirt to decide. This is"},{"line_number":5,"context_line":"    because allowed values are platform dependant, and the previous code was"},{"line_number":6,"context_line":"    not guarateend to be supported on all platforms. If any of your flavors are"},{"line_number":7,"context_line":"    using the quota:cpu_shares extra spec, you may need to resize to a"},{"line_number":8,"context_line":"    supported value before upgrading."}],"source_content_type":"text/x-yaml","patch_set":1,"id":"f9979baa_8c2d1f61","line":6,"range":{"start_line":6,"start_character":8,"end_line":6,"end_character":18},"updated":"2022-06-08 10:42:27.000000000","message":"typo","commit_id":"6571ff43ee8219fa3b6a3353f5cc166b9ec0ebef"},{"author":{"_account_id":8864,"name":"Artom Lifshitz","email":"notartom@gmail.com","username":"artom"},"change_message_id":"cf4b8fd586c575533ae96937b5da187d36ff5dcd","unresolved":false,"context_lines":[{"line_number":3,"context_line":"    In the libvirt driver, the default value of the ``\u003ccputune\u003e\u003cshares\u003e``"},{"line_number":4,"context_line":"    element has been remved, and is now left to libvirt to decide. This is"},{"line_number":5,"context_line":"    because allowed values are platform dependant, and the previous code was"},{"line_number":6,"context_line":"    not guarateend to be supported on all platforms. If any of your flavors are"},{"line_number":7,"context_line":"    using the quota:cpu_shares extra spec, you may need to resize to a"},{"line_number":8,"context_line":"    supported value before upgrading."}],"source_content_type":"text/x-yaml","patch_set":1,"id":"34b5e048_a4d04e36","line":6,"range":{"start_line":6,"start_character":8,"end_line":6,"end_character":18},"in_reply_to":"f9979baa_8c2d1f61","updated":"2022-06-13 17:07:40.000000000","message":"Done","commit_id":"6571ff43ee8219fa3b6a3353f5cc166b9ec0ebef"},{"author":{"_account_id":9708,"name":"Balazs Gibizer","display_name":"gibi","email":"gibizer@gmail.com","username":"gibi"},"change_message_id":"3a98037a5d4db138b8710ecf37c05cc0091841b3","unresolved":true,"context_lines":[],"source_content_type":"","patch_set":2,"id":"23cda13a_66ffd0b2","line":9,"updated":"2022-06-14 07:51:47.000000000","message":"Please note that live migration will remove the default value from the domain automatically. \n\nBtw, can this automatic default removal cause that a VM live migrated to a host will get significantly less CPU shares than it got on the source host? Unfortunately libvirt does not state what will be the default shares value if not provided in the domain. But if the libvirt default is lot less than the nova calculated default was, then I can see trouble. E.g:\n* Have two hosts (A, B) each with two VMs, each VM with a 8 vCPUs, and no shares defined in the flavor so every VM is defaulted to 8 * 1024 shares on both hosts.\n* Nova is upgraded with this patch\n* One VM is now live migrated from host A to host B. On Host B there will be now 3 VMs. Two with 8*1024 shares as before and one with whatever the libvirt default will be. If that default is \u003c\u003c 8 * 1024 then the migrated VM will be starved as far as I understand.","commit_id":"9556180bc391659fdfdf58ddaf7ff8fcf5ae51a2"},{"author":{"_account_id":8864,"name":"Artom Lifshitz","email":"notartom@gmail.com","username":"artom"},"change_message_id":"e93e6a9934837cf46b93760f8739ed1942878d0a","unresolved":true,"context_lines":[],"source_content_type":"","patch_set":2,"id":"73022699_79e3f3af","line":9,"in_reply_to":"221e9b24_bf956e41","updated":"2022-06-17 16:31:35.000000000","message":"\u003e This more and more looks like to me as a failure of libvirt to hide the underlying layer. So if Nova want to fix this well then Nova needs to check the host OS for default and adjust the values based on that. That sounds bad. :/\n\nI think this is where Sean\u0027s idea to deprecate the current quota extra specs and provider something new with a higher level of abstraction comes in.\n\n\u003e To be constructive; can we not change the value during migration for a single VM but change it for every VM on a host at once? I assume these share values can be updated dynamically while the VM is running happily. So can we remove the values from every domains at nova-compute service startup?\n\nWe could, but that would be complementary to this patch, as we still need to not generate the broken XML with values \u003e 10000 for future instances.","commit_id":"9556180bc391659fdfdf58ddaf7ff8fcf5ae51a2"},{"author":{"_account_id":8864,"name":"Artom Lifshitz","email":"notartom@gmail.com","username":"artom"},"change_message_id":"939f423da456c5371c667ff94c834714069e3a17","unresolved":true,"context_lines":[],"source_content_type":"","patch_set":2,"id":"f597256f_238c9e47","line":9,"in_reply_to":"23cda13a_66ffd0b2","updated":"2022-06-14 15:03:46.000000000","message":"So, small note: based on my reading of the libvirt docs, there is no libvirt default, but rather an OS default: \"If this is omitted, it defaults to the OS provided defaults\" [1]\n\nTo answer the question, that\u0027s possible, but that isn\u0027t limited to live migration. In your case, if a VM on A gets rebooted, and the OS default is less than the Nova default, it will suddenly have less CPU shares than the other VMs. I\u0027m not sure how we can avoid this...\n\n[1] https://libvirt.org/formatdomain.html#cpu-tuning","commit_id":"9556180bc391659fdfdf58ddaf7ff8fcf5ae51a2"},{"author":{"_account_id":9708,"name":"Balazs Gibizer","display_name":"gibi","email":"gibizer@gmail.com","username":"gibi"},"change_message_id":"80452664a895581fde03f63d35ec231043f873f7","unresolved":true,"context_lines":[],"source_content_type":"","patch_set":2,"id":"9734470a_6ca84eac","line":9,"in_reply_to":"73022699_79e3f3af","updated":"2022-06-23 16:37:18.000000000","message":"\u003e \u003e This more and more looks like to me as a failure of libvirt to hide the underlying layer. So if Nova want to fix this well then Nova needs to check the host OS for default and adjust the values based on that. That sounds bad. :/\n\u003e \n\u003e I think this is where Sean\u0027s idea to deprecate the current quota extra specs and provider something new with a higher level of abstraction comes in.\n\u003e \n\nThat sounds good.\n\n\u003e \u003e To be constructive; can we not change the value during migration for a single VM but change it for every VM on a host at once? I assume these share values can be updated dynamically while the VM is running happily. So can we remove the values from every domains at nova-compute service startup?\n\u003e \n\u003e We could, but that would be complementary to this patch, as we still need to not generate the broken XML with values \u003e 10000 for future instances.\n\nI\u0027m OK not to generate the default len(cpu) * 1024 value for new instances. \nWhat I\u0027m questioning is if we:\na) want to remove that default value during live-migration of an instance and potentially put the instance in starvation on the destination\nb) instead keep the value during migration, but remove the value at nova-compute startup. This way all the defaulted VM gets the value reset at the same time decreasing the chance of starvation.","commit_id":"9556180bc391659fdfdf58ddaf7ff8fcf5ae51a2"},{"author":{"_account_id":11604,"name":"sean mooney","email":"smooney@redhat.com","username":"sean-k-mooney"},"change_message_id":"71f09d5f05503c17532ef376d34ddac0865633fb","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":2,"id":"1960fc5f_38da48b3","line":9,"in_reply_to":"9734470a_6ca84eac","updated":"2022-07-15 09:44:21.000000000","message":"marking this resolved\n\nthe starvation risk while possible i think is less problematic then requriing cold migration for rooling upgrade or other more disruptive workarounds where you are doing a host os upgrade.\nthis si now docuemented so i think this si safe to proceed wtih.\n\nthanks gibi for raising that concern.","commit_id":"9556180bc391659fdfdf58ddaf7ff8fcf5ae51a2"},{"author":{"_account_id":9708,"name":"Balazs Gibizer","display_name":"gibi","email":"gibizer@gmail.com","username":"gibi"},"change_message_id":"524ee602296af2629ad783fff68e756b5a7148e9","unresolved":true,"context_lines":[],"source_content_type":"","patch_set":2,"id":"221e9b24_bf956e41","line":9,"in_reply_to":"f597256f_238c9e47","updated":"2022-06-17 07:25:58.000000000","message":"This more and more looks like to me as a failure of libvirt to hide the underlying layer. So if Nova want to fix this well then Nova needs to check the host OS for default and adjust the values based on that. That sounds bad. :/\n\nTo be constructive; can we not change the value during migration for a single VM but change it for every VM on a host at once? I assume these share values can be updated dynamically while the VM is running happily. So can we remove the values from every domains at nova-compute service startup?","commit_id":"9556180bc391659fdfdf58ddaf7ff8fcf5ae51a2"}]}