)]}'
{"specs/mitaka/approved/rbd-volume-encryption.rst":[{"author":{"_account_id":15961,"name":"lisali","email":"xiaoyan.li@intel.com","username":"lisali"},"change_message_id":"a9aa894320d855a21c078a87d9aa52c6791bc822","unresolved":false,"context_lines":[{"line_number":21,"context_line":"qemu doesn\u0027t support native encryption at all at the moment. In order to"},{"line_number":22,"context_line":"be able to put any actual encryptors in-between the ceph rbd volume and VM,"},{"line_number":23,"context_line":"we need to attach the volume first to the hypervisor using the kernel\u0027s rbd"},{"line_number":24,"context_line":"driver (krdb), build the encryption layer on top and give libvirt this newly"},{"line_number":25,"context_line":"created, encrypted block device."},{"line_number":26,"context_line":""},{"line_number":27,"context_line":"Use Cases"}],"source_content_type":"text/x-rst","patch_set":2,"id":"1a26ad4f_3b954d3e","line":24,"updated":"2015-11-02 06:53:06.000000000","message":"Any additional package needs to install?","commit_id":"3b7f9355f696ee948bdabcaf00ad05c108570543"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"958a0daf0ed7e474ecddb09a4baea3df1174bbcc","unresolved":false,"context_lines":[{"line_number":21,"context_line":"qemu doesn\u0027t support native encryption at all at the moment. In order to"},{"line_number":22,"context_line":"be able to put any actual encryptors in-between the ceph rbd volume and VM,"},{"line_number":23,"context_line":"we need to attach the volume first to the hypervisor using the kernel\u0027s rbd"},{"line_number":24,"context_line":"driver (krdb), build the encryption layer on top and give libvirt this newly"},{"line_number":25,"context_line":"created, encrypted block device."},{"line_number":26,"context_line":""},{"line_number":27,"context_line":"Use Cases"}],"source_content_type":"text/x-rst","patch_set":2,"id":"1a26ad4f_9c49dbb9","line":24,"in_reply_to":"1a26ad4f_3b954d3e","updated":"2015-11-02 22:19:08.000000000","message":"Or is this `modprobe rbd`?\n\nhttp://docs.ceph.com/docs/v0.71/rbd/rbd-ko/","commit_id":"3b7f9355f696ee948bdabcaf00ad05c108570543"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"958a0daf0ed7e474ecddb09a4baea3df1174bbcc","unresolved":false,"context_lines":[{"line_number":28,"context_line":"----------"},{"line_number":29,"context_line":""},{"line_number":30,"context_line":"Anybody who wants to attach encrypted ceph volumes to VMs. Currently this is"},{"line_number":31,"context_line":"not support at all."},{"line_number":32,"context_line":""},{"line_number":33,"context_line":"Proposed change"},{"line_number":34,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"}],"source_content_type":"text/x-rst","patch_set":2,"id":"1a26ad4f_5c1903d6","line":31,"updated":"2015-11-02 22:19:08.000000000","message":"s/support/supported/","commit_id":"3b7f9355f696ee948bdabcaf00ad05c108570543"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"958a0daf0ed7e474ecddb09a4baea3df1174bbcc","unresolved":false,"context_lines":[{"line_number":37,"context_line":"volume is encrypted. If it is, then it configures the libvirt configuration"},{"line_number":38,"context_line":"to pass through a block device instead of the usual ceph path."},{"line_number":39,"context_line":"We also need to implement connect_volume() and disconnect_volume() which has"},{"line_number":40,"context_line":"been no-ops so far and for encrypted volumes they will handle krdb mappingi and"},{"line_number":41,"context_line":"unmapping the volumes."},{"line_number":42,"context_line":""},{"line_number":43,"context_line":"Nothing changes for the non-encrypted case."}],"source_content_type":"text/x-rst","patch_set":2,"id":"1a26ad4f_bcddb7e1","line":40,"updated":"2015-11-02 22:19:08.000000000","message":"s/mapping/mappingi/","commit_id":"3b7f9355f696ee948bdabcaf00ad05c108570543"},{"author":{"_account_id":1779,"name":"Daniel Berrange","email":"berrange@redhat.com","username":"berrange"},"change_message_id":"bae19db97f52fc2e558a5c4440cecc9c01e0a79f","unresolved":false,"context_lines":[{"line_number":38,"context_line":"to pass through a block device instead of the usual ceph path."},{"line_number":39,"context_line":"We also need to implement connect_volume() and disconnect_volume() which has"},{"line_number":40,"context_line":"been no-ops so far and for encrypted volumes they will handle krdb mappingi and"},{"line_number":41,"context_line":"unmapping the volumes."},{"line_number":42,"context_line":""},{"line_number":43,"context_line":"Nothing changes for the non-encrypted case."},{"line_number":44,"context_line":""}],"source_content_type":"text/x-rst","patch_set":2,"id":"1a26ad4f_f1f58dec","line":41,"updated":"2015-11-03 07:50:04.000000000","message":"I really do not want to see us go down the route of using dm-crypt for RBD device encryption as it forces us to have two totally different I/O stacks for RBD devices. The normal client I/O path handled totally inside QEMU, vs the encrypted code path which will be handled entirely by the in-kernel RBD driver. This adds an ongoing support burden at multiple levels - OpenStack has to maintain 2 different code paths for setting up RBD volumes. Vendors have to support 2 completely separate I/O stacks for RBD volumes. Cloud admins have to debug \u0026 understand two completely separate host OS integration stacks.  \n\nThe direction we have been going in OpenStack has been to move *away* from in-kernel network clients, towards the in-QEMU network clients for a number of reasons. The Nova / host OS configuration is *much* less error prone \u0026 more reliable when using the QEMU client rather than in-kernel - eg look at the history of hacks we\u0027ve had to do where we arbitrarily re-try LVM/dm-crypt commands which randomly fail for unknown reasons.  The in-QEMU clients give better performance isolation of guests, because all the CPU overhead associated with the I/O backend is attributed to the QEMU process, rather than generically seen as kernel overhead. The in-QEMU clients have better performance overall due to simpler + shorter I/O paths. It has better security characteristics as it does not require nova to run rootwrap commands, and the unencrypted data is only exposed to the QEMU VM that uses it, rather than potentially any process on the host.","commit_id":"3b7f9355f696ee948bdabcaf00ad05c108570543"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"4dde2c7184bcb4f8d0fcfa4f684c2b1bff611fe6","unresolved":false,"context_lines":[{"line_number":38,"context_line":"to pass through a block device instead of the usual ceph path."},{"line_number":39,"context_line":"We also need to implement connect_volume() and disconnect_volume() which has"},{"line_number":40,"context_line":"been no-ops so far and for encrypted volumes they will handle krdb mappingi and"},{"line_number":41,"context_line":"unmapping the volumes."},{"line_number":42,"context_line":""},{"line_number":43,"context_line":"Nothing changes for the non-encrypted case."},{"line_number":44,"context_line":""}],"source_content_type":"text/x-rst","patch_set":2,"id":"9a8ffd7b_bdba01e0","line":41,"in_reply_to":"1a26ad4f_f1f58dec","updated":"2015-11-25 02:50:44.000000000","message":"\"It has better security characteristics as it does not require nova to run rootwrap commands, and the unencrypted data is only exposed to the QEMU VM that uses it, rather than potentially any process on the host.\"\n\nSure, QEMU handling this natively is ideal, but you can\u0027t really say not having any encryption support for rbd-backed volumes is better than nothing, which is what we have today - at least from a security standpoint.  And we already have the dm-crypt path for iscsi today, so this just adds to that, again, not ideally, but it\u0027s not like it\u0027s new bad.","commit_id":"3b7f9355f696ee948bdabcaf00ad05c108570543"},{"author":{"_account_id":1779,"name":"Daniel Berrange","email":"berrange@redhat.com","username":"berrange"},"change_message_id":"fe1f0921e14185516116545979e8839ed9697156","unresolved":false,"context_lines":[{"line_number":38,"context_line":"to pass through a block device instead of the usual ceph path."},{"line_number":39,"context_line":"We also need to implement connect_volume() and disconnect_volume() which has"},{"line_number":40,"context_line":"been no-ops so far and for encrypted volumes they will handle krdb mappingi and"},{"line_number":41,"context_line":"unmapping the volumes."},{"line_number":42,"context_line":""},{"line_number":43,"context_line":"Nothing changes for the non-encrypted case."},{"line_number":44,"context_line":""}],"source_content_type":"text/x-rst","patch_set":2,"id":"9a8ffd7b_1096874c","line":41,"in_reply_to":"9a8ffd7b_bdba01e0","updated":"2015-11-25 10:11:48.000000000","message":"The case with iscsi is different, because iSCSI access is already going via the kernel iSCSI client. IOW, you\u0027re not changing the I/O path - it is still QEMU accessing a block device, backed by kernel iscsi client.  With RBD, toggling the encrypted flag on a volume is resulting in a *completely* different I/O stack / path being taken. It is switching from in-QEMU RBD client, to an in-kernel RBD client - this is a very significant change as the two I/O stacks have considerably different performance, tuning characteristics. I really don\u0027t think we should be making such a significant change in I/O path as a side-effect of toggling the encrypted flag.","commit_id":"3b7f9355f696ee948bdabcaf00ad05c108570543"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"958a0daf0ed7e474ecddb09a4baea3df1174bbcc","unresolved":false,"context_lines":[{"line_number":45,"context_line":"Alternatives"},{"line_number":46,"context_line":"------------"},{"line_number":47,"context_line":""},{"line_number":48,"context_line":"None."},{"line_number":49,"context_line":""},{"line_number":50,"context_line":"Data model impact"},{"line_number":51,"context_line":"-----------------"}],"source_content_type":"text/x-rst","patch_set":2,"id":"1a26ad4f_7cfabf44","line":48,"updated":"2015-11-02 22:19:08.000000000","message":"I thought QEMU native encryption support was the preferred alternative from danpb but it\u0027s not available yet. Are there details on that and pros/cons from this design beside the schedule issues?\n\nI\u0027m wondering if there is a tactical path forward that could be replaced eventually with the QEMU native encryption support once that\u0027s available and supported by major distros.","commit_id":"3b7f9355f696ee948bdabcaf00ad05c108570543"},{"author":{"_account_id":1779,"name":"Daniel Berrange","email":"berrange@redhat.com","username":"berrange"},"change_message_id":"bae19db97f52fc2e558a5c4440cecc9c01e0a79f","unresolved":false,"context_lines":[{"line_number":45,"context_line":"Alternatives"},{"line_number":46,"context_line":"------------"},{"line_number":47,"context_line":""},{"line_number":48,"context_line":"None."},{"line_number":49,"context_line":""},{"line_number":50,"context_line":"Data model impact"},{"line_number":51,"context_line":"-----------------"}],"source_content_type":"text/x-rst","patch_set":2,"id":"1a26ad4f_b1715578","line":48,"in_reply_to":"1a26ad4f_7cfabf44","updated":"2015-11-03 07:50:04.000000000","message":"Yes, I am working on adding native support to QEMU for the LUKS format, that can be leveraged by all QEMU drivers (rbd, iscsi, glusterfs, nfs, qcow2). I\u0027d expect this to be ready for use by the Nxxxx release cycle.\n\nI\u0027m really loathe to see us go down the route of implementing a short-term solution for this, as it means we\u0027ll be forced to support 2 solutions for encryption for multiple releases to come, with all the disadvantages I outline above.","commit_id":"3b7f9355f696ee948bdabcaf00ad05c108570543"},{"author":{"_account_id":15961,"name":"lisali","email":"xiaoyan.li@intel.com","username":"lisali"},"change_message_id":"a9aa894320d855a21c078a87d9aa52c6791bc822","unresolved":false,"context_lines":[{"line_number":76,"context_line":"------------------"},{"line_number":77,"context_line":""},{"line_number":78,"context_line":"The krdb driver does not support caching, however, given it\u0027s the only way"},{"line_number":79,"context_line":"to get encrypted volumes working currently, there are no alternatives."},{"line_number":80,"context_line":""},{"line_number":81,"context_line":"Other deployer impact"},{"line_number":82,"context_line":"---------------------"}],"source_content_type":"text/x-rst","patch_set":2,"id":"1a26ad4f_5bfcc971","line":79,"updated":"2015-11-02 06:53:06.000000000","message":"Do we need to log something to show the volumes don\u0027t support caching?","commit_id":"3b7f9355f696ee948bdabcaf00ad05c108570543"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"958a0daf0ed7e474ecddb09a4baea3df1174bbcc","unresolved":false,"context_lines":[{"line_number":82,"context_line":"---------------------"},{"line_number":83,"context_line":""},{"line_number":84,"context_line":"For this functionality the hypervisor\u0027s kernel needs to support krdb, the"},{"line_number":85,"context_line":"in-kernel rbd driver. It has been upstream for ages so should not be a problem."},{"line_number":86,"context_line":""},{"line_number":87,"context_line":"Developer impact"},{"line_number":88,"context_line":"----------------"}],"source_content_type":"text/x-rst","patch_set":2,"id":"1a26ad4f_bc8b77c2","line":85,"updated":"2015-11-02 22:19:08.000000000","message":"I\u0027m assuming the kernel module comes in a certain version of rbd, do we know that minimum version?","commit_id":"3b7f9355f696ee948bdabcaf00ad05c108570543"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"958a0daf0ed7e474ecddb09a4baea3df1174bbcc","unresolved":false,"context_lines":[{"line_number":87,"context_line":"Developer impact"},{"line_number":88,"context_line":"----------------"},{"line_number":89,"context_line":""},{"line_number":90,"context_line":"We will have two different methods to handle ceph volumes based on the"},{"line_number":91,"context_line":"encryption flag which introduces two code paths."},{"line_number":92,"context_line":""},{"line_number":93,"context_line":"Implementation"}],"source_content_type":"text/x-rst","patch_set":2,"id":"1a26ad4f_5cc343ed","line":90,"updated":"2015-11-02 22:19:08.000000000","message":"Seems you should dig into some details here in the \u0027proposed change\u0027 section - I seem to remember you asking several chicken-and-egg type questions in IRC about how this would be implemented in the volume driver layer.","commit_id":"3b7f9355f696ee948bdabcaf00ad05c108570543"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"958a0daf0ed7e474ecddb09a4baea3df1174bbcc","unresolved":false,"context_lines":[{"line_number":103,"context_line":"----------"},{"line_number":104,"context_line":""},{"line_number":105,"context_line":"* break out the rbd libvirt driver from the net libvirt driver as per"},{"line_number":106,"context_line":"  the current libvirt cleanup effort"},{"line_number":107,"context_line":"* implement connect_volume()/disconnect_volume() and unit tests"},{"line_number":108,"context_line":"* re-enable the tempest scenario for encrypted volume in the ceph gate"},{"line_number":109,"context_line":""}],"source_content_type":"text/x-rst","patch_set":2,"id":"1a26ad4f_9ca55b48","line":106,"updated":"2015-11-02 22:19:08.000000000","message":"What libvirt cleanup effort?  Link?\n\nAlso, does any of the work belong in os-brick?\n\nhttps://github.com/openstack/os-brick/blob/master/os_brick/initiator/connector.py#L1407","commit_id":"3b7f9355f696ee948bdabcaf00ad05c108570543"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"958a0daf0ed7e474ecddb09a4baea3df1174bbcc","unresolved":false,"context_lines":[{"line_number":119,"context_line":"For the encrypted case unit tests will be added to verify mapping/unmapping."},{"line_number":120,"context_line":"There is already a tempest encrypted rbd volume test which had been failing,"},{"line_number":121,"context_line":"so is disabled at the moment but will get re-enabled as soon as this"},{"line_number":122,"context_line":"change gets merged (has to be in two commits in order to not get the gate"},{"line_number":123,"context_line":"failing)"},{"line_number":124,"context_line":""},{"line_number":125,"context_line":"Documentation Impact"}],"source_content_type":"text/x-rst","patch_set":2,"id":"1a26ad4f_5c3823ef","line":122,"updated":"2015-11-02 22:19:08.000000000","message":"There should be a change that uses Depends-On in the commit message to show that the nova change makes the encrypted volume test pass with the ceph job, which I thought you had demonstrated at one point.","commit_id":"3b7f9355f696ee948bdabcaf00ad05c108570543"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"958a0daf0ed7e474ecddb09a4baea3df1174bbcc","unresolved":false,"context_lines":[{"line_number":125,"context_line":"Documentation Impact"},{"line_number":126,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"},{"line_number":127,"context_line":""},{"line_number":128,"context_line":"The usage of krbd for the encrypted use case needs to be documented to let"},{"line_number":129,"context_line":"operations know about the kernel dependency."},{"line_number":130,"context_line":""},{"line_number":131,"context_line":"References"}],"source_content_type":"text/x-rst","patch_set":2,"id":"1a26ad4f_1c15cb47","line":128,"updated":"2015-11-02 22:19:08.000000000","message":"Here?\n\nhttp://docs.openstack.org/liberty/config-reference/content/ceph-rados.html","commit_id":"3b7f9355f696ee948bdabcaf00ad05c108570543"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"958a0daf0ed7e474ecddb09a4baea3df1174bbcc","unresolved":false,"context_lines":[{"line_number":131,"context_line":"References"},{"line_number":132,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"},{"line_number":133,"context_line":""},{"line_number":134,"context_line":"In the code review the consensus was that it needs a spec, but otherwise"},{"line_number":135,"context_line":"the code has already been implemented."}],"source_content_type":"text/x-rst","patch_set":2,"id":"1a26ad4f_3f019905","line":134,"updated":"2015-11-02 22:19:08.000000000","message":"Might want to link to the code POC as a reference.","commit_id":"3b7f9355f696ee948bdabcaf00ad05c108570543"},{"author":{"_account_id":3189,"name":"Chet Burgess","email":"cfb@metacloud.com","username":"cfb-n"},"change_message_id":"7c20fd2a0f448d575180a23481eabe53a010fc23","unresolved":false,"context_lines":[{"line_number":12,"context_line":""},{"line_number":13,"context_line":"Instead of using qemu\u0027s librbd driver to access ceph volumes we need to attach"},{"line_number":14,"context_line":"the volume to the hypervisor using the in-kernel rbd driver."},{"line_number":15,"context_line":""},{"line_number":16,"context_line":"Problem description"},{"line_number":17,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"},{"line_number":18,"context_line":""}],"source_content_type":"text/x-rst","patch_set":3,"id":"da85f559_5fb33b0a","line":15,"updated":"2015-11-11 21:53:16.000000000","message":"In general, I\u0027m in favor of doing as little as possible in the baremetal kernel, especially data access. Ideally the baremetal kernel is involved in acceleration and minimal networking for the VM and nothing else. Everything else should be confined to a per VM process and/or process space where possible.","commit_id":"4333eebe0db61d741c491a13408a37ac558758f9"},{"author":{"_account_id":8582,"name":"Zoltan Arnold Nagy","email":"nag@zurich.ibm.com","username":"zoltan"},"change_message_id":"c77a53a3bb1d52b325869cae36368f28c7b8d7e6","unresolved":false,"context_lines":[{"line_number":12,"context_line":""},{"line_number":13,"context_line":"Instead of using qemu\u0027s librbd driver to access ceph volumes we need to attach"},{"line_number":14,"context_line":"the volume to the hypervisor using the in-kernel rbd driver."},{"line_number":15,"context_line":""},{"line_number":16,"context_line":"Problem description"},{"line_number":17,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"},{"line_number":18,"context_line":""}],"source_content_type":"text/x-rst","patch_set":3,"id":"da85f559_b4db457b","line":15,"in_reply_to":"da85f559_5fb33b0a","updated":"2015-11-12 14:54:35.000000000","message":"Sure, this would be the general direction once qemu has native encryption support. I\u0027d prefer not to use the hypervisor kernel at all for tasks like this, but at the moment this is the only doable way. Also, the current encryption framework is using hypervisor mapped drives s well, so this would be a simple extension to that until we have the first in place.","commit_id":"4333eebe0db61d741c491a13408a37ac558758f9"},{"author":{"_account_id":3189,"name":"Chet Burgess","email":"cfb@metacloud.com","username":"cfb-n"},"change_message_id":"7c20fd2a0f448d575180a23481eabe53a010fc23","unresolved":false,"context_lines":[{"line_number":27,"context_line":"This requires the kernel to be compiled with either RBD block device support"},{"line_number":28,"context_line":"built-in or at least available as a module. It also requires the ceph config"},{"line_number":29,"context_line":"file to be present on the machine, as well as the package containing the"},{"line_number":30,"context_line":"\u0027rbd\u0027 utility."},{"line_number":31,"context_line":""},{"line_number":32,"context_line":"Use Cases"},{"line_number":33,"context_line":"----------"}],"source_content_type":"text/x-rst","patch_set":3,"id":"da85f559_3c0be52b","line":30,"updated":"2015-11-11 21:53:16.000000000","message":"Does this have any impact on live migrations? Presumably its safe to have 2 process accessing an dm-crypt volume on different servers?","commit_id":"4333eebe0db61d741c491a13408a37ac558758f9"},{"author":{"_account_id":15524,"name":"dane-fichter","email":"dane.fichter@jhuapl.edu","username":"dane-fichter"},"change_message_id":"13324dd223d4d4c2655d5c43ed6026763c8ded3f","unresolved":false,"context_lines":[{"line_number":42,"context_line":"volume is encrypted. If it is, then it configures the libvirt configuration"},{"line_number":43,"context_line":"to pass through a block device instead of the usual ceph path."},{"line_number":44,"context_line":"We also need to implement connect_volume() and disconnect_volume() which has"},{"line_number":45,"context_line":"been no-ops so far and for encrypted volumes they will handle krdb mapping and"},{"line_number":46,"context_line":"unmapping the volumes."},{"line_number":47,"context_line":""},{"line_number":48,"context_line":"Nothing changes for the non-encrypted case."}],"source_content_type":"text/x-rst","patch_set":3,"id":"da85f559_fcd9f457","line":45,"updated":"2015-11-11 14:40:30.000000000","message":"What about the LUKS encryptor? What about ephemeral storage encryption? There are definitely examples of cases where calling connect_volume() and disconnect_volume() do more than a no-op encryptor.","commit_id":"4333eebe0db61d741c491a13408a37ac558758f9"},{"author":{"_account_id":8582,"name":"Zoltan Arnold Nagy","email":"nag@zurich.ibm.com","username":"zoltan"},"change_message_id":"c77a53a3bb1d52b325869cae36368f28c7b8d7e6","unresolved":false,"context_lines":[{"line_number":42,"context_line":"volume is encrypted. If it is, then it configures the libvirt configuration"},{"line_number":43,"context_line":"to pass through a block device instead of the usual ceph path."},{"line_number":44,"context_line":"We also need to implement connect_volume() and disconnect_volume() which has"},{"line_number":45,"context_line":"been no-ops so far and for encrypted volumes they will handle krdb mapping and"},{"line_number":46,"context_line":"unmapping the volumes."},{"line_number":47,"context_line":""},{"line_number":48,"context_line":"Nothing changes for the non-encrypted case."}],"source_content_type":"text/x-rst","patch_set":3,"id":"da85f559_340bd5d7","line":45,"in_reply_to":"da85f559_fcd9f457","updated":"2015-11-12 14:54:35.000000000","message":"You\u0027re right, in the spec I was strickly talking about the ceph case. For existing iSCSI/FC volume encryption, these are not no-ops.","commit_id":"4333eebe0db61d741c491a13408a37ac558758f9"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"d576abbe6d3aeaf691ad33c5bdfb0d86862b5544","unresolved":false,"context_lines":[{"line_number":57,"context_line":"The drawback is that this is still months out as this is a planned feature"},{"line_number":58,"context_line":"only, not an implemented one that we could already build OpenStack support for,"},{"line_number":59,"context_line":"and would mean that OpenStack won\u0027t have RBD encryption available during most"},{"line_number":60,"context_line":"of 2016."},{"line_number":61,"context_line":""},{"line_number":62,"context_line":"Data model impact"},{"line_number":63,"context_line":"-----------------"}],"source_content_type":"text/x-rst","patch_set":3,"id":"da85f559_8d4422b7","line":60,"updated":"2015-11-11 19:14:36.000000000","message":"In PS2 Dan said:\n\n\"I\u0027m really loathe to see us go down the route of implementing a short-term solution for this, as it means we\u0027ll be forced to support 2 solutions for encryption for multiple releases to come, with all the disadvantages I outline above.\"\n\n@Dan, what if we had the approach outlined here until native QEMU encryption lands, and then we deprecate (for a cycle) and then remove the non-native QEMU encryption approach here?  I\u0027m not sure I get the support concern there, unless that\u0027s more from a vendor perspective?","commit_id":"4333eebe0db61d741c491a13408a37ac558758f9"},{"author":{"_account_id":8582,"name":"Zoltan Arnold Nagy","email":"nag@zurich.ibm.com","username":"zoltan"},"change_message_id":"b52b819192557e24f864686972fc5a84d80d30e9","unresolved":false,"context_lines":[{"line_number":57,"context_line":"The drawback is that this is still months out as this is a planned feature"},{"line_number":58,"context_line":"only, not an implemented one that we could already build OpenStack support for,"},{"line_number":59,"context_line":"and would mean that OpenStack won\u0027t have RBD encryption available during most"},{"line_number":60,"context_line":"of 2016."},{"line_number":61,"context_line":""},{"line_number":62,"context_line":"Data model impact"},{"line_number":63,"context_line":"-----------------"}],"source_content_type":"text/x-rst","patch_set":3,"id":"da85f559_fb917653","line":60,"in_reply_to":"da85f559_0a22e6b9","updated":"2015-11-13 15:34:08.000000000","message":"I assume that qemu would gain LUKS support so the on-disk format is the same as with our current encryptors. Once we switch the IO paths (deprecate the hypervisor-based approach that this spec proposes) I would assume it to just work.","commit_id":"4333eebe0db61d741c491a13408a37ac558758f9"},{"author":{"_account_id":8582,"name":"Zoltan Arnold Nagy","email":"nag@zurich.ibm.com","username":"zoltan"},"change_message_id":"c77a53a3bb1d52b325869cae36368f28c7b8d7e6","unresolved":false,"context_lines":[{"line_number":57,"context_line":"The drawback is that this is still months out as this is a planned feature"},{"line_number":58,"context_line":"only, not an implemented one that we could already build OpenStack support for,"},{"line_number":59,"context_line":"and would mean that OpenStack won\u0027t have RBD encryption available during most"},{"line_number":60,"context_line":"of 2016."},{"line_number":61,"context_line":""},{"line_number":62,"context_line":"Data model impact"},{"line_number":63,"context_line":"-----------------"}],"source_content_type":"text/x-rst","patch_set":3,"id":"da85f559_94eac1d9","line":60,"in_reply_to":"da85f559_26ae59e6","updated":"2015-11-12 14:54:35.000000000","message":"Wouldn\u0027t this argument hold true as well for how cinder volume encryption is implemented?","commit_id":"4333eebe0db61d741c491a13408a37ac558758f9"},{"author":{"_account_id":1779,"name":"Daniel Berrange","email":"berrange@redhat.com","username":"berrange"},"change_message_id":"5c2cdae8d05902b610c74b2ab01161519929bff4","unresolved":false,"context_lines":[{"line_number":57,"context_line":"The drawback is that this is still months out as this is a planned feature"},{"line_number":58,"context_line":"only, not an implemented one that we could already build OpenStack support for,"},{"line_number":59,"context_line":"and would mean that OpenStack won\u0027t have RBD encryption available during most"},{"line_number":60,"context_line":"of 2016."},{"line_number":61,"context_line":""},{"line_number":62,"context_line":"Data model impact"},{"line_number":63,"context_line":"-----------------"}],"source_content_type":"text/x-rst","patch_set":3,"id":"da85f559_26ae59e6","line":60,"in_reply_to":"da85f559_8d4422b7","updated":"2015-11-12 09:17:17.000000000","message":"Deleting it is hard because we have to cope with the fact that the compute host may have pre-existing VMs running from earlier versions of Nova that use this feature.\n\nThe support concern is not just from a vendor POV - it is from both a vendor \u0026 an upstream POV. It is about the burden of having to deal with two completely separate ways to manage RBD storage. That\u0027s double the potential for bugs - new bugs that will be invisible most of the time and suddenly appear when you turn on encryption, due Nova switching to a different storage stack \u0026 code path. Two separate storage stacks for developers and administrators to understand and troubleshoot. Two separate storage stacks that you need to consider performance benchmarking of. Turning on encryption for a volume means that network and compute overhead associated with the VM and controllable via cgroups is now all attributed to the kernel and not controllable, so you loose ability todo VM performance isolation. It is an overall bad idea IMHO","commit_id":"4333eebe0db61d741c491a13408a37ac558758f9"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"7bf26d2398e0f89415fe33ab535fad01c4ccd160","unresolved":false,"context_lines":[{"line_number":57,"context_line":"The drawback is that this is still months out as this is a planned feature"},{"line_number":58,"context_line":"only, not an implemented one that we could already build OpenStack support for,"},{"line_number":59,"context_line":"and would mean that OpenStack won\u0027t have RBD encryption available during most"},{"line_number":60,"context_line":"of 2016."},{"line_number":61,"context_line":""},{"line_number":62,"context_line":"Data model impact"},{"line_number":63,"context_line":"-----------------"}],"source_content_type":"text/x-rst","patch_set":3,"id":"da85f559_0a22e6b9","line":60,"in_reply_to":"da85f559_94eac1d9","updated":"2015-11-13 14:53:00.000000000","message":"Is there a possible migration path from this approach to native QEMU encryption once that lands?\n\nRe: the performance concerns, I\u0027m assuming an operator that wants to use encrypted RBD volumes is placing a priority on security over performance so that\u0027s less of an issue for them, it\u0027s like someone is required to use this by default.","commit_id":"4333eebe0db61d741c491a13408a37ac558758f9"},{"author":{"_account_id":1779,"name":"Daniel Berrange","email":"berrange@redhat.com","username":"berrange"},"change_message_id":"4517d6b7cedf710a8801ede75caf483aefd64b85","unresolved":false,"context_lines":[{"line_number":57,"context_line":"The drawback is that this is still months out as this is a planned feature"},{"line_number":58,"context_line":"only, not an implemented one that we could already build OpenStack support for,"},{"line_number":59,"context_line":"and would mean that OpenStack won\u0027t have RBD encryption available during most"},{"line_number":60,"context_line":"of 2016."},{"line_number":61,"context_line":""},{"line_number":62,"context_line":"Data model impact"},{"line_number":63,"context_line":"-----------------"}],"source_content_type":"text/x-rst","patch_set":3,"id":"da85f559_fb697626","line":60,"in_reply_to":"da85f559_fb917653","updated":"2015-11-13 15:40:41.000000000","message":"The on-disk format supports a *huge* range of tunable parameters and I would not expect 100% parity in the total set of combinations that are supported in both in an initial implementation. So just because the on-disk format is standard, doesn\u0027t guarantee that the impl can just be switched for existing data. In addition the only way to switch a guest is to shut it down and start it again. Historically we\u0027ve allowed for Nova to be upgraded with *existing* VMs remaining active. When we previously changed directory layout for instance local disks, we had to keep compatibility around for handling existing running VMs with the old layout for multiple releases, as we don\u0027t require users to restart their VMs on a host during upgrade.","commit_id":"4333eebe0db61d741c491a13408a37ac558758f9"},{"author":{"_account_id":15524,"name":"dane-fichter","email":"dane.fichter@jhuapl.edu","username":"dane-fichter"},"change_message_id":"13324dd223d4d4c2655d5c43ed6026763c8ded3f","unresolved":false,"context_lines":[{"line_number":88,"context_line":"------------------"},{"line_number":89,"context_line":""},{"line_number":90,"context_line":"The in-kernel rdb driver does not support caching, however, given it\u0027s the only"},{"line_number":91,"context_line":"way to get encrypted volumes working at the time of writing this spec, this is"},{"line_number":92,"context_line":"known tradeoff."},{"line_number":93,"context_line":""},{"line_number":94,"context_line":"Other deployer impact"}],"source_content_type":"text/x-rst","patch_set":3,"id":"da85f559_1cc63868","line":91,"updated":"2015-11-11 14:40:30.000000000","message":"Still not following how this is the \u0027only way to get encrypted volumes working\u0027. Perhaps it is the only way to get encrypted ceph volumes working?","commit_id":"4333eebe0db61d741c491a13408a37ac558758f9"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"0431e100b3d27fa68c5b5d8247021549516873b3","unresolved":false,"context_lines":[{"line_number":88,"context_line":"------------------"},{"line_number":89,"context_line":""},{"line_number":90,"context_line":"The in-kernel rdb driver does not support caching, however, given it\u0027s the only"},{"line_number":91,"context_line":"way to get encrypted volumes working at the time of writing this spec, this is"},{"line_number":92,"context_line":"known tradeoff."},{"line_number":93,"context_line":""},{"line_number":94,"context_line":"Other deployer impact"}],"source_content_type":"text/x-rst","patch_set":3,"id":"da85f559_6de3fea4","line":91,"in_reply_to":"da85f559_1cc63868","updated":"2015-11-11 19:15:52.000000000","message":"Yes, this is meant for ceph-backed volume encryption, not like iSCIS/fibrechannel/LVM etc.","commit_id":"4333eebe0db61d741c491a13408a37ac558758f9"},{"author":{"_account_id":3189,"name":"Chet Burgess","email":"cfb@metacloud.com","username":"cfb-n"},"change_message_id":"7c20fd2a0f448d575180a23481eabe53a010fc23","unresolved":false,"context_lines":[{"line_number":99,"context_line":"implementation is able to talk to any recent ceph\u0027s rbd volumes, so there"},{"line_number":100,"context_line":"is no version dependence. It\u0027s true that usually the kernel driver lacks"},{"line_number":101,"context_line":"behind in features, but again, these shouldn\u0027t have any practical impact."},{"line_number":102,"context_line":""},{"line_number":103,"context_line":"Developer impact"},{"line_number":104,"context_line":"----------------"},{"line_number":105,"context_line":""}],"source_content_type":"text/x-rst","patch_set":3,"id":"da85f559_bfefc71c","line":102,"updated":"2015-11-11 21:53:16.000000000","message":"I\u0027m not entirely sure that there is *no impact* to having down reved clients. There have been different features and functionality released in the past that have required newer clients and thus a newer kernel (http://ceph.com/planet/feature-set-mismatch-error-on-ceph-kernel-client/) to take advantage of.\n\nThe benefit to how QEMU handles this is that the QEMU process simply loads the updated RBD driver on its next invocation and is now running an updated client. Its much more palatable to suspend or restart VMs to get the new driver then its to reboot all your hypervisors.\n\nAdditionally I\u0027m not sure that you can safely use the in-kernel RBD driver on servers that are also running OSDs. There have been issues in the past (I can\u0027t find a good link for this) around deadlocking when you try and do this. If this is still the case it would preclude this code being used by anyone who chooses to deploy a converged architecture (compute and OSD on the same physical server).","commit_id":"4333eebe0db61d741c491a13408a37ac558758f9"},{"author":{"_account_id":8582,"name":"Zoltan Arnold Nagy","email":"nag@zurich.ibm.com","username":"zoltan"},"change_message_id":"c77a53a3bb1d52b325869cae36368f28c7b8d7e6","unresolved":false,"context_lines":[{"line_number":99,"context_line":"implementation is able to talk to any recent ceph\u0027s rbd volumes, so there"},{"line_number":100,"context_line":"is no version dependence. It\u0027s true that usually the kernel driver lacks"},{"line_number":101,"context_line":"behind in features, but again, these shouldn\u0027t have any practical impact."},{"line_number":102,"context_line":""},{"line_number":103,"context_line":"Developer impact"},{"line_number":104,"context_line":"----------------"},{"line_number":105,"context_line":""}],"source_content_type":"text/x-rst","patch_set":3,"id":"da85f559_c323a9bb","line":102,"in_reply_to":"da85f559_bfefc71c","updated":"2015-11-12 14:54:35.000000000","message":"People will reboot their hypervisors anyway to patch their kernels with vendor supplied fixes, for which they will live migrate away the machines.\n\nYou are absolutely correct about the hyperconverged case and that\u0027s something needs to be documented. I\u0027ll add this as a deployer impact plus a documentation impact.\n\nI think the qemu benefits are very clear to everybody at this point but they won\u0027t happen for a while. As Matt suggested we could add this extension in the next release and deprecate it together once the qemu side (and the openstack implementation to support it...) is there.","commit_id":"4333eebe0db61d741c491a13408a37ac558758f9"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"f5a36b0e5d4fda58817850a8baebdf5ff815c644","unresolved":false,"context_lines":[{"line_number":43,"context_line":"volume is encrypted. If it is, then it configures the libvirt configuration"},{"line_number":44,"context_line":"to pass through a block device instead of the usual ceph path."},{"line_number":45,"context_line":"We also need to implement connect_volume() and disconnect_volume() in the rbd"},{"line_number":46,"context_line":"handling code (which ha been no-ops so far) and for encrypted volumes these"},{"line_number":47,"context_line":"will handle krdb mapping and unmapping the volumes."},{"line_number":48,"context_line":""},{"line_number":49,"context_line":"Nothing changes for the non-encrypted case."}],"source_content_type":"text/x-rst","patch_set":4,"id":"9a8ffd7b_ddbb45c3","line":46,"updated":"2015-11-25 02:56:10.000000000","message":"s/which ha/which has/","commit_id":"c31c7b69a459a5d0be2f9c78408e6cc85e79d43f"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"f5a36b0e5d4fda58817850a8baebdf5ff815c644","unresolved":false,"context_lines":[{"line_number":59,"context_line":"only, not an implemented one that we could already build OpenStack support for,"},{"line_number":60,"context_line":"and would mean that OpenStack won\u0027t have RBD encryption available during most"},{"line_number":61,"context_line":"of 2016."},{"line_number":62,"context_line":"Once nova has native encryption support in order to allow operators"},{"line_number":63,"context_line":"an easy migration path we presume that it will support the way the current"},{"line_number":64,"context_line":"encryptors use LUKS so after a VM restart the exact same volume will be"},{"line_number":65,"context_line":"available."}],"source_content_type":"text/x-rst","patch_set":4,"id":"9a8ffd7b_5da19593","line":62,"updated":"2015-11-25 02:56:10.000000000","message":"s/nova/qemu/","commit_id":"c31c7b69a459a5d0be2f9c78408e6cc85e79d43f"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"f5a36b0e5d4fda58817850a8baebdf5ff815c644","unresolved":false,"context_lines":[{"line_number":108,"context_line":"For this functionality the hypervisor\u0027s kernel needs to support krdb, the"},{"line_number":109,"context_line":"in-kernel rbd driver. My understanding is that any recent kernel\u0027s rbd"},{"line_number":110,"context_line":"implementation is able to talk to any recent ceph\u0027s rbd volumes, so there"},{"line_number":111,"context_line":"is no version dependence  It\u0027s true that usually the kernel driver lacks"},{"line_number":112,"context_line":"behind in features, but again, these shouldn\u0027t have any practical impact."},{"line_number":113,"context_line":""},{"line_number":114,"context_line":"Developer impact"}],"source_content_type":"text/x-rst","patch_set":4,"id":"9a8ffd7b_fdc40941","line":111,"updated":"2015-11-25 02:56:10.000000000","message":"s/dependence/dependence./","commit_id":"c31c7b69a459a5d0be2f9c78408e6cc85e79d43f"}]}