)]}'
{"specs/newton/approved/use-libvirt-storage-pools.rst":[{"author":{"_account_id":9555,"name":"Matthew Booth","email":"mbooth@redhat.com","username":"MatthewBooth"},"change_message_id":"35c2eef6dbb330fac88f9c8aaad063230aa7e448","unresolved":false,"context_lines":[{"line_number":52,"context_line":"instance directory to a new directory called _disk in the directory"},{"line_number":53,"context_line":"referenced by the instances_path configuration item. A libvirt storage"},{"line_number":54,"context_line":"pool  (:code:`nova-disks-pool`) will be configured to reference"},{"line_number":55,"context_line":"this new _disk directory."},{"line_number":56,"context_line":""},{"line_number":57,"context_line":"A symbolic link will be created in the instance directory to the new"},{"line_number":58,"context_line":"location of this file. These links will be removed the first time the"}],"source_content_type":"text/x-rst","patch_set":3,"id":"dab17558_d10f6e86","line":55,"updated":"2016-05-09 11:06:53.000000000","message":"Bear in mind that we don\u0027t want to put all file-backed disks in the same storage pool. For example, the \u0027Qcow2\u0027 and \u0027NoBacking\u0027 storage pools both use file storage, but have different semantics. I\u0027d suggest that we\u0027d want to store them separately, which might mean we need more than 1 default storage pool, although only 1 will be used on any particular compute host.\n\nIn the same way as directory locations are currently common across a deployment, we should make storage pool names common across a deployment. So the default Qcow2 storage pool name should be the same on all compute hosts which have one, and different to the default NoBacking storage pool name.","commit_id":"ba7ad3dfd3fc28d1d586a22a16f63a98677aa3fe"},{"author":{"_account_id":9555,"name":"Matthew Booth","email":"mbooth@redhat.com","username":"MatthewBooth"},"change_message_id":"54833b44fe87b82691db2119fa498fc28d93a558","unresolved":false,"context_lines":[{"line_number":52,"context_line":"instance directory to a new directory called _disk in the directory"},{"line_number":53,"context_line":"referenced by the instances_path configuration item. A libvirt storage"},{"line_number":54,"context_line":"pool  (:code:`nova-disks-pool`) will be configured to reference"},{"line_number":55,"context_line":"this new _disk directory."},{"line_number":56,"context_line":""},{"line_number":57,"context_line":"A symbolic link will be created in the instance directory to the new"},{"line_number":58,"context_line":"location of this file. These links will be removed the first time the"}],"source_content_type":"text/x-rst","patch_set":3,"id":"dab17558_a507fb7f","line":55,"in_reply_to":"dab17558_946dcf9a","updated":"2016-05-09 13:30:32.000000000","message":"We want to ensure that all (libvirt) volumes in a particular storage pool are of the same type, which means not storing qcow2 and \u0027raw\u0027 in the same pool, for eg, as these have different semantics. I\u0027m saying that it\u0027s insufficient to have a single storage pool, as the default configuration will produce both qcow2 (for regular disks) and raw (for config disks) and these should be in different storage pools.","commit_id":"ba7ad3dfd3fc28d1d586a22a16f63a98677aa3fe"},{"author":{"_account_id":6608,"name":"Paul Carlton","email":"paul.carlton2@hpe.com","username":"paul-carlton2"},"change_message_id":"41fe43fc18fb4680ada1b5bebba7429f01ce3367","unresolved":false,"context_lines":[{"line_number":52,"context_line":"instance directory to a new directory called _disk in the directory"},{"line_number":53,"context_line":"referenced by the instances_path configuration item. A libvirt storage"},{"line_number":54,"context_line":"pool  (:code:`nova-disks-pool`) will be configured to reference"},{"line_number":55,"context_line":"this new _disk directory."},{"line_number":56,"context_line":""},{"line_number":57,"context_line":"A symbolic link will be created in the instance directory to the new"},{"line_number":58,"context_line":"location of this file. These links will be removed the first time the"}],"source_content_type":"text/x-rst","patch_set":3,"id":"dab17558_946dcf9a","line":55,"in_reply_to":"dab17558_d10f6e86","updated":"2016-05-09 12:02:45.000000000","message":"I see a storage pool as representing a an entity like a disk directory or lvm volume group that contains files which are represented as storage pool volumes. Some backend implementations will place files in one or more pools, e.g. LVM would place data disk files on in the lvm pool and config drive in the disk pool.","commit_id":"ba7ad3dfd3fc28d1d586a22a16f63a98677aa3fe"},{"author":{"_account_id":9555,"name":"Matthew Booth","email":"mbooth@redhat.com","username":"MatthewBooth"},"change_message_id":"35c2eef6dbb330fac88f9c8aaad063230aa7e448","unresolved":false,"context_lines":[{"line_number":58,"context_line":"location of this file. These links will be removed the first time the"},{"line_number":59,"context_line":"instance domain is destroyed and recreated (as in nova operations like"},{"line_number":60,"context_line":"hard reboot, rebuild, stop/start). From that point onwards the libvirt"},{"line_number":61,"context_line":"storage pools and volumes will be used to manage the instance."},{"line_number":62,"context_line":""},{"line_number":63,"context_line":"The existing [1_] Image class and its sub classes (in imagebackend.py)"},{"line_number":64,"context_line":"will be left unchanged. A new base class called Disk and sub classes"}],"source_content_type":"text/x-rst","patch_set":3,"id":"dab17558_b1958ae5","line":61,"updated":"2016-05-09 11:06:53.000000000","message":"I remain unconvinced by this. I think you may find that maintaining these links, even temporarily, is more complicated in code and introduces more bugs than writing a clean upgrade and having separate code paths. The persistent storage metadata can keep track of which instances have upgraded.\n\nI\u0027d defer to actually seeing the code, though. If you can make it uncomplicated then I guess I\u0027d be fine with it.","commit_id":"ba7ad3dfd3fc28d1d586a22a16f63a98677aa3fe"},{"author":{"_account_id":6608,"name":"Paul Carlton","email":"paul.carlton2@hpe.com","username":"paul-carlton2"},"change_message_id":"41fe43fc18fb4680ada1b5bebba7429f01ce3367","unresolved":false,"context_lines":[{"line_number":58,"context_line":"location of this file. These links will be removed the first time the"},{"line_number":59,"context_line":"instance domain is destroyed and recreated (as in nova operations like"},{"line_number":60,"context_line":"hard reboot, rebuild, stop/start). From that point onwards the libvirt"},{"line_number":61,"context_line":"storage pools and volumes will be used to manage the instance."},{"line_number":62,"context_line":""},{"line_number":63,"context_line":"The existing [1_] Image class and its sub classes (in imagebackend.py)"},{"line_number":64,"context_line":"will be left unchanged. A new base class called Disk and sub classes"}],"source_content_type":"text/x-rst","patch_set":3,"id":"dab17558_794ae4ad","line":61,"in_reply_to":"dab17558_b1958ae5","updated":"2016-05-09 12:02:45.000000000","message":"Done","commit_id":"ba7ad3dfd3fc28d1d586a22a16f63a98677aa3fe"},{"author":{"_account_id":9555,"name":"Matthew Booth","email":"mbooth@redhat.com","username":"MatthewBooth"},"change_message_id":"35c2eef6dbb330fac88f9c8aaad063230aa7e448","unresolved":false,"context_lines":[{"line_number":66,"context_line":"pools and volumes.  The Image class and sub classes for each backend"},{"line_number":67,"context_line":"will remain to support cases where the use_storage_pools configuration"},{"line_number":68,"context_line":"flag is not set to true. This will be removed in the P release as"},{"line_number":69,"context_line":"described in `Other deployer impact`_ section below."},{"line_number":70,"context_line":""},{"line_number":71,"context_line":"The cache of images downloaded from Glance would be placed into a"},{"line_number":72,"context_line":"storage pool (:code:`nova-base-images-pool`). This is done simply by"}],"source_content_type":"text/x-rst","patch_set":3,"id":"dab17558_b1310a0e","line":69,"updated":"2016-05-09 11:06:53.000000000","message":"In order to do this we will have to be able to guarantee that all instances have been upgraded. I think this requires some more definition. Can we do it automatically, or do we give operators a tool to ensure it has happened before upgrading to P?","commit_id":"ba7ad3dfd3fc28d1d586a22a16f63a98677aa3fe"},{"author":{"_account_id":6608,"name":"Paul Carlton","email":"paul.carlton2@hpe.com","username":"paul-carlton2"},"change_message_id":"41fe43fc18fb4680ada1b5bebba7429f01ce3367","unresolved":false,"context_lines":[{"line_number":66,"context_line":"pools and volumes.  The Image class and sub classes for each backend"},{"line_number":67,"context_line":"will remain to support cases where the use_storage_pools configuration"},{"line_number":68,"context_line":"flag is not set to true. This will be removed in the P release as"},{"line_number":69,"context_line":"described in `Other deployer impact`_ section below."},{"line_number":70,"context_line":""},{"line_number":71,"context_line":"The cache of images downloaded from Glance would be placed into a"},{"line_number":72,"context_line":"storage pool (:code:`nova-base-images-pool`). This is done simply by"}],"source_content_type":"text/x-rst","patch_set":3,"id":"dab17558_195bd888","line":69,"in_reply_to":"dab17558_b1310a0e","updated":"2016-05-09 12:02:45.000000000","message":"Done","commit_id":"ba7ad3dfd3fc28d1d586a22a16f63a98677aa3fe"},{"author":{"_account_id":9555,"name":"Matthew Booth","email":"mbooth@redhat.com","username":"MatthewBooth"},"change_message_id":"35c2eef6dbb330fac88f9c8aaad063230aa7e448","unresolved":false,"context_lines":[{"line_number":123,"context_line":"with other backends like RBD and LVM. This approach was rejected"},{"line_number":124,"context_line":"because it makes migration more difficult and only provides any"},{"line_number":125,"context_line":"benefit if we use it to replace the existing backend implementations"},{"line_number":126,"context_line":"with a single storage pool backend."},{"line_number":127,"context_line":""},{"line_number":128,"context_line":"Data model impact"},{"line_number":129,"context_line":"-----------------"}],"source_content_type":"text/x-rst","patch_set":3,"id":"dab17558_838c7780","line":126,"updated":"2016-05-09 11:06:53.000000000","message":"Unless I\u0027ve misunderstood, what you\u0027re describe above is actually changing the layout. Did you forget to update this?","commit_id":"ba7ad3dfd3fc28d1d586a22a16f63a98677aa3fe"},{"author":{"_account_id":6608,"name":"Paul Carlton","email":"paul.carlton2@hpe.com","username":"paul-carlton2"},"change_message_id":"41fe43fc18fb4680ada1b5bebba7429f01ce3367","unresolved":false,"context_lines":[{"line_number":123,"context_line":"with other backends like RBD and LVM. This approach was rejected"},{"line_number":124,"context_line":"because it makes migration more difficult and only provides any"},{"line_number":125,"context_line":"benefit if we use it to replace the existing backend implementations"},{"line_number":126,"context_line":"with a single storage pool backend."},{"line_number":127,"context_line":""},{"line_number":128,"context_line":"Data model impact"},{"line_number":129,"context_line":"-----------------"}],"source_content_type":"text/x-rst","patch_set":3,"id":"dab17558_b97c4cec","line":126,"in_reply_to":"dab17558_838c7780","updated":"2016-05-09 12:02:45.000000000","message":"Done","commit_id":"ba7ad3dfd3fc28d1d586a22a16f63a98677aa3fe"},{"author":{"_account_id":9555,"name":"Matthew Booth","email":"mbooth@redhat.com","username":"MatthewBooth"},"change_message_id":"35c2eef6dbb330fac88f9c8aaad063230aa7e448","unresolved":false,"context_lines":[{"line_number":179,"context_line":"directory pool in the current image cache directory.  This will"},{"line_number":180,"context_line":"automatically generate storage volumes for all the backing images."},{"line_number":181,"context_line":""},{"line_number":182,"context_line":"Nova would also create the :code:`nova-disks-pool` storage pool."},{"line_number":183,"context_line":"This will automatically generate storage volumes for all the disk files"},{"line_number":184,"context_line":"used by instances."},{"line_number":185,"context_line":""}],"source_content_type":"text/x-rst","patch_set":3,"id":"dab17558_0313670e","line":182,"updated":"2016-05-09 11:06:53.000000000","message":"Bikeshed: nova-default-storage-pool? There may be a point where we want more than 1, but there will still only be 1 default.","commit_id":"ba7ad3dfd3fc28d1d586a22a16f63a98677aa3fe"},{"author":{"_account_id":9555,"name":"Matthew Booth","email":"mbooth@redhat.com","username":"MatthewBooth"},"change_message_id":"35c2eef6dbb330fac88f9c8aaad063230aa7e448","unresolved":false,"context_lines":[{"line_number":181,"context_line":""},{"line_number":182,"context_line":"Nova would also create the :code:`nova-disks-pool` storage pool."},{"line_number":183,"context_line":"This will automatically generate storage volumes for all the disk files"},{"line_number":184,"context_line":"used by instances."},{"line_number":185,"context_line":""},{"line_number":186,"context_line":"If a non directory based backend is used, like RBD or LVM then, in"},{"line_number":187,"context_line":"addition to the above Nova will create a storage pool of the appropriate"}],"source_content_type":"text/x-rst","patch_set":3,"id":"dab17558_e3d3c336","line":184,"updated":"2016-05-09 11:06:53.000000000","message":"See above, I think you want to separate these by type. If the host is configured to use qcow2 (the default), then config disks will still be in a separate, non-qcow2 pool.","commit_id":"ba7ad3dfd3fc28d1d586a22a16f63a98677aa3fe"},{"author":{"_account_id":12408,"name":"Maxim Nestratov","email":"mnestratov@virtuozzo.com","username":"mnestratov"},"change_message_id":"339ded171b8aacfa8360ec577d0a9b452ef21e59","unresolved":false,"context_lines":[{"line_number":245,"context_line":""},{"line_number":246,"context_line":"The minimum libvirt version for Newton will provide the basic support"},{"line_number":247,"context_line":"for storage pools and volumes. Support for ploop storage pools has only"},{"line_number":248,"context_line":"recently been added so deployments using ploop will not be able to turn"},{"line_number":249,"context_line":"on the libvirt storage pools configuration flag unless they are running"},{"line_number":250,"context_line":"that version."},{"line_number":251,"context_line":""}],"source_content_type":"text/x-rst","patch_set":3,"id":"dab17558_537466ce","line":248,"range":{"start_line":248,"start_character":47,"end_line":248,"end_character":51},"updated":"2016-05-06 09:29:59.000000000","message":"Are we going to require this particular version of libvirt for using ploop images in runtime?","commit_id":"ba7ad3dfd3fc28d1d586a22a16f63a98677aa3fe"},{"author":{"_account_id":6608,"name":"Paul Carlton","email":"paul.carlton2@hpe.com","username":"paul-carlton2"},"change_message_id":"c89eb5979fd3d98f066020a0f121cf701f415e96","unresolved":false,"context_lines":[{"line_number":245,"context_line":""},{"line_number":246,"context_line":"The minimum libvirt version for Newton will provide the basic support"},{"line_number":247,"context_line":"for storage pools and volumes. Support for ploop storage pools has only"},{"line_number":248,"context_line":"recently been added so deployments using ploop will not be able to turn"},{"line_number":249,"context_line":"on the libvirt storage pools configuration flag unless they are running"},{"line_number":250,"context_line":"that version."},{"line_number":251,"context_line":""}],"source_content_type":"text/x-rst","patch_set":3,"id":"dab17558_f92579ae","line":248,"range":{"start_line":248,"start_character":47,"end_line":248,"end_character":51},"in_reply_to":"dab17558_537466ce","updated":"2016-05-06 09:52:51.000000000","message":"Yes that was my thinking that the minimum version for ploop and storage pools would be set to x.y.z.  I\u0027m guessing you would know what \u0027x.y.z\u0027 is?","commit_id":"ba7ad3dfd3fc28d1d586a22a16f63a98677aa3fe"},{"author":{"_account_id":12408,"name":"Maxim Nestratov","email":"mnestratov@virtuozzo.com","username":"mnestratov"},"change_message_id":"40df62f54e96e18f80dfc5f15c1fc1c12e6fbb7b","unresolved":false,"context_lines":[{"line_number":245,"context_line":""},{"line_number":246,"context_line":"The minimum libvirt version for Newton will provide the basic support"},{"line_number":247,"context_line":"for storage pools and volumes. Support for ploop storage pools has only"},{"line_number":248,"context_line":"recently been added so deployments using ploop will not be able to turn"},{"line_number":249,"context_line":"on the libvirt storage pools configuration flag unless they are running"},{"line_number":250,"context_line":"that version."},{"line_number":251,"context_line":""}],"source_content_type":"text/x-rst","patch_set":3,"id":"dab17558_28820853","line":248,"range":{"start_line":248,"start_character":47,"end_line":248,"end_character":51},"in_reply_to":"dab17558_f92579ae","updated":"2016-05-06 10:34:50.000000000","message":"Sure. Ploop support was released in 1.3.4.","commit_id":"ba7ad3dfd3fc28d1d586a22a16f63a98677aa3fe"},{"author":{"_account_id":9555,"name":"Matthew Booth","email":"mbooth@redhat.com","username":"MatthewBooth"},"change_message_id":"25586da7845250c085c90050d280786a9a1ee218","unresolved":false,"context_lines":[{"line_number":65,"context_line":"for each backend type will be created to implement the use of libvirt"},{"line_number":66,"context_line":"pools and volumes.  The Image class and sub classes for each backend"},{"line_number":67,"context_line":"will remain to support cases where the use_storage_pools configuration"},{"line_number":68,"context_line":"flag is not set to true. This will be removed in the P release as"},{"line_number":69,"context_line":"described in `Other deployer impact`_ section below. When the code is"},{"line_number":70,"context_line":"removed and the used of storage pools becomes default any instances"},{"line_number":71,"context_line":"which have not been recreated using storage pools code will still"}],"source_content_type":"text/x-rst","patch_set":4,"id":"dab17558_6516b3f2","line":68,"updated":"2016-05-09 13:32:16.000000000","message":"Thinking this through, I think use_storage_pools should always be true. In other words, it should just go away.\n\nWe can\u0027t do libvirt storage pools for the ploop backend yet, anyway, so this migration is going to be incremental as the capability arises.","commit_id":"0039abd37d94e707d4491101700413946c7174e7"},{"author":{"_account_id":9555,"name":"Matthew Booth","email":"mbooth@redhat.com","username":"MatthewBooth"},"change_message_id":"25586da7845250c085c90050d280786a9a1ee218","unresolved":false,"context_lines":[{"line_number":75,"context_line":"scenario where an instance\u0027s disk file is not in the nova-disk-pool"},{"line_number":76,"context_line":"will need to remain until we are sure that no instances with disk"},{"line_number":77,"context_line":"files in their instance directory remain, i.e. all instances have"},{"line_number":78,"context_line":"been destroyed and recreated since implementation of storage pools."},{"line_number":79,"context_line":""},{"line_number":80,"context_line":"The new classes inherrting from \u0027Disk\u0027 backend will manage the backend"},{"line_number":81,"context_line":"files as appropriate for that backend type unsing the appropriate"}],"source_content_type":"text/x-rst","patch_set":4,"id":"dab17558_e5cd83b8","line":78,"updated":"2016-05-09 13:32:16.000000000","message":"This explicitly punts the deprecation problem into the future. I\u0027m cool with that, but I\u0027d like to see it gone as soon as possible.","commit_id":"0039abd37d94e707d4491101700413946c7174e7"},{"author":{"_account_id":9555,"name":"Matthew Booth","email":"mbooth@redhat.com","username":"MatthewBooth"},"change_message_id":"25586da7845250c085c90050d280786a9a1ee218","unresolved":false,"context_lines":[{"line_number":77,"context_line":"files in their instance directory remain, i.e. all instances have"},{"line_number":78,"context_line":"been destroyed and recreated since implementation of storage pools."},{"line_number":79,"context_line":""},{"line_number":80,"context_line":"The new classes inherrting from \u0027Disk\u0027 backend will manage the backend"},{"line_number":81,"context_line":"files as appropriate for that backend type unsing the appropriate"},{"line_number":82,"context_line":"storage pool for each file."},{"line_number":83,"context_line":""}],"source_content_type":"text/x-rst","patch_set":4,"id":"dab17558_85f2b7d0","line":80,"updated":"2016-05-09 13:32:16.000000000","message":"nit: inheriting","commit_id":"0039abd37d94e707d4491101700413946c7174e7"},{"author":{"_account_id":6608,"name":"Paul Carlton","email":"paul.carlton2@hpe.com","username":"paul-carlton2"},"change_message_id":"e117709c32a07afc5fceca0cc8f3fad00a6f33df","unresolved":false,"context_lines":[{"line_number":264,"context_line":"recently been added so deployments using ploop will not be able to turn"},{"line_number":265,"context_line":"on the libvirt storage pools configuration flag unless they are running"},{"line_number":266,"context_line":"that version."},{"line_number":267,"context_line":""},{"line_number":268,"context_line":"This spec is dependant on spec \u0027Persist libvirt instance storage"},{"line_number":269,"context_line":"metadata\u0027, https://review.openstack.org/#/c/302117"},{"line_number":270,"context_line":""}],"source_content_type":"text/x-rst","patch_set":4,"id":"dab17558_34f4e385","line":267,"updated":"2016-05-09 12:11:14.000000000","message":"The minimum libvirt version for support of storage pools with ploop is 1.3.4. Thus, we will check for this version or higher when ploop backend is used with storage pools flag set to on and terminate with an error indicating that storage pools cannot be used with ploop until libvirt version 1.3.4 if the libvirt version is less than this. This will mean that if we want to remove the old code in P, the minimum libvirt version for ploop users will be 1.3.4. in P. We should add a release note in Newton and Ocata to highlight this.  I\u0027ll add this to the next revision of this spec if no one objects.","commit_id":"0039abd37d94e707d4491101700413946c7174e7"},{"author":{"_account_id":7461,"name":"Paul Murray","email":"pmurray@hpe.com","username":"ptm"},"change_message_id":"13eb78bfe99919a6faf9385041e29da937808931","unresolved":false,"context_lines":[{"line_number":33,"context_line":""},{"line_number":34,"context_line":"Use Cases"},{"line_number":35,"context_line":"---------"},{"line_number":36,"context_line":""},{"line_number":37,"context_line":"Developer: This will facilitate removing SSH from resize/migrate, as it"},{"line_number":38,"context_line":"will allow us to use virStorageVolUpload and virStorageVolDownload to"},{"line_number":39,"context_line":"migrate storage."}],"source_content_type":"text/x-rst","patch_set":7,"id":"dab17558_067e8c69","line":36,"updated":"2016-05-16 14:26:38.000000000","message":"I think the primary use case is:\n\n\"As a cloud operator I consider passwordless SSH access between my compute nodes an unacceptable security risk and would prefer a more controlled API/protocol to transfer instance data instead.\"\n\nThere are operators that explicitly exclude resize/migrate for this reason.","commit_id":"c65903934ff7ac7c4bd6b079c3e439f57d7612a9"},{"author":{"_account_id":18247,"name":"Mark Doffman","email":"mdoffman@linux.vnet.ibm.com","username":"doffm"},"change_message_id":"2e00758e542c9aa147705219cae3f6cb4e5ced1d","unresolved":false,"context_lines":[{"line_number":36,"context_line":""},{"line_number":37,"context_line":"Developer: This will facilitate removing SSH from resize/migrate, as it"},{"line_number":38,"context_line":"will allow us to use virStorageVolUpload and virStorageVolDownload to"},{"line_number":39,"context_line":"migrate storage."},{"line_number":40,"context_line":""},{"line_number":41,"context_line":"Developer: Second, this will simplify adding in support for pool types"},{"line_number":42,"context_line":"that are supported by libvirt but not supported by Nova (such as"}],"source_content_type":"text/x-rst","patch_set":7,"id":"dab17558_4ec58b3a","line":39,"updated":"2016-05-13 19:06:26.000000000","message":"You will have to excuse my ignorance here, but how will this improve the security over using SSH? Will it improve security? Will it improve performance? Could there be any reason someone might want to continue using SSH or some other method of transfer for the images?\n\nWhat do virStorageVolUpload and virStorageVolDownload do behind the scenes? How do the computes authenticate each other? What user does the receiving process run as?","commit_id":"c65903934ff7ac7c4bd6b079c3e439f57d7612a9"},{"author":{"_account_id":6608,"name":"Paul Carlton","email":"paul.carlton2@hpe.com","username":"paul-carlton2"},"change_message_id":"dea980488fa61d2d7d16986d85468326453e8b19","unresolved":false,"context_lines":[{"line_number":36,"context_line":""},{"line_number":37,"context_line":"Developer: This will facilitate removing SSH from resize/migrate, as it"},{"line_number":38,"context_line":"will allow us to use virStorageVolUpload and virStorageVolDownload to"},{"line_number":39,"context_line":"migrate storage."},{"line_number":40,"context_line":""},{"line_number":41,"context_line":"Developer: Second, this will simplify adding in support for pool types"},{"line_number":42,"context_line":"that are supported by libvirt but not supported by Nova (such as"}],"source_content_type":"text/x-rst","patch_set":7,"id":"dab17558_a0805fc0","line":39,"in_reply_to":"dab17558_4ec58b3a","updated":"2016-05-16 07:22:51.000000000","message":"There is a widely held view that the ssh based solution is less secure because it requires the storage of ssh keys on the compute nodes and enabling of passwordless ssh.  The libvirt volume based migration can be configured to use TLS between libvirt daemons. Performance is not expected.  The goal is to implement a libvirt based solution for cold migration that performs as well as the ssh based solution before removing that option.\n\nThe way it works is that you create a virStream between to libvirt daemons and then use upload on one end and download on the other end to transfer data","commit_id":"c65903934ff7ac7c4bd6b079c3e439f57d7612a9"},{"author":{"_account_id":7461,"name":"Paul Murray","email":"pmurray@hpe.com","username":"ptm"},"change_message_id":"13eb78bfe99919a6faf9385041e29da937808931","unresolved":false,"context_lines":[{"line_number":36,"context_line":""},{"line_number":37,"context_line":"Developer: This will facilitate removing SSH from resize/migrate, as it"},{"line_number":38,"context_line":"will allow us to use virStorageVolUpload and virStorageVolDownload to"},{"line_number":39,"context_line":"migrate storage."},{"line_number":40,"context_line":""},{"line_number":41,"context_line":"Developer: Second, this will simplify adding in support for pool types"},{"line_number":42,"context_line":"that are supported by libvirt but not supported by Nova (such as"}],"source_content_type":"text/x-rst","patch_set":7,"id":"dab17558_e3953ae4","line":39,"in_reply_to":"dab17558_a0805fc0","updated":"2016-05-16 14:26:38.000000000","message":"The SSH approach implicitly requires remote access to the hosts file system. The source host creates directories on the destination host using SSH and copies files to them. The destination host doesn\u0027t know where the files will go or what the files are.","commit_id":"c65903934ff7ac7c4bd6b079c3e439f57d7612a9"},{"author":{"_account_id":4393,"name":"Dan Smith","email":"dms@danplanet.com","username":"danms"},"change_message_id":"ef9ece8984f2c5af25ac5a02f5e8abb58d28d324","unresolved":false,"context_lines":[{"line_number":36,"context_line":""},{"line_number":37,"context_line":"Operator: As a cloud operator I consider passwordless SSH access"},{"line_number":38,"context_line":"between my compute nodes an unacceptable security risk and would prefer"},{"line_number":39,"context_line":"a more controlled API/protocol to transfer instance data instead. Some"},{"line_number":40,"context_line":"cloud operators explicitly exclude resize and cold migrate for this"},{"line_number":41,"context_line":"reason."},{"line_number":42,"context_line":""},{"line_number":43,"context_line":"Developer: This will facilitate removing SSH from resize/migrate, as it"},{"line_number":44,"context_line":"will allow us to use virStorageVolUpload and virStorageVolDownload to"}],"source_content_type":"text/x-rst","patch_set":8,"id":"bab6814e_74e452df","line":41,"range":{"start_line":39,"start_character":66,"end_line":41,"end_character":7},"updated":"2016-05-26 16:53:21.000000000","message":"This is part of the problem not part of a user story, IMHO.","commit_id":"51015fb0980091c985276ccd8149871003ac5ac8"},{"author":{"_account_id":6608,"name":"Paul Carlton","email":"paul.carlton2@hpe.com","username":"paul-carlton2"},"change_message_id":"9ba166c1321321655639743382a5a6bf8e1edfc4","unresolved":false,"context_lines":[{"line_number":36,"context_line":""},{"line_number":37,"context_line":"Operator: As a cloud operator I consider passwordless SSH access"},{"line_number":38,"context_line":"between my compute nodes an unacceptable security risk and would prefer"},{"line_number":39,"context_line":"a more controlled API/protocol to transfer instance data instead. Some"},{"line_number":40,"context_line":"cloud operators explicitly exclude resize and cold migrate for this"},{"line_number":41,"context_line":"reason."},{"line_number":42,"context_line":""},{"line_number":43,"context_line":"Developer: This will facilitate removing SSH from resize/migrate, as it"},{"line_number":44,"context_line":"will allow us to use virStorageVolUpload and virStorageVolDownload to"}],"source_content_type":"text/x-rst","patch_set":8,"id":"9abb7d3a_09d11dce","line":41,"range":{"start_line":39,"start_character":66,"end_line":41,"end_character":7},"in_reply_to":"bab6814e_74e452df","updated":"2016-05-27 08:44:10.000000000","message":"Done","commit_id":"51015fb0980091c985276ccd8149871003ac5ac8"},{"author":{"_account_id":4393,"name":"Dan Smith","email":"dms@danplanet.com","username":"danms"},"change_message_id":"ef9ece8984f2c5af25ac5a02f5e8abb58d28d324","unresolved":false,"context_lines":[{"line_number":40,"context_line":"cloud operators explicitly exclude resize and cold migrate for this"},{"line_number":41,"context_line":"reason."},{"line_number":42,"context_line":""},{"line_number":43,"context_line":"Developer: This will facilitate removing SSH from resize/migrate, as it"},{"line_number":44,"context_line":"will allow us to use virStorageVolUpload and virStorageVolDownload to"},{"line_number":45,"context_line":"migrate storage."},{"line_number":46,"context_line":""},{"line_number":47,"context_line":"Developer: Second, this will simplify adding in support for pool types"},{"line_number":48,"context_line":"that are supported by libvirt but not supported by Nova (such as"}],"source_content_type":"text/x-rst","patch_set":8,"id":"bab6814e_d4ec1ebd","line":45,"range":{"start_line":43,"start_character":11,"end_line":45,"end_character":16},"updated":"2016-05-26 16:53:21.000000000","message":"This does\u0027t seem like a user story, nor is it really necessary or possible to form it into one, IMHO. I would just remove it.","commit_id":"51015fb0980091c985276ccd8149871003ac5ac8"},{"author":{"_account_id":6608,"name":"Paul Carlton","email":"paul.carlton2@hpe.com","username":"paul-carlton2"},"change_message_id":"9ba166c1321321655639743382a5a6bf8e1edfc4","unresolved":false,"context_lines":[{"line_number":40,"context_line":"cloud operators explicitly exclude resize and cold migrate for this"},{"line_number":41,"context_line":"reason."},{"line_number":42,"context_line":""},{"line_number":43,"context_line":"Developer: This will facilitate removing SSH from resize/migrate, as it"},{"line_number":44,"context_line":"will allow us to use virStorageVolUpload and virStorageVolDownload to"},{"line_number":45,"context_line":"migrate storage."},{"line_number":46,"context_line":""},{"line_number":47,"context_line":"Developer: Second, this will simplify adding in support for pool types"},{"line_number":48,"context_line":"that are supported by libvirt but not supported by Nova (such as"}],"source_content_type":"text/x-rst","patch_set":8,"id":"9abb7d3a_691d49bd","line":45,"range":{"start_line":43,"start_character":11,"end_line":45,"end_character":16},"in_reply_to":"bab6814e_d4ec1ebd","updated":"2016-05-27 08:44:10.000000000","message":"Done","commit_id":"51015fb0980091c985276ccd8149871003ac5ac8"},{"author":{"_account_id":4393,"name":"Dan Smith","email":"dms@danplanet.com","username":"danms"},"change_message_id":"ef9ece8984f2c5af25ac5a02f5e8abb58d28d324","unresolved":false,"context_lines":[{"line_number":44,"context_line":"will allow us to use virStorageVolUpload and virStorageVolDownload to"},{"line_number":45,"context_line":"migrate storage."},{"line_number":46,"context_line":""},{"line_number":47,"context_line":"Developer: Second, this will simplify adding in support for pool types"},{"line_number":48,"context_line":"that are supported by libvirt but not supported by Nova (such as"},{"line_number":49,"context_line":"Sheepdog)."},{"line_number":50,"context_line":""},{"line_number":51,"context_line":"Developer: this will (in the long run) simplify the imagebackend code,"},{"line_number":52,"context_line":"making it easier to maintain."}],"source_content_type":"text/x-rst","patch_set":8,"id":"bab6814e_540e6e8c","line":49,"range":{"start_line":47,"start_character":11,"end_line":49,"end_character":10},"updated":"2016-05-26 16:53:21.000000000","message":"This is fine, but should be written like a user story.\n\n\"As a developer I want to be able to add support for new backends that libvirt provides without a lot of special-casing code in Nova\"","commit_id":"51015fb0980091c985276ccd8149871003ac5ac8"},{"author":{"_account_id":6608,"name":"Paul Carlton","email":"paul.carlton2@hpe.com","username":"paul-carlton2"},"change_message_id":"9ba166c1321321655639743382a5a6bf8e1edfc4","unresolved":false,"context_lines":[{"line_number":44,"context_line":"will allow us to use virStorageVolUpload and virStorageVolDownload to"},{"line_number":45,"context_line":"migrate storage."},{"line_number":46,"context_line":""},{"line_number":47,"context_line":"Developer: Second, this will simplify adding in support for pool types"},{"line_number":48,"context_line":"that are supported by libvirt but not supported by Nova (such as"},{"line_number":49,"context_line":"Sheepdog)."},{"line_number":50,"context_line":""},{"line_number":51,"context_line":"Developer: this will (in the long run) simplify the imagebackend code,"},{"line_number":52,"context_line":"making it easier to maintain."}],"source_content_type":"text/x-rst","patch_set":8,"id":"9abb7d3a_89206df9","line":49,"range":{"start_line":47,"start_character":11,"end_line":49,"end_character":10},"in_reply_to":"bab6814e_540e6e8c","updated":"2016-05-27 08:44:10.000000000","message":"Done","commit_id":"51015fb0980091c985276ccd8149871003ac5ac8"},{"author":{"_account_id":4393,"name":"Dan Smith","email":"dms@danplanet.com","username":"danms"},"change_message_id":"ef9ece8984f2c5af25ac5a02f5e8abb58d28d324","unresolved":false,"context_lines":[{"line_number":48,"context_line":"that are supported by libvirt but not supported by Nova (such as"},{"line_number":49,"context_line":"Sheepdog)."},{"line_number":50,"context_line":""},{"line_number":51,"context_line":"Developer: this will (in the long run) simplify the imagebackend code,"},{"line_number":52,"context_line":"making it easier to maintain."},{"line_number":53,"context_line":""},{"line_number":54,"context_line":"Proposed change"},{"line_number":55,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"}],"source_content_type":"text/x-rst","patch_set":8,"id":"bab6814e_34612a3a","line":52,"range":{"start_line":51,"start_character":11,"end_line":52,"end_character":29},"updated":"2016-05-26 16:53:21.000000000","message":"Also not written as a user story. This is a benefit, but I\u0027m not sure \"As a developer I want life to suck less\" is really important to document here :)","commit_id":"51015fb0980091c985276ccd8149871003ac5ac8"},{"author":{"_account_id":6608,"name":"Paul Carlton","email":"paul.carlton2@hpe.com","username":"paul-carlton2"},"change_message_id":"9ba166c1321321655639743382a5a6bf8e1edfc4","unresolved":false,"context_lines":[{"line_number":48,"context_line":"that are supported by libvirt but not supported by Nova (such as"},{"line_number":49,"context_line":"Sheepdog)."},{"line_number":50,"context_line":""},{"line_number":51,"context_line":"Developer: this will (in the long run) simplify the imagebackend code,"},{"line_number":52,"context_line":"making it easier to maintain."},{"line_number":53,"context_line":""},{"line_number":54,"context_line":"Proposed change"},{"line_number":55,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"}],"source_content_type":"text/x-rst","patch_set":8,"id":"9abb7d3a_29234107","line":52,"range":{"start_line":51,"start_character":11,"end_line":52,"end_character":29},"in_reply_to":"bab6814e_34612a3a","updated":"2016-05-27 08:44:10.000000000","message":"Done","commit_id":"51015fb0980091c985276ccd8149871003ac5ac8"},{"author":{"_account_id":4393,"name":"Dan Smith","email":"dms@danplanet.com","username":"danms"},"change_message_id":"ef9ece8984f2c5af25ac5a02f5e8abb58d28d324","unresolved":false,"context_lines":[{"line_number":57,"context_line":"that implements libvirt storage pools it will check to see if the"},{"line_number":58,"context_line":"storage pools :code:`nova-image-cache-pool` and :code:`nova-disk-pool`"},{"line_number":59,"context_line":"already existed. If not, the :code:`nova-image-cache-pool` storage pool"},{"line_number":60,"context_line":"will be created as a directory pool referencing current image cache"},{"line_number":61,"context_line":"directory, i.e. the sub directory named by the configuration flag"},{"line_number":62,"context_line":":code:`image_cache_subdirectory_name` in the instances directory as"},{"line_number":63,"context_line":"defined by the configuration flag  :code:`compute.instances_path`. This"}],"source_content_type":"text/x-rst","patch_set":8,"id":"bab6814e_f49042ef","line":60,"range":{"start_line":60,"start_character":0,"end_line":60,"end_character":15},"updated":"2016-05-26 16:53:21.000000000","message":"Does the nova user typically have permissions to do this in libvirt? If we can, making it just work is goodness... just need to fail gracefully if not.","commit_id":"51015fb0980091c985276ccd8149871003ac5ac8"},{"author":{"_account_id":6608,"name":"Paul Carlton","email":"paul.carlton2@hpe.com","username":"paul-carlton2"},"change_message_id":"9ba166c1321321655639743382a5a6bf8e1edfc4","unresolved":false,"context_lines":[{"line_number":57,"context_line":"that implements libvirt storage pools it will check to see if the"},{"line_number":58,"context_line":"storage pools :code:`nova-image-cache-pool` and :code:`nova-disk-pool`"},{"line_number":59,"context_line":"already existed. If not, the :code:`nova-image-cache-pool` storage pool"},{"line_number":60,"context_line":"will be created as a directory pool referencing current image cache"},{"line_number":61,"context_line":"directory, i.e. the sub directory named by the configuration flag"},{"line_number":62,"context_line":":code:`image_cache_subdirectory_name` in the instances directory as"},{"line_number":63,"context_line":"defined by the configuration flag  :code:`compute.instances_path`. This"}],"source_content_type":"text/x-rst","patch_set":8,"id":"9abb7d3a_be176553","line":60,"range":{"start_line":60,"start_character":0,"end_line":60,"end_character":15},"in_reply_to":"bab6814e_f49042ef","updated":"2016-05-27 08:44:10.000000000","message":"Seems so my implementation seems to work, see https://review.openstack.org/#/c/314134/ which is a WIP at present but creation of pools and volumes works fine.","commit_id":"51015fb0980091c985276ccd8149871003ac5ac8"},{"author":{"_account_id":4393,"name":"Dan Smith","email":"dms@danplanet.com","username":"danms"},"change_message_id":"ef9ece8984f2c5af25ac5a02f5e8abb58d28d324","unresolved":false,"context_lines":[{"line_number":62,"context_line":":code:`image_cache_subdirectory_name` in the instances directory as"},{"line_number":63,"context_line":"defined by the configuration flag  :code:`compute.instances_path`. This"},{"line_number":64,"context_line":"will automatically generate storage volumes for all the backing images."},{"line_number":65,"context_line":"The pool :code:`nova-disk-pool` will also be create referencing the sub"},{"line_number":66,"context_line":"directory :code:`_disk` in the instances directory."},{"line_number":67,"context_line":""},{"line_number":68,"context_line":"If a non directory based backend is used, like RBD or LVM then, in"}],"source_content_type":"text/x-rst","patch_set":8,"id":"bab6814e_f47e82ad","line":65,"range":{"start_line":65,"start_character":45,"end_line":65,"end_character":51},"updated":"2016-05-26 16:53:21.000000000","message":"s/create/created/\n\nalso, s/sub // IMHO","commit_id":"51015fb0980091c985276ccd8149871003ac5ac8"},{"author":{"_account_id":6608,"name":"Paul Carlton","email":"paul.carlton2@hpe.com","username":"paul-carlton2"},"change_message_id":"9ba166c1321321655639743382a5a6bf8e1edfc4","unresolved":false,"context_lines":[{"line_number":62,"context_line":":code:`image_cache_subdirectory_name` in the instances directory as"},{"line_number":63,"context_line":"defined by the configuration flag  :code:`compute.instances_path`. This"},{"line_number":64,"context_line":"will automatically generate storage volumes for all the backing images."},{"line_number":65,"context_line":"The pool :code:`nova-disk-pool` will also be create referencing the sub"},{"line_number":66,"context_line":"directory :code:`_disk` in the instances directory."},{"line_number":67,"context_line":""},{"line_number":68,"context_line":"If a non directory based backend is used, like RBD or LVM then, in"}],"source_content_type":"text/x-rst","patch_set":8,"id":"9abb7d3a_c9dc351b","line":65,"range":{"start_line":65,"start_character":45,"end_line":65,"end_character":51},"in_reply_to":"bab6814e_f47e82ad","updated":"2016-05-27 08:44:10.000000000","message":"Done","commit_id":"51015fb0980091c985276ccd8149871003ac5ac8"},{"author":{"_account_id":12408,"name":"Maxim Nestratov","email":"mnestratov@virtuozzo.com","username":"mnestratov"},"change_message_id":"85e52d88b797bab11a7591256af581e056f8f6d9","unresolved":false,"context_lines":[{"line_number":65,"context_line":"The pool :code:`nova-disk-pool` will also be create referencing the sub"},{"line_number":66,"context_line":"directory :code:`_disk` in the instances directory."},{"line_number":67,"context_line":""},{"line_number":68,"context_line":"If a non directory based backend is used, like RBD or LVM then, in"},{"line_number":69,"context_line":"addition to the above Nova will create a storage pool of the appropriate"},{"line_number":70,"context_line":"type and storage volumes for each disk on that storage medium, i.e."},{"line_number":71,"context_line":":code:`nova-lvm-pool`, :code:`nova-ploop-pool` or :code:`nova-rbd-pool`"}],"source_content_type":"text/x-rst","patch_set":8,"id":"bab6814e_209ec406","line":68,"range":{"start_line":68,"start_character":0,"end_line":68,"end_character":40},"updated":"2016-05-23 16:35:57.000000000","message":"Actually ploop images are compatible with directory storage pools and I think there is no need in creating a special ploop \u0027nova-ploop-pool\u0027","commit_id":"51015fb0980091c985276ccd8149871003ac5ac8"},{"author":{"_account_id":6608,"name":"Paul Carlton","email":"paul.carlton2@hpe.com","username":"paul-carlton2"},"change_message_id":"8a7f50d64ef6a117f25887ffe5f04cc4fa00ba20","unresolved":false,"context_lines":[{"line_number":65,"context_line":"The pool :code:`nova-disk-pool` will also be create referencing the sub"},{"line_number":66,"context_line":"directory :code:`_disk` in the instances directory."},{"line_number":67,"context_line":""},{"line_number":68,"context_line":"If a non directory based backend is used, like RBD or LVM then, in"},{"line_number":69,"context_line":"addition to the above Nova will create a storage pool of the appropriate"},{"line_number":70,"context_line":"type and storage volumes for each disk on that storage medium, i.e."},{"line_number":71,"context_line":":code:`nova-lvm-pool`, :code:`nova-ploop-pool` or :code:`nova-rbd-pool`"}],"source_content_type":"text/x-rst","patch_set":8,"id":"bab6814e_a3e2bbd2","line":68,"range":{"start_line":68,"start_character":0,"end_line":68,"end_character":40},"in_reply_to":"bab6814e_209ec406","updated":"2016-05-23 17:24:47.000000000","message":"If this works then fine","commit_id":"51015fb0980091c985276ccd8149871003ac5ac8"},{"author":{"_account_id":4393,"name":"Dan Smith","email":"dms@danplanet.com","username":"danms"},"change_message_id":"ef9ece8984f2c5af25ac5a02f5e8abb58d28d324","unresolved":false,"context_lines":[{"line_number":70,"context_line":"type and storage volumes for each disk on that storage medium, i.e."},{"line_number":71,"context_line":":code:`nova-lvm-pool`, :code:`nova-ploop-pool` or :code:`nova-rbd-pool`"},{"line_number":72,"context_line":"will be created of the appropriate pool type to manage disk files"},{"line_number":73,"context_line":"stored on that medium."},{"line_number":74,"context_line":""},{"line_number":75,"context_line":"A new base class called Disk and sub classes for each backend type such"},{"line_number":76,"context_line":"as NoBacking (previously raw), Qcow2, Lvm, Rbd and Ploop will be added"}],"source_content_type":"text/x-rst","patch_set":8,"id":"bab6814e_d72ee08a","line":73,"updated":"2016-05-26 16:53:21.000000000","message":"This is a legit question, but, do we have to encode the name of the storage backend type in the name? Meaning, is it a problem to have a nova-instances directory pool and a nova-instances rbd pool?","commit_id":"51015fb0980091c985276ccd8149871003ac5ac8"},{"author":{"_account_id":6608,"name":"Paul Carlton","email":"paul.carlton2@hpe.com","username":"paul-carlton2"},"change_message_id":"9ba166c1321321655639743382a5a6bf8e1edfc4","unresolved":false,"context_lines":[{"line_number":70,"context_line":"type and storage volumes for each disk on that storage medium, i.e."},{"line_number":71,"context_line":":code:`nova-lvm-pool`, :code:`nova-ploop-pool` or :code:`nova-rbd-pool`"},{"line_number":72,"context_line":"will be created of the appropriate pool type to manage disk files"},{"line_number":73,"context_line":"stored on that medium."},{"line_number":74,"context_line":""},{"line_number":75,"context_line":"A new base class called Disk and sub classes for each backend type such"},{"line_number":76,"context_line":"as NoBacking (previously raw), Qcow2, Lvm, Rbd and Ploop will be added"}],"source_content_type":"text/x-rst","patch_set":8,"id":"9abb7d3a_49ca0551","line":73,"in_reply_to":"bab6814e_d72ee08a","updated":"2016-05-27 08:44:10.000000000","message":"The name of the pools do not denote the type, for example the nova-ploop-pool may be a directory type pool per Maxim\u0027s suggestion below. \n\nStorage pools do not directly align with backend types. For example Flat and Qcow2 disks will both reside in the nova-disk-pool, the volume entity will be different for Flat and Qcow2.\n\n\nAlso at present some backends (specifically LVM) create config disks on the compute node disk filesystem rather than on LVM.  In LVM case this is due to minimum file size issues.\nSo in the LVM case there will be two pools in use, the lvm pool and disk pool.\n\nAt present we effectively only support one backend type on a compute node but the imagebackend refactor and this work will aim to remove that restriction in terms of code implementation so in the future we could support different backend types per instance or even per disk.\n\nThat said I\u0027m open to changing the names of these pools, we just need one for each filesystem directory we store disk files in plus one for each specific backend type.","commit_id":"51015fb0980091c985276ccd8149871003ac5ac8"},{"author":{"_account_id":4393,"name":"Dan Smith","email":"dms@danplanet.com","username":"danms"},"change_message_id":"ef9ece8984f2c5af25ac5a02f5e8abb58d28d324","unresolved":false,"context_lines":[{"line_number":73,"context_line":"stored on that medium."},{"line_number":74,"context_line":""},{"line_number":75,"context_line":"A new base class called Disk and sub classes for each backend type such"},{"line_number":76,"context_line":"as NoBacking (previously raw), Qcow2, Lvm, Rbd and Ploop will be added"},{"line_number":77,"context_line":"to imagebackend.py to implement the use of libvirt pools and volumes."},{"line_number":78,"context_line":"When the implementation of the storage pools version of each backend"},{"line_number":79,"context_line":"type is complete the Backend constructor will be amended to use the new"}],"source_content_type":"text/x-rst","patch_set":8,"id":"bab6814e_17534818","line":76,"range":{"start_line":76,"start_character":3,"end_line":76,"end_character":12},"updated":"2016-05-26 16:53:21.000000000","message":"This is Flat now, right?\n\nAlso, per Maxim\u0027s comment above, do we need a Ploop one or will we be covered by Flat here?","commit_id":"51015fb0980091c985276ccd8149871003ac5ac8"},{"author":{"_account_id":6608,"name":"Paul Carlton","email":"paul.carlton2@hpe.com","username":"paul-carlton2"},"change_message_id":"9ba166c1321321655639743382a5a6bf8e1edfc4","unresolved":false,"context_lines":[{"line_number":73,"context_line":"stored on that medium."},{"line_number":74,"context_line":""},{"line_number":75,"context_line":"A new base class called Disk and sub classes for each backend type such"},{"line_number":76,"context_line":"as NoBacking (previously raw), Qcow2, Lvm, Rbd and Ploop will be added"},{"line_number":77,"context_line":"to imagebackend.py to implement the use of libvirt pools and volumes."},{"line_number":78,"context_line":"When the implementation of the storage pools version of each backend"},{"line_number":79,"context_line":"type is complete the Backend constructor will be amended to use the new"}],"source_content_type":"text/x-rst","patch_set":8,"id":"9abb7d3a_fe40cd09","line":76,"range":{"start_line":76,"start_character":3,"end_line":76,"end_character":12},"in_reply_to":"bab6814e_17534818","updated":"2016-05-27 08:44:10.000000000","message":"see comment below re ploop","commit_id":"51015fb0980091c985276ccd8149871003ac5ac8"},{"author":{"_account_id":4393,"name":"Dan Smith","email":"dms@danplanet.com","username":"danms"},"change_message_id":"ef9ece8984f2c5af25ac5a02f5e8abb58d28d324","unresolved":false,"context_lines":[{"line_number":90,"context_line":"The new LibvirtDiskMetadata object introduced in [1_] will be used to"},{"line_number":91,"context_line":"select to backend type to use for each disk. This will include details"},{"line_number":92,"context_line":"of the current location of the disk file. The new backend classes will"},{"line_number":93,"context_line":"use this information to determine if the a file move is required."},{"line_number":94,"context_line":""},{"line_number":95,"context_line":"The existing [1_] Image class and its sub classes (in imagebackend.py)"},{"line_number":96,"context_line":"will be left unchanged. The existing Image class and sub classes for"}],"source_content_type":"text/x-rst","patch_set":8,"id":"bab6814e_f78de47c","line":93,"range":{"start_line":93,"start_character":37,"end_line":93,"end_character":40},"updated":"2016-05-26 16:53:21.000000000","message":"s/the// ?","commit_id":"51015fb0980091c985276ccd8149871003ac5ac8"},{"author":{"_account_id":6608,"name":"Paul Carlton","email":"paul.carlton2@hpe.com","username":"paul-carlton2"},"change_message_id":"9ba166c1321321655639743382a5a6bf8e1edfc4","unresolved":false,"context_lines":[{"line_number":90,"context_line":"The new LibvirtDiskMetadata object introduced in [1_] will be used to"},{"line_number":91,"context_line":"select to backend type to use for each disk. This will include details"},{"line_number":92,"context_line":"of the current location of the disk file. The new backend classes will"},{"line_number":93,"context_line":"use this information to determine if the a file move is required."},{"line_number":94,"context_line":""},{"line_number":95,"context_line":"The existing [1_] Image class and its sub classes (in imagebackend.py)"},{"line_number":96,"context_line":"will be left unchanged. The existing Image class and sub classes for"}],"source_content_type":"text/x-rst","patch_set":8,"id":"9abb7d3a_3edd3571","line":93,"range":{"start_line":93,"start_character":37,"end_line":93,"end_character":40},"in_reply_to":"bab6814e_f78de47c","updated":"2016-05-27 08:44:10.000000000","message":"Done","commit_id":"51015fb0980091c985276ccd8149871003ac5ac8"},{"author":{"_account_id":12408,"name":"Maxim Nestratov","email":"mnestratov@virtuozzo.com","username":"mnestratov"},"change_message_id":"85e52d88b797bab11a7591256af581e056f8f6d9","unresolved":false,"context_lines":[{"line_number":105,"context_line":"storage pool (:code:`nova-base-images-pool`). This is done simply by"},{"line_number":106,"context_line":"instructing libvirt that Nova\u0027s image cache directory"},{"line_number":107,"context_line":"(:code:`/var/lib/nova/_base`) is a directory storage pool, and as such"},{"line_number":108,"context_line":"does not affect directory layout (and is thus compatible with both the"},{"line_number":109,"context_line":"legacy image backends and the new image backend proposed below)."},{"line_number":110,"context_line":""},{"line_number":111,"context_line":"If we are not using COW, the libvirt :code:`pool.createXMLFrom` method"},{"line_number":112,"context_line":"could be used to appropriately copy the template image from the source"}],"source_content_type":"text/x-rst","patch_set":8,"id":"bab6814e_db97e9e9","line":109,"range":{"start_line":108,"start_character":33,"end_line":109,"end_character":63},"updated":"2016-05-23 16:35:57.000000000","message":"This is a bit more complex with ploop images because currently they are stored as files in cache directory and when a new instance is spawned they become directories. When we start to use storage pools, ploop volumes are always directories, thus the storage pool volume refreshing code won\u0027t detect ploop files as volumes. Not to pollute system disk upgrading code should simply delete all ploop images from cache prior switching to storage pool scheme.","commit_id":"51015fb0980091c985276ccd8149871003ac5ac8"},{"author":{"_account_id":12408,"name":"Maxim Nestratov","email":"mnestratov@virtuozzo.com","username":"mnestratov"},"change_message_id":"7e275b740ff25f6f64728cc6bf9e8bca86cf1a4d","unresolved":false,"context_lines":[{"line_number":105,"context_line":"storage pool (:code:`nova-base-images-pool`). This is done simply by"},{"line_number":106,"context_line":"instructing libvirt that Nova\u0027s image cache directory"},{"line_number":107,"context_line":"(:code:`/var/lib/nova/_base`) is a directory storage pool, and as such"},{"line_number":108,"context_line":"does not affect directory layout (and is thus compatible with both the"},{"line_number":109,"context_line":"legacy image backends and the new image backend proposed below)."},{"line_number":110,"context_line":""},{"line_number":111,"context_line":"If we are not using COW, the libvirt :code:`pool.createXMLFrom` method"},{"line_number":112,"context_line":"could be used to appropriately copy the template image from the source"}],"source_content_type":"text/x-rst","patch_set":8,"id":"bab6814e_c6bc2dde","line":109,"range":{"start_line":108,"start_character":33,"end_line":109,"end_character":63},"in_reply_to":"bab6814e_c3625f3d","updated":"2016-05-23 17:38:56.000000000","message":"No. Deleting image cache files won\u0027t destroy instance data.\nTo play with loop images you will need to install Virtuozzo first. Here are howtos: https://openvz.org/Quick_installation_of_Virtuozzo_7\nhttps://openvz.org/Setup_OpenStack_with_Virtuozzo_7","commit_id":"51015fb0980091c985276ccd8149871003ac5ac8"},{"author":{"_account_id":6608,"name":"Paul Carlton","email":"paul.carlton2@hpe.com","username":"paul-carlton2"},"change_message_id":"9ba166c1321321655639743382a5a6bf8e1edfc4","unresolved":false,"context_lines":[{"line_number":105,"context_line":"storage pool (:code:`nova-base-images-pool`). This is done simply by"},{"line_number":106,"context_line":"instructing libvirt that Nova\u0027s image cache directory"},{"line_number":107,"context_line":"(:code:`/var/lib/nova/_base`) is a directory storage pool, and as such"},{"line_number":108,"context_line":"does not affect directory layout (and is thus compatible with both the"},{"line_number":109,"context_line":"legacy image backends and the new image backend proposed below)."},{"line_number":110,"context_line":""},{"line_number":111,"context_line":"If we are not using COW, the libvirt :code:`pool.createXMLFrom` method"},{"line_number":112,"context_line":"could be used to appropriately copy the template image from the source"}],"source_content_type":"text/x-rst","patch_set":8,"id":"9abb7d3a_fe59ed8f","line":109,"range":{"start_line":108,"start_character":33,"end_line":109,"end_character":63},"in_reply_to":"bab6814e_c6bc2dde","updated":"2016-05-27 08:44:10.000000000","message":"Thanks, for now I\u0027ve updated the spec to assume that the storage pool type used for ploop backend is a dir type pool.  I\u0027ll have a play with ploop backend in devstack and develop a strategy for implementation an migration","commit_id":"51015fb0980091c985276ccd8149871003ac5ac8"},{"author":{"_account_id":6608,"name":"Paul Carlton","email":"paul.carlton2@hpe.com","username":"paul-carlton2"},"change_message_id":"8a7f50d64ef6a117f25887ffe5f04cc4fa00ba20","unresolved":false,"context_lines":[{"line_number":105,"context_line":"storage pool (:code:`nova-base-images-pool`). This is done simply by"},{"line_number":106,"context_line":"instructing libvirt that Nova\u0027s image cache directory"},{"line_number":107,"context_line":"(:code:`/var/lib/nova/_base`) is a directory storage pool, and as such"},{"line_number":108,"context_line":"does not affect directory layout (and is thus compatible with both the"},{"line_number":109,"context_line":"legacy image backends and the new image backend proposed below)."},{"line_number":110,"context_line":""},{"line_number":111,"context_line":"If we are not using COW, the libvirt :code:`pool.createXMLFrom` method"},{"line_number":112,"context_line":"could be used to appropriately copy the template image from the source"}],"source_content_type":"text/x-rst","patch_set":8,"id":"bab6814e_c3625f3d","line":109,"range":{"start_line":108,"start_character":33,"end_line":109,"end_character":63},"in_reply_to":"bab6814e_db97e9e9","updated":"2016-05-23 17:24:47.000000000","message":"Deleting these files will destroy instance data?\n\nMy understanding of ploop backend is limited, I\u0027d like to get a devstack environment running with a ploop backend so I can explore this further, do you have any guidance on how to do this please?","commit_id":"51015fb0980091c985276ccd8149871003ac5ac8"},{"author":{"_account_id":6608,"name":"Paul Carlton","email":"paul.carlton2@hpe.com","username":"paul-carlton2"},"change_message_id":"7df3dcdbb9768dd24e6d63f0a39a91db4585dec0","unresolved":false,"context_lines":[{"line_number":105,"context_line":"storage pool (:code:`nova-base-images-pool`). This is done simply by"},{"line_number":106,"context_line":"instructing libvirt that Nova\u0027s image cache directory"},{"line_number":107,"context_line":"(:code:`/var/lib/nova/_base`) is a directory storage pool, and as such"},{"line_number":108,"context_line":"does not affect directory layout (and is thus compatible with both the"},{"line_number":109,"context_line":"legacy image backends and the new image backend proposed below)."},{"line_number":110,"context_line":""},{"line_number":111,"context_line":"If we are not using COW, the libvirt :code:`pool.createXMLFrom` method"},{"line_number":112,"context_line":"could be used to appropriately copy the template image from the source"}],"source_content_type":"text/x-rst","patch_set":8,"id":"dada55a8_fdbc9dbc","line":109,"range":{"start_line":108,"start_character":33,"end_line":109,"end_character":63},"in_reply_to":"bab6814e_db97e9e9","updated":"2016-07-20 22:58:46.000000000","message":"ok","commit_id":"51015fb0980091c985276ccd8149871003ac5ac8"},{"author":{"_account_id":4393,"name":"Dan Smith","email":"dms@danplanet.com","username":"danms"},"change_message_id":"ef9ece8984f2c5af25ac5a02f5e8abb58d28d324","unresolved":false,"context_lines":[{"line_number":190,"context_line":"For live migration/upgrade from OpenStack Mitaka to OpenStack Newton,"},{"line_number":191,"context_line":"the legacy image backends (and support for them in Nova\u0027s image cache)"},{"line_number":192,"context_line":"will be left in place for the next release (Ocata), but will be marked"},{"line_number":193,"context_line":"as deprecated. In the P release, the legacy backends will be removed"},{"line_number":194,"context_line":"(as well as support for them in the image cache manager). This will"},{"line_number":195,"context_line":"mean after upgrading to the P release the live-snapshot operation will"},{"line_number":196,"context_line":"not be available on any instance not recreated using the new backend"}],"source_content_type":"text/x-rst","patch_set":8,"id":"bab6814e_57b670f9","line":193,"range":{"start_line":193,"start_character":3,"end_line":193,"end_character":13},"updated":"2016-05-26 16:53:21.000000000","message":"I think you can mark them deprecated in Newton if we have storage pools fully working and then potentially remove them in Ocata.\n\nHowever, since this is going to effectively force operators to do a full reboot of all instances before they can drop the old stuff, more time may be good.","commit_id":"51015fb0980091c985276ccd8149871003ac5ac8"},{"author":{"_account_id":6608,"name":"Paul Carlton","email":"paul.carlton2@hpe.com","username":"paul-carlton2"},"change_message_id":"9ba166c1321321655639743382a5a6bf8e1edfc4","unresolved":false,"context_lines":[{"line_number":190,"context_line":"For live migration/upgrade from OpenStack Mitaka to OpenStack Newton,"},{"line_number":191,"context_line":"the legacy image backends (and support for them in Nova\u0027s image cache)"},{"line_number":192,"context_line":"will be left in place for the next release (Ocata), but will be marked"},{"line_number":193,"context_line":"as deprecated. In the P release, the legacy backends will be removed"},{"line_number":194,"context_line":"(as well as support for them in the image cache manager). This will"},{"line_number":195,"context_line":"mean after upgrading to the P release the live-snapshot operation will"},{"line_number":196,"context_line":"not be available on any instance not recreated using the new backend"}],"source_content_type":"text/x-rst","patch_set":8,"id":"9abb7d3a_5e8d997a","line":193,"range":{"start_line":193,"start_character":3,"end_line":193,"end_character":13},"in_reply_to":"bab6814e_57b670f9","updated":"2016-05-27 08:44:10.000000000","message":"agreed, if we can pull this removal in all well and good but best to be conservative.","commit_id":"51015fb0980091c985276ccd8149871003ac5ac8"},{"author":{"_account_id":4393,"name":"Dan Smith","email":"dms@danplanet.com","username":"danms"},"change_message_id":"ef9ece8984f2c5af25ac5a02f5e8abb58d28d324","unresolved":false,"context_lines":[{"line_number":195,"context_line":"mean after upgrading to the P release the live-snapshot operation will"},{"line_number":196,"context_line":"not be available on any instance not recreated using the new backend"},{"line_number":197,"context_line":"classes.  The live-migration operation could be used to upgrade the"},{"line_number":198,"context_line":"instance without application downtime."},{"line_number":199,"context_line":""},{"line_number":200,"context_line":"A utility will be provided to identify instances still using the legacy"},{"line_number":201,"context_line":"disk layout so that operators can manage their upgrade."}],"source_content_type":"text/x-rst","patch_set":8,"id":"bab6814e_f74b24af","line":198,"updated":"2016-05-26 16:53:21.000000000","message":"So, I was going to ask or confirm, but I hope this is it...\n\nI assume that live and cold migration from older compute nodes to newer ones that are using pools is fully supportable, yes?","commit_id":"51015fb0980091c985276ccd8149871003ac5ac8"},{"author":{"_account_id":6608,"name":"Paul Carlton","email":"paul.carlton2@hpe.com","username":"paul-carlton2"},"change_message_id":"9ba166c1321321655639743382a5a6bf8e1edfc4","unresolved":false,"context_lines":[{"line_number":195,"context_line":"mean after upgrading to the P release the live-snapshot operation will"},{"line_number":196,"context_line":"not be available on any instance not recreated using the new backend"},{"line_number":197,"context_line":"classes.  The live-migration operation could be used to upgrade the"},{"line_number":198,"context_line":"instance without application downtime."},{"line_number":199,"context_line":""},{"line_number":200,"context_line":"A utility will be provided to identify instances still using the legacy"},{"line_number":201,"context_line":"disk layout so that operators can manage their upgrade."}],"source_content_type":"text/x-rst","patch_set":8,"id":"9abb7d3a_3e6895ba","line":198,"in_reply_to":"bab6814e_f74b24af","updated":"2016-05-27 08:44:10.000000000","message":"live migration will not be impacted by this change, i.e. you can live migrate instances from pre storage pool nodes to storage pool nodes because the instance creation on the target will be executed using the new storage pool approach and once the instance is started paused libvirt/qemu will copy the data.\n\nCold migration from older nodes would have to use the old mechanism since the code to support storage volume migration is not available.  This may work if the cloud operator has enabled ssh keys to support it and left the same enabled on the upgraded nodes but the files would be copied to the old style locations on the upgraded node.\n\nCold migration implies a shutdown instance so a start operation is required at some point.  When start is executed the upgraded node will detect the old style instance file layout and move files to the new layout creating storage volumes as required.\n\nSo the user can start the instance on the old node and live migrate it or cold migrate it using the the old cold migration mechanism them start it on the new node.","commit_id":"51015fb0980091c985276ccd8149871003ac5ac8"},{"author":{"_account_id":4393,"name":"Dan Smith","email":"dms@danplanet.com","username":"danms"},"change_message_id":"ef9ece8984f2c5af25ac5a02f5e8abb58d28d324","unresolved":false,"context_lines":[{"line_number":230,"context_line":"    paul-carlton2"},{"line_number":231,"context_line":""},{"line_number":232,"context_line":"Other contributors:"},{"line_number":233,"context_line":"    None"},{"line_number":234,"context_line":""},{"line_number":235,"context_line":"Work Items"},{"line_number":236,"context_line":"----------"}],"source_content_type":"text/x-rst","patch_set":8,"id":"bab6814e_9d6e392d","line":233,"updated":"2016-05-26 16:53:21.000000000","message":"Is this list of other contributors correct?","commit_id":"51015fb0980091c985276ccd8149871003ac5ac8"},{"author":{"_account_id":6608,"name":"Paul Carlton","email":"paul.carlton2@hpe.com","username":"paul-carlton2"},"change_message_id":"9ba166c1321321655639743382a5a6bf8e1edfc4","unresolved":false,"context_lines":[{"line_number":230,"context_line":"    paul-carlton2"},{"line_number":231,"context_line":""},{"line_number":232,"context_line":"Other contributors:"},{"line_number":233,"context_line":"    None"},{"line_number":234,"context_line":""},{"line_number":235,"context_line":"Work Items"},{"line_number":236,"context_line":"----------"}],"source_content_type":"text/x-rst","patch_set":8,"id":"9abb7d3a_436dc631","line":233,"in_reply_to":"bab6814e_9d6e392d","updated":"2016-05-27 08:44:10.000000000","message":"Done","commit_id":"51015fb0980091c985276ccd8149871003ac5ac8"},{"author":{"_account_id":4393,"name":"Dan Smith","email":"dms@danplanet.com","username":"danms"},"change_message_id":"ef9ece8984f2c5af25ac5a02f5e8abb58d28d324","unresolved":false,"context_lines":[{"line_number":245,"context_line":"   volumes for backing stores."},{"line_number":246,"context_line":""},{"line_number":247,"context_line":"3. Implement the functionality required to support transition by"},{"line_number":248,"context_line":"   creating pools and volumes based on the existing files."},{"line_number":249,"context_line":""},{"line_number":250,"context_line":""},{"line_number":251,"context_line":"Dependencies"}],"source_content_type":"text/x-rst","patch_set":8,"id":"bab6814e_fd7e65e2","line":248,"updated":"2016-05-26 16:53:21.000000000","message":"Since we can\u0027t have a gap in support, this may need to be before #2?","commit_id":"51015fb0980091c985276ccd8149871003ac5ac8"},{"author":{"_account_id":6608,"name":"Paul Carlton","email":"paul.carlton2@hpe.com","username":"paul-carlton2"},"change_message_id":"9ba166c1321321655639743382a5a6bf8e1edfc4","unresolved":false,"context_lines":[{"line_number":245,"context_line":"   volumes for backing stores."},{"line_number":246,"context_line":""},{"line_number":247,"context_line":"3. Implement the functionality required to support transition by"},{"line_number":248,"context_line":"   creating pools and volumes based on the existing files."},{"line_number":249,"context_line":""},{"line_number":250,"context_line":""},{"line_number":251,"context_line":"Dependencies"}],"source_content_type":"text/x-rst","patch_set":8,"id":"9abb7d3a_a3647249","line":248,"in_reply_to":"bab6814e_fd7e65e2","updated":"2016-05-27 08:44:10.000000000","message":"Done","commit_id":"51015fb0980091c985276ccd8149871003ac5ac8"},{"author":{"_account_id":12408,"name":"Maxim Nestratov","email":"mnestratov@virtuozzo.com","username":"mnestratov"},"change_message_id":"7e4daf22e386b0283794f08af7bcd90a2901a966","unresolved":false,"context_lines":[{"line_number":62,"context_line":"If a non directory based backend is used, like RBD or LVM then, in"},{"line_number":63,"context_line":"addition to the above Nova will create a storage pool of the appropriate"},{"line_number":64,"context_line":"type and storage volumes for each disk on that storage medium, i.e."},{"line_number":65,"context_line":":code:`nova-lvm-pool`, :code:`nova-ploop-pool` or :code:`nova-rbd-pool`"},{"line_number":66,"context_line":"will be created of the appropriate pool type to manage disk files"},{"line_number":67,"context_line":"stored on that medium."},{"line_number":68,"context_line":""}],"source_content_type":"text/x-rst","patch_set":9,"id":"9abb7d3a_f8b8e0e7","line":65,"range":{"start_line":65,"start_character":23,"end_line":65,"end_character":45},"updated":"2016-05-27 09:20:25.000000000","message":"I\u0027m still against introducing \u0027nova-ploop-pool\u0027. \u0027nova-disk-pool\u0027 is ok for ploop images.","commit_id":"2b22dc5e2e1135fe65363c0f5b14c9b5f5b1fabd"},{"author":{"_account_id":12408,"name":"Maxim Nestratov","email":"mnestratov@virtuozzo.com","username":"mnestratov"},"change_message_id":"6e2ff261b1e96d545b0c2a41f7340a76506191ec","unresolved":false,"context_lines":[{"line_number":62,"context_line":"If a non directory based backend is used, like RBD or LVM then, in"},{"line_number":63,"context_line":"addition to the above Nova will create a storage pool of the appropriate"},{"line_number":64,"context_line":"type and storage volumes for each disk on that storage medium, i.e."},{"line_number":65,"context_line":":code:`nova-lvm-pool`, :code:`nova-ploop-pool` or :code:`nova-rbd-pool`"},{"line_number":66,"context_line":"will be created of the appropriate pool type to manage disk files"},{"line_number":67,"context_line":"stored on that medium."},{"line_number":68,"context_line":""}],"source_content_type":"text/x-rst","patch_set":9,"id":"7aa08908_98e5dddd","line":65,"range":{"start_line":65,"start_character":23,"end_line":65,"end_character":45},"in_reply_to":"7aa08908_41bee465","updated":"2016-06-07 16:18:00.000000000","message":"Paul, what do you think about implementing storage pools in a way when we can choose what image backend we use in runtime, depending on either image format or additional image format mapping config option as I suggested in previous cycle (https://review.openstack.org/#/c/240163/) or even some image metadata (storage_pool\u003dpoolname)? I didn\u0027t resubmit it this cycle because I thought it would conflict with storage pools activity. Does it make sense?","commit_id":"2b22dc5e2e1135fe65363c0f5b14c9b5f5b1fabd"},{"author":{"_account_id":6608,"name":"Paul Carlton","email":"paul.carlton2@hpe.com","username":"paul-carlton2"},"change_message_id":"9a65f4b2c05be23b930fe7ea0e2f8f2ed36e3974","unresolved":false,"context_lines":[{"line_number":62,"context_line":"If a non directory based backend is used, like RBD or LVM then, in"},{"line_number":63,"context_line":"addition to the above Nova will create a storage pool of the appropriate"},{"line_number":64,"context_line":"type and storage volumes for each disk on that storage medium, i.e."},{"line_number":65,"context_line":":code:`nova-lvm-pool`, :code:`nova-ploop-pool` or :code:`nova-rbd-pool`"},{"line_number":66,"context_line":"will be created of the appropriate pool type to manage disk files"},{"line_number":67,"context_line":"stored on that medium."},{"line_number":68,"context_line":""}],"source_content_type":"text/x-rst","patch_set":9,"id":"7aa08908_41bee465","line":65,"range":{"start_line":65,"start_character":23,"end_line":65,"end_character":45},"in_reply_to":"7aa08908_44a3954f","updated":"2016-06-07 15:14:09.000000000","message":"Matt, my thinking was that although at present you can only have one backend type on a compute node we should not do anything to prevent the use of multiple backends on a single compute node in the future.  Thus we need separate pools for different backend types.  Max may be right that we can get away with using a disk type pool for ploop (as we can for other shared storage implementations like nfs that are invisible to the nova compute).  However, my analysis leads me to believe that for lvm and rbd we need a pool of the appropriate type. I\u0027m happy to call them something else but not sure I see the point.  My thinking is that when a backend type is first used the appropriate pool will be created unless the backend implementation is such that the nova-disk-pool (a directory type pool mapped to \u003ccompute.instances_path\u003e/_disks) covers it as with flat and qcow2 and maybe ploop too id Max is right. I was thinking that the backend type class would define the pool type to be used and pool name so a new pool of this type will be created using the configuration information applicable to that backend type when the backend was first used. If the pool name is nova-disk-pool then the pre exsiting pool can be used.","commit_id":"2b22dc5e2e1135fe65363c0f5b14c9b5f5b1fabd"},{"author":{"_account_id":12408,"name":"Maxim Nestratov","email":"mnestratov@virtuozzo.com","username":"mnestratov"},"change_message_id":"fdf68b6ff1e0e0512427c0801597dad3160cdf4c","unresolved":false,"context_lines":[{"line_number":62,"context_line":"If a non directory based backend is used, like RBD or LVM then, in"},{"line_number":63,"context_line":"addition to the above Nova will create a storage pool of the appropriate"},{"line_number":64,"context_line":"type and storage volumes for each disk on that storage medium, i.e."},{"line_number":65,"context_line":":code:`nova-lvm-pool`, :code:`nova-ploop-pool` or :code:`nova-rbd-pool`"},{"line_number":66,"context_line":"will be created of the appropriate pool type to manage disk files"},{"line_number":67,"context_line":"stored on that medium."},{"line_number":68,"context_line":""}],"source_content_type":"text/x-rst","patch_set":9,"id":"7aa08908_4114249f","line":65,"range":{"start_line":65,"start_character":23,"end_line":65,"end_character":45},"in_reply_to":"7aa08908_44a3954f","updated":"2016-06-07 15:10:53.000000000","message":"Why should we ignore it in the spec? Isn\u0027t the purpose of  specs to agree on the way we go? Speaking about maturity, what kind of maturity we need to disscuss how it will be implemented. Could we just require Virtuozzo CI passing just not to break what already works. I\u0027ll take all the parts necessary to fix any issues with ploop.","commit_id":"2b22dc5e2e1135fe65363c0f5b14c9b5f5b1fabd"},{"author":{"_account_id":6608,"name":"Paul Carlton","email":"paul.carlton2@hpe.com","username":"paul-carlton2"},"change_message_id":"37a42fce9f3f1e0bb3f7010620463e0cf52a5a74","unresolved":false,"context_lines":[{"line_number":62,"context_line":"If a non directory based backend is used, like RBD or LVM then, in"},{"line_number":63,"context_line":"addition to the above Nova will create a storage pool of the appropriate"},{"line_number":64,"context_line":"type and storage volumes for each disk on that storage medium, i.e."},{"line_number":65,"context_line":":code:`nova-lvm-pool`, :code:`nova-ploop-pool` or :code:`nova-rbd-pool`"},{"line_number":66,"context_line":"will be created of the appropriate pool type to manage disk files"},{"line_number":67,"context_line":"stored on that medium."},{"line_number":68,"context_line":""}],"source_content_type":"text/x-rst","patch_set":9,"id":"dada55a8_52db184f","line":65,"range":{"start_line":65,"start_character":23,"end_line":65,"end_character":45},"in_reply_to":"7aa08908_98e5dddd","updated":"2016-07-20 22:30:39.000000000","message":"We should not do anything to prevent implementation of this spec in a subsequent cycle","commit_id":"2b22dc5e2e1135fe65363c0f5b14c9b5f5b1fabd"},{"author":{"_account_id":12408,"name":"Maxim Nestratov","email":"mnestratov@virtuozzo.com","username":"mnestratov"},"change_message_id":"ab91e3cdd3c05eb3ecc0e7bcddfe8eda274e8107","unresolved":false,"context_lines":[{"line_number":62,"context_line":"If a non directory based backend is used, like RBD or LVM then, in"},{"line_number":63,"context_line":"addition to the above Nova will create a storage pool of the appropriate"},{"line_number":64,"context_line":"type and storage volumes for each disk on that storage medium, i.e."},{"line_number":65,"context_line":":code:`nova-lvm-pool`, :code:`nova-ploop-pool` or :code:`nova-rbd-pool`"},{"line_number":66,"context_line":"will be created of the appropriate pool type to manage disk files"},{"line_number":67,"context_line":"stored on that medium."},{"line_number":68,"context_line":""}],"source_content_type":"text/x-rst","patch_set":9,"id":"9abb7d3a_5e87e6d6","line":65,"range":{"start_line":65,"start_character":23,"end_line":65,"end_character":45},"in_reply_to":"9abb7d3a_3b7422ce","updated":"2016-05-31 06:45:09.000000000","message":"I don\u0027t get your point. Ploop image files will appear where we tell them to appear. The only difference of ploop images from qcow/qcow2 is a format itself and the fact that beside disk image there is another file called DiskDescriptor.xml (and snapshot files, if any, but we don\u0027t use them in nova). Thus, image path we use as an instance\u0027 virtual disk points to a directory where image and and DiskDescriptor.xml are resided. I.e. if we use nova-disk-pool which maps to CONF.instances_path + _disks, then there any Flat and Qcow2 files will be stored *plus* ploop directories. Why should we prevent it and create another storage for ploop directories?","commit_id":"2b22dc5e2e1135fe65363c0f5b14c9b5f5b1fabd"},{"author":{"_account_id":9555,"name":"Matthew Booth","email":"mbooth@redhat.com","username":"MatthewBooth"},"change_message_id":"f27cf883778d99a4d3bb7d4ac3e75f157dea06bb","unresolved":false,"context_lines":[{"line_number":62,"context_line":"If a non directory based backend is used, like RBD or LVM then, in"},{"line_number":63,"context_line":"addition to the above Nova will create a storage pool of the appropriate"},{"line_number":64,"context_line":"type and storage volumes for each disk on that storage medium, i.e."},{"line_number":65,"context_line":":code:`nova-lvm-pool`, :code:`nova-ploop-pool` or :code:`nova-rbd-pool`"},{"line_number":66,"context_line":"will be created of the appropriate pool type to manage disk files"},{"line_number":67,"context_line":"stored on that medium."},{"line_number":68,"context_line":""}],"source_content_type":"text/x-rst","patch_set":9,"id":"7aa08908_44a3954f","line":65,"range":{"start_line":65,"start_character":23,"end_line":65,"end_character":45},"in_reply_to":"9abb7d3a_5e87e6d6","updated":"2016-06-07 14:54:35.000000000","message":"2 things. I\u0027m also against \u0027nova-lvm-pool\u0027 and \u0027nova-rbd-pool\u0027, but only because these things can be introspected from the pool itself. We can continue to use nova-disk-pool, and if that refers to an RBD pool then we will use the appropriate RBD implementation for those disks.\n\nSecondly, I think we should probably ignore ploop for this round and not provide a libvirt storage pools implementation for it. We\u0027re not forced to do them all at once, so lets wait for it to land and mature upstream.","commit_id":"2b22dc5e2e1135fe65363c0f5b14c9b5f5b1fabd"},{"author":{"_account_id":6608,"name":"Paul Carlton","email":"paul.carlton2@hpe.com","username":"paul-carlton2"},"change_message_id":"37a42fce9f3f1e0bb3f7010620463e0cf52a5a74","unresolved":false,"context_lines":[{"line_number":62,"context_line":"If a non directory based backend is used, like RBD or LVM then, in"},{"line_number":63,"context_line":"addition to the above Nova will create a storage pool of the appropriate"},{"line_number":64,"context_line":"type and storage volumes for each disk on that storage medium, i.e."},{"line_number":65,"context_line":":code:`nova-lvm-pool`, :code:`nova-ploop-pool` or :code:`nova-rbd-pool`"},{"line_number":66,"context_line":"will be created of the appropriate pool type to manage disk files"},{"line_number":67,"context_line":"stored on that medium."},{"line_number":68,"context_line":""}],"source_content_type":"text/x-rst","patch_set":9,"id":"dada55a8_d2ab8873","line":65,"range":{"start_line":65,"start_character":23,"end_line":65,"end_character":45},"in_reply_to":"9abb7d3a_5e87e6d6","updated":"2016-07-20 22:30:39.000000000","message":"I get it now, there is no ploop storage pool, but there is a new storage volume type of ploop which can be used to describe ploop images, i.e. the directory containing the DiskDescriptor.xml and root.hds files.","commit_id":"2b22dc5e2e1135fe65363c0f5b14c9b5f5b1fabd"},{"author":{"_account_id":6608,"name":"Paul Carlton","email":"paul.carlton2@hpe.com","username":"paul-carlton2"},"change_message_id":"3f713cb62a8b7ca6641569e9d08a040524ac59fb","unresolved":false,"context_lines":[{"line_number":62,"context_line":"If a non directory based backend is used, like RBD or LVM then, in"},{"line_number":63,"context_line":"addition to the above Nova will create a storage pool of the appropriate"},{"line_number":64,"context_line":"type and storage volumes for each disk on that storage medium, i.e."},{"line_number":65,"context_line":":code:`nova-lvm-pool`, :code:`nova-ploop-pool` or :code:`nova-rbd-pool`"},{"line_number":66,"context_line":"will be created of the appropriate pool type to manage disk files"},{"line_number":67,"context_line":"stored on that medium."},{"line_number":68,"context_line":""}],"source_content_type":"text/x-rst","patch_set":9,"id":"9abb7d3a_3b7422ce","line":65,"range":{"start_line":65,"start_character":23,"end_line":65,"end_character":45},"in_reply_to":"9abb7d3a_f8b8e0e7","updated":"2016-05-27 09:49:40.000000000","message":"We need a separate pool, it can be a disk type pool but the nova-disk-pool will map to CONF.instances_path + _disks where any Flat and Qcow2 files will be stored that are not using ploop or other non disk filesystem backends.  I\u0027m assuming that the ploop backend files will appear in the file system at a different place in the filesystem or at  least might be in which case we can support this by having an other pool, remember I don\u0027t want to do anything to prevent support in the future for multiple backend types on a compute node.  However, as I said I am not familiar with ploop so I may be misunderstanding something here.  Next week I will build a ploop devstack and try out my storage pools implementation there which may inform my views on this point.","commit_id":"2b22dc5e2e1135fe65363c0f5b14c9b5f5b1fabd"},{"author":{"_account_id":6608,"name":"Paul Carlton","email":"paul.carlton2@hpe.com","username":"paul-carlton2"},"change_message_id":"37a42fce9f3f1e0bb3f7010620463e0cf52a5a74","unresolved":false,"context_lines":[{"line_number":62,"context_line":"If a non directory based backend is used, like RBD or LVM then, in"},{"line_number":63,"context_line":"addition to the above Nova will create a storage pool of the appropriate"},{"line_number":64,"context_line":"type and storage volumes for each disk on that storage medium, i.e."},{"line_number":65,"context_line":":code:`nova-lvm-pool`, :code:`nova-ploop-pool` or :code:`nova-rbd-pool`"},{"line_number":66,"context_line":"will be created of the appropriate pool type to manage disk files"},{"line_number":67,"context_line":"stored on that medium."},{"line_number":68,"context_line":""}],"source_content_type":"text/x-rst","patch_set":9,"id":"dada55a8_32b304eb","line":65,"range":{"start_line":65,"start_character":23,"end_line":65,"end_character":45},"in_reply_to":"9abb7d3a_f8b8e0e7","updated":"2016-07-20 22:30:39.000000000","message":"agreed, see below","commit_id":"2b22dc5e2e1135fe65363c0f5b14c9b5f5b1fabd"},{"author":{"_account_id":16907,"name":"Diana Clarke","email":"diana.joan.clarke@gmail.com","username":"diana-clarke"},"change_message_id":"0a64454df6f0ec3cb2d4ed14bf3e5becf96508c9","unresolved":false,"context_lines":[{"line_number":89,"context_line":"The existing [1_] Image class and its sub classes (in imagebackend.py)"},{"line_number":90,"context_line":"will be left unchanged. The existing Image class and sub classes for"},{"line_number":91,"context_line":"each backend will remain to support actions on existing instances which"},{"line_number":92,"context_line":"do not involve a dstroy and recreate, i.e. live snapshot."},{"line_number":93,"context_line":""},{"line_number":94,"context_line":"The legacy backends would be left in place to maintain the live upgrade"},{"line_number":95,"context_line":"functionality (e.g. Newton-\u003eOcata). See the `Other deployer impact`_"}],"source_content_type":"text/x-rst","patch_set":9,"id":"7aa08908_ab573724","line":92,"range":{"start_line":92,"start_character":17,"end_line":92,"end_character":23},"updated":"2016-06-09 03:24:54.000000000","message":"destroy","commit_id":"2b22dc5e2e1135fe65363c0f5b14c9b5f5b1fabd"}],"specs/ocata/approved/use-libvirt-storage-pools.rst":[{"author":{"_account_id":1779,"name":"Daniel Berrange","email":"berrange@redhat.com","username":"berrange"},"change_message_id":"4f21363bccf34a7058a7d1ff4f8a23bec201d804","unresolved":false,"context_lines":[{"line_number":188,"context_line":"mean after upgrading to the P release the live-snapshot operation will"},{"line_number":189,"context_line":"not be available on any instance not recreated using the new backend"},{"line_number":190,"context_line":"classes.  The live-migration operation could be used to upgrade the"},{"line_number":191,"context_line":"instance without application downtime."},{"line_number":192,"context_line":""},{"line_number":193,"context_line":"A utility will be provided to identify instances still using the legacy"},{"line_number":194,"context_line":"disk layout so that operators can manage their upgrade."}],"source_content_type":"text/x-rst","patch_set":11,"id":"bacf61ea_0af9e9c8","line":191,"updated":"2016-08-03 08:48:49.000000000","message":"In the paragraph starting line 77, you state the instances will only have their disks moved to the new layout in operations that imply a cold restart.  This makes sense because no QEMU process will have the disk open at this time.\n\nHere though, you are implying that live migration will also be  point at which disks are moved to the new layout.\n\nIf true, then how will this be done during live migration ? When you move the disk images you really only want todo so when no QEMU has them open. During live migration this is never the case - the disk is always open by 1 QEMU and sometimes by both QEMUs. Are you proposing to move the disk to the new layout, despite them still being open by existing QEMU on the source. If so that needs to be spelled out clearly and what failure scenarios this can lead to.\n\n\nIf we\u0027re not proposing to move disks to the new layout during a live migration, then IMHO deleting the legacy code after just 1 release is way too premature. Last time we changed the instance disk layout was in Folsom, and we\u0027re only just considering deleting the code for understanding the legacy layout in this Mitaka release. That is probably too excessive a grace period, but I really think more than 1 release cycle is required, if the admin is going to be forced to cold-restart the guests to get them switched to the new layout","commit_id":"b8afa59747706ad8317dfa25c782de37c1dda945"},{"author":{"_account_id":1779,"name":"Daniel Berrange","email":"berrange@redhat.com","username":"berrange"},"change_message_id":"b08250b671433040f7b363c357f9b0ba66e5875d","unresolved":false,"context_lines":[{"line_number":188,"context_line":"mean after upgrading to the P release the live-snapshot operation will"},{"line_number":189,"context_line":"not be available on any instance not recreated using the new backend"},{"line_number":190,"context_line":"classes.  The live-migration operation could be used to upgrade the"},{"line_number":191,"context_line":"instance without application downtime."},{"line_number":192,"context_line":""},{"line_number":193,"context_line":"A utility will be provided to identify instances still using the legacy"},{"line_number":194,"context_line":"disk layout so that operators can manage their upgrade."}],"source_content_type":"text/x-rst","patch_set":11,"id":"bacf61ea_2a1dcdbd","line":191,"in_reply_to":"bacf61ea_0af9e9c8","updated":"2016-08-03 08:54:07.000000000","message":"Also we should say what we\u0027re doing about saved images. eg if a guest is saved using the virt driver \u0027suspend\u0027 method, this saves the state of the guest to disk, including the XML config which is then automatically used by libvirt when starting the guest again later - even if the application is passing new XML into the domainCreateXML API call, libvirt will still use the XML embedded in the saved state image.\n\nSo either you\u0027ll break saved instance images when dropping support for the legacy code, or you\u0027ll need to provide provide to re-write the XML for the saved image to fix the paths","commit_id":"b8afa59747706ad8317dfa25c782de37c1dda945"},{"author":{"_account_id":6608,"name":"Paul Carlton","email":"paul.carlton2@hpe.com","username":"paul-carlton2"},"change_message_id":"54bd39cca6c2dc43efb46e67ca53b881fd2f6cb0","unresolved":false,"context_lines":[{"line_number":188,"context_line":"mean after upgrading to the P release the live-snapshot operation will"},{"line_number":189,"context_line":"not be available on any instance not recreated using the new backend"},{"line_number":190,"context_line":"classes.  The live-migration operation could be used to upgrade the"},{"line_number":191,"context_line":"instance without application downtime."},{"line_number":192,"context_line":""},{"line_number":193,"context_line":"A utility will be provided to identify instances still using the legacy"},{"line_number":194,"context_line":"disk layout so that operators can manage their upgrade."}],"source_content_type":"text/x-rst","patch_set":11,"id":"bacf61ea_501b841c","line":191,"in_reply_to":"bacf61ea_0af9e9c8","updated":"2016-08-03 12:31:28.000000000","message":"As discussed in IRC, I\u0027ll address this by implementing live migration so that old to new converts to new format by updating the domain xml on the target node.  Also will address new to old migrations by preventing use of the new disk format until the all compute nodes are upgraded to the new version.","commit_id":"b8afa59747706ad8317dfa25c782de37c1dda945"},{"author":{"_account_id":6608,"name":"Paul Carlton","email":"paul.carlton2@hpe.com","username":"paul-carlton2"},"change_message_id":"54bd39cca6c2dc43efb46e67ca53b881fd2f6cb0","unresolved":false,"context_lines":[{"line_number":188,"context_line":"mean after upgrading to the P release the live-snapshot operation will"},{"line_number":189,"context_line":"not be available on any instance not recreated using the new backend"},{"line_number":190,"context_line":"classes.  The live-migration operation could be used to upgrade the"},{"line_number":191,"context_line":"instance without application downtime."},{"line_number":192,"context_line":""},{"line_number":193,"context_line":"A utility will be provided to identify instances still using the legacy"},{"line_number":194,"context_line":"disk layout so that operators can manage their upgrade."}],"source_content_type":"text/x-rst","patch_set":11,"id":"bacf61ea_303b70ed","line":191,"in_reply_to":"bacf61ea_2a1dcdbd","updated":"2016-08-03 12:31:28.000000000","message":"Good point, again as discussed in IRC I will ensure that an instance saved (suspend in Nova parlance) on an old node but restarted (nova resume) on a new node or after the node is updated can be restored using the new disk locations","commit_id":"b8afa59747706ad8317dfa25c782de37c1dda945"},{"author":{"_account_id":1779,"name":"Daniel Berrange","email":"berrange@redhat.com","username":"berrange"},"change_message_id":"b776ca170b9b5075e958671469f024c8e07f3769","unresolved":false,"context_lines":[{"line_number":152,"context_line":"For the :code:`disk` XML element in the :code:`domain` element supplied"},{"line_number":153,"context_line":"to libvirt on instance creation, a type of :code:`volume` can be"},{"line_number":154,"context_line":"supplied, with the :code:`\u003csource\u003e` element specifying the pool name"},{"line_number":155,"context_line":"and volume name.OpenStack Ocata to OpenStack P"},{"line_number":156,"context_line":""},{"line_number":157,"context_line":".. [1] Note that the proposed changes assume the prior implementation of"},{"line_number":158,"context_line":"   spec \u0027Persist libvirt instance storage metadata\u0027,"}],"source_content_type":"text/x-rst","patch_set":12,"id":"9ad45d7e_961b2080","line":155,"updated":"2016-08-09 16:12:33.000000000","message":"This last incomplete sentence looks like a mistaken addition","commit_id":"a538fe0d3511c5df42334bdd2858f3ff8f00bff0"},{"author":{"_account_id":1779,"name":"Daniel Berrange","email":"berrange@redhat.com","username":"berrange"},"change_message_id":"b776ca170b9b5075e958671469f024c8e07f3769","unresolved":false,"context_lines":[{"line_number":223,"context_line":"running using the old disk file locations will not be impacted by the"},{"line_number":224,"context_line":"removal of the legacy image backend classes. However if the instance"},{"line_number":225,"context_line":"is rebuilt, hard rebooted, stop/started, suspended/resumed or migrated"},{"line_number":226,"context_line":"it will be converted to the new format."},{"line_number":227,"context_line":""},{"line_number":228,"context_line":"A utility will be provided to identify instances still using the legacy"},{"line_number":229,"context_line":"disk layout so that operators can manage their upgrade to the Q release."}],"source_content_type":"text/x-rst","patch_set":12,"id":"9ad45d7e_ca77ac9d","line":226,"updated":"2016-08-09 16:12:33.000000000","message":"If an instance is using the legacy layout and is on Q where the legacy classes are removed, then is there a chance we\u0027re going to leak files when an instance is deleted ? Or will we still correctly delete instance files despite legacy code being removed.","commit_id":"a538fe0d3511c5df42334bdd2858f3ff8f00bff0"},{"author":{"_account_id":1779,"name":"Daniel Berrange","email":"berrange@redhat.com","username":"berrange"},"change_message_id":"b776ca170b9b5075e958671469f024c8e07f3769","unresolved":false,"context_lines":[{"line_number":230,"context_line":""},{"line_number":231,"context_line":"The live-migration operation could be used to upgrade an instance"},{"line_number":232,"context_line":"without application downtime."},{"line_number":233,"context_line":""},{"line_number":234,"context_line":""},{"line_number":235,"context_line":"Developer impact"},{"line_number":236,"context_line":"----------------"}],"source_content_type":"text/x-rst","patch_set":12,"id":"9ad45d7e_aa4a38c3","line":233,"updated":"2016-08-09 16:12:33.000000000","message":"I think you\u0027ve covered all scenarios here but I find the text a bit hard to follow, so I think it might not be clear to others.\n\nCan we clarify the different scenarios a bit. IIUC, we have this\n\n a) Completely new deployment of Ocata\n     - min Nova RPC API version is set to \"ocata\" immediately\n     - all instances will use new disk layout immediately\n\n b) Upgrade from Newton to Ocata\n     - initially min nova RPC API version is set to \"newton\"\n     - all instances (both new \u0026 pre-existing) will continue to use old disk layout\n     - once all Newton compute nodes are gone, min RPC API version is set to \"ocata\"\n     - new instances will now use the new disk layout\n     - old instances will automatically convert to new disk layout when they are stopped/started, hard rebooted, rebuilt, suspened/resumed, or migrated\n     - when deploying Pxxx, there is continued support for new \u0026 old layouts\n     - when deploying Qxxx, support for old disk layout is removed, but conversion is still possible\n     - any instances still using old disk layout in Qxxx will have certain functionality (live snapshot) blocked until they are converted.","commit_id":"a538fe0d3511c5df42334bdd2858f3ff8f00bff0"}]}