)]}'
{"specs/queens/approved/cinder-volume-multi-attach.rst":[{"author":{"_account_id":782,"name":"John Garbutt","email":"john@johngarbutt.com","username":"johngarbutt"},"change_message_id":"47a02c0b3cce0b0271582c2b39e26d84cf7d959d","unresolved":false,"context_lines":[{"line_number":31,"context_line":"require access to the same volume although only one accesses"},{"line_number":32,"context_line":"actively. When the active one goes down, the passive one can take"},{"line_number":33,"context_line":"over quickly and has access to the data."},{"line_number":34,"context_line":""},{"line_number":35,"context_line":""},{"line_number":36,"context_line":""},{"line_number":37,"context_line":"Proposed change"}],"source_content_type":"text/x-rst","patch_set":1,"id":"7f287b81_5129b234","line":34,"updated":"2017-09-07 17:55:46.000000000","message":"I think we are missing the use case around read-only volumes. Like a set of reference genome data a bunch of servers all want access to.\n\nIts almost like attaching to a cinder snapshot... but that\u0027s probably too strange?","commit_id":"b71ebd3aaa350d785b3d4966e5dd26556b477f69"},{"author":{"_account_id":782,"name":"John Garbutt","email":"john@johngarbutt.com","username":"johngarbutt"},"change_message_id":"61456ddf5d247a4d669eee039bc31e7b667a289b","unresolved":false,"context_lines":[{"line_number":30,"context_line":"with two nodes where one is active and one is passive. Both"},{"line_number":31,"context_line":"require access to the same volume although only one accesses"},{"line_number":32,"context_line":"actively. When the active one goes down, the passive one can take"},{"line_number":33,"context_line":"over quickly and has access to the data."},{"line_number":34,"context_line":""},{"line_number":35,"context_line":""},{"line_number":36,"context_line":""}],"source_content_type":"text/x-rst","patch_set":3,"id":"7f515b1d_eee6dae8","line":33,"updated":"2017-09-25 10:46:31.000000000","message":"Nit: tempting to spit this into two paragraphs one for the read-only case (i.e. efficiently share a data set between multiple nodes) and the read-write case you cover here.","commit_id":"82aae4c2bb44554ae83c71f9124e3a661d4d2d6c"},{"author":{"_account_id":9562,"name":"Ildiko Vancsa","email":"ildiko.vancsa@gmail.com","username":"ildikov"},"change_message_id":"01b070aa2d96f16318b526a13293ba2f793dfb13","unresolved":false,"context_lines":[{"line_number":30,"context_line":"with two nodes where one is active and one is passive. Both"},{"line_number":31,"context_line":"require access to the same volume although only one accesses"},{"line_number":32,"context_line":"actively. When the active one goes down, the passive one can take"},{"line_number":33,"context_line":"over quickly and has access to the data."},{"line_number":34,"context_line":""},{"line_number":35,"context_line":""},{"line_number":36,"context_line":""}],"source_content_type":"text/x-rst","patch_set":3,"id":"7f515b1d_fcc5638f","line":33,"in_reply_to":"7f515b1d_eee6dae8","updated":"2017-09-25 20:14:07.000000000","message":"Done","commit_id":"82aae4c2bb44554ae83c71f9124e3a661d4d2d6c"},{"author":{"_account_id":782,"name":"John Garbutt","email":"john@johngarbutt.com","username":"johngarbutt"},"change_message_id":"61456ddf5d247a4d669eee039bc31e7b667a289b","unresolved":false,"context_lines":[{"line_number":44,"context_line":"Cinder will only allow a volume to be attached more than once if its"},{"line_number":45,"context_line":"\u0027multiattach\u0027 flag is set on the volume at create time. Nova is expected to"},{"line_number":46,"context_line":"rely on Cinder to do the check on the volume state when it\u0027s reserving the"},{"line_number":47,"context_line":"volume on the API level by calling attachment_create. In addition there are"},{"line_number":48,"context_line":"cases where we will not enable \u0027multi-attach\u0027, it is Nova\u0027s responsibility to"},{"line_number":49,"context_line":"disable the functionality and send a proper API response to the caller. Details"},{"line_number":50,"context_line":"are explained below."},{"line_number":51,"context_line":""},{"line_number":52,"context_line":"Nova also needs to know when it can safely disconnect the volume. Cinder"},{"line_number":53,"context_line":"returns a new flag called \u0027uses_shared_connection\u0027 and the back end name to"}],"source_content_type":"text/x-rst","patch_set":3,"id":"7f515b1d_4ed72eef","line":50,"range":{"start_line":47,"start_character":54,"end_line":50,"end_character":20},"updated":"2017-09-25 10:46:31.000000000","message":"This bit I find a little confusing, maybe just delete this bit.","commit_id":"82aae4c2bb44554ae83c71f9124e3a661d4d2d6c"},{"author":{"_account_id":9562,"name":"Ildiko Vancsa","email":"ildiko.vancsa@gmail.com","username":"ildikov"},"change_message_id":"01b070aa2d96f16318b526a13293ba2f793dfb13","unresolved":false,"context_lines":[{"line_number":44,"context_line":"Cinder will only allow a volume to be attached more than once if its"},{"line_number":45,"context_line":"\u0027multiattach\u0027 flag is set on the volume at create time. Nova is expected to"},{"line_number":46,"context_line":"rely on Cinder to do the check on the volume state when it\u0027s reserving the"},{"line_number":47,"context_line":"volume on the API level by calling attachment_create. In addition there are"},{"line_number":48,"context_line":"cases where we will not enable \u0027multi-attach\u0027, it is Nova\u0027s responsibility to"},{"line_number":49,"context_line":"disable the functionality and send a proper API response to the caller. Details"},{"line_number":50,"context_line":"are explained below."},{"line_number":51,"context_line":""},{"line_number":52,"context_line":"Nova also needs to know when it can safely disconnect the volume. Cinder"},{"line_number":53,"context_line":"returns a new flag called \u0027uses_shared_connection\u0027 and the back end name to"}],"source_content_type":"text/x-rst","patch_set":3,"id":"7f515b1d_c8b8abc9","line":50,"range":{"start_line":47,"start_character":54,"end_line":50,"end_character":20},"in_reply_to":"7f515b1d_4ed72eef","updated":"2017-09-25 20:14:07.000000000","message":"Done","commit_id":"82aae4c2bb44554ae83c71f9124e3a661d4d2d6c"},{"author":{"_account_id":782,"name":"John Garbutt","email":"john@johngarbutt.com","username":"johngarbutt"},"change_message_id":"61456ddf5d247a4d669eee039bc31e7b667a289b","unresolved":false,"context_lines":[{"line_number":49,"context_line":"disable the functionality and send a proper API response to the caller. Details"},{"line_number":50,"context_line":"are explained below."},{"line_number":51,"context_line":""},{"line_number":52,"context_line":"Nova also needs to know when it can safely disconnect the volume. Cinder"},{"line_number":53,"context_line":"returns a new flag called \u0027uses_shared_connection\u0027 and the back end name to"},{"line_number":54,"context_line":"indictate whether the back end shares the connections or not. In case it does"},{"line_number":55,"context_line":"we will lock on host level in order to serialize the volume operations on"},{"line_number":56,"context_line":"and prevent from removing the connection in case multiple attachments are using"},{"line_number":57,"context_line":"it. The new information is contained by the volume info. os-brick already"},{"line_number":58,"context_line":"provides the required functionality."},{"line_number":59,"context_line":""},{"line_number":60,"context_line":"By default libvirt assumes all disks are exclusively used by a single guest."},{"line_number":61,"context_line":"If you want to share disks between instances, you need to tell libvirt"}],"source_content_type":"text/x-rst","patch_set":3,"id":"7f515b1d_0e1936d6","line":58,"range":{"start_line":52,"start_character":0,"end_line":58,"end_character":36},"updated":"2017-09-25 10:46:31.000000000","message":"I would turn this upside down, something like:\n\nThere are problems today when multiple volume attachments share a single connection to the volume backend. If we do not take case, multi-attach would make these problems much worse. The simplest fix is to serialize all attach and detach operations involving a shared connection. To do this Cinder will expose a volume info property of \u0027users_shared_connection\u0027, when true a lock will be placed around all attachment_update and attachment_delete calls, and the associated calls to os-brick.\n\nNote: We assume the detach and attach related calls to Cinder are synchronous so there will be no races between os-brick operations on the host and cinder operations on the backend. Any driver deviation from this pattern will be considered a bug.","commit_id":"82aae4c2bb44554ae83c71f9124e3a661d4d2d6c"},{"author":{"_account_id":9562,"name":"Ildiko Vancsa","email":"ildiko.vancsa@gmail.com","username":"ildikov"},"change_message_id":"01b070aa2d96f16318b526a13293ba2f793dfb13","unresolved":false,"context_lines":[{"line_number":49,"context_line":"disable the functionality and send a proper API response to the caller. Details"},{"line_number":50,"context_line":"are explained below."},{"line_number":51,"context_line":""},{"line_number":52,"context_line":"Nova also needs to know when it can safely disconnect the volume. Cinder"},{"line_number":53,"context_line":"returns a new flag called \u0027uses_shared_connection\u0027 and the back end name to"},{"line_number":54,"context_line":"indictate whether the back end shares the connections or not. In case it does"},{"line_number":55,"context_line":"we will lock on host level in order to serialize the volume operations on"},{"line_number":56,"context_line":"and prevent from removing the connection in case multiple attachments are using"},{"line_number":57,"context_line":"it. The new information is contained by the volume info. os-brick already"},{"line_number":58,"context_line":"provides the required functionality."},{"line_number":59,"context_line":""},{"line_number":60,"context_line":"By default libvirt assumes all disks are exclusively used by a single guest."},{"line_number":61,"context_line":"If you want to share disks between instances, you need to tell libvirt"}],"source_content_type":"text/x-rst","patch_set":3,"id":"7f515b1d_5ce0f7cc","line":58,"range":{"start_line":52,"start_character":0,"end_line":58,"end_character":36},"in_reply_to":"7f515b1d_0e1936d6","updated":"2017-09-25 20:14:07.000000000","message":"Done","commit_id":"82aae4c2bb44554ae83c71f9124e3a661d4d2d6c"},{"author":{"_account_id":782,"name":"John Garbutt","email":"john@johngarbutt.com","username":"johngarbutt"},"change_message_id":"61456ddf5d247a4d669eee039bc31e7b667a289b","unresolved":false,"context_lines":[{"line_number":64,"context_line":"lock on the disk, that all I/O caching is disabled, and any SELinux labeling"},{"line_number":65,"context_line":"allows use by all domains."},{"line_number":66,"context_line":""},{"line_number":67,"context_line":"Nova needs to set this \u0027shareable\u0027 flag for the multi-attach disks of the"},{"line_number":68,"context_line":"instances. Only the libvirt driver is modified to support multi-attach, for"},{"line_number":69,"context_line":"all other virt drivers this capability is disabled, the information is stored"},{"line_number":70,"context_line":"among the virt driver capabilities dict in the base ComputeDriver. Nova should"}],"source_content_type":"text/x-rst","patch_set":3,"id":"7f515b1d_4e1e0ebb","line":67,"range":{"start_line":67,"start_character":61,"end_line":67,"end_character":66},"updated":"2017-09-25 10:46:31.000000000","message":"Nit: volumes\n\nI would probably add something about we are looking for mulit-attach \u003d True on the volume, we are not looking to see if there are multiple attachments on a volume. i.e. we have to do this even on the first attachment to the volume.","commit_id":"82aae4c2bb44554ae83c71f9124e3a661d4d2d6c"},{"author":{"_account_id":9562,"name":"Ildiko Vancsa","email":"ildiko.vancsa@gmail.com","username":"ildikov"},"change_message_id":"01b070aa2d96f16318b526a13293ba2f793dfb13","unresolved":false,"context_lines":[{"line_number":64,"context_line":"lock on the disk, that all I/O caching is disabled, and any SELinux labeling"},{"line_number":65,"context_line":"allows use by all domains."},{"line_number":66,"context_line":""},{"line_number":67,"context_line":"Nova needs to set this \u0027shareable\u0027 flag for the multi-attach disks of the"},{"line_number":68,"context_line":"instances. Only the libvirt driver is modified to support multi-attach, for"},{"line_number":69,"context_line":"all other virt drivers this capability is disabled, the information is stored"},{"line_number":70,"context_line":"among the virt driver capabilities dict in the base ComputeDriver. Nova should"}],"source_content_type":"text/x-rst","patch_set":3,"id":"7f515b1d_88007369","line":67,"range":{"start_line":67,"start_character":61,"end_line":67,"end_character":66},"in_reply_to":"7f515b1d_4e1e0ebb","updated":"2017-09-25 20:14:07.000000000","message":"Done","commit_id":"82aae4c2bb44554ae83c71f9124e3a661d4d2d6c"},{"author":{"_account_id":782,"name":"John Garbutt","email":"john@johngarbutt.com","username":"johngarbutt"},"change_message_id":"61456ddf5d247a4d669eee039bc31e7b667a289b","unresolved":false,"context_lines":[{"line_number":71,"context_line":"reject the attach request in case the hypervisor does not support it, but"},{"line_number":72,"context_line":"with the current API it is not possible. This could probably be solved with"},{"line_number":73,"context_line":"policies later on but as a first we will leave it for the computes to fail in"},{"line_number":74,"context_line":"case of not running libvirt."},{"line_number":75,"context_line":""},{"line_number":76,"context_line":"Due to the need to add the \u0027shareable\u0027 flag to the guest xml and further"},{"line_number":77,"context_line":"possible changes in the computes for detach we need to check whether the min"}],"source_content_type":"text/x-rst","patch_set":3,"id":"7f515b1d_2e62b23f","line":74,"updated":"2017-09-25 10:46:31.000000000","message":"I would reword this to:\n\nThis spec will only enable this feature for libvirt, all other drivers should reject attach calls to multi-attach volumes, until that driver adds support, etc...","commit_id":"82aae4c2bb44554ae83c71f9124e3a661d4d2d6c"},{"author":{"_account_id":9562,"name":"Ildiko Vancsa","email":"ildiko.vancsa@gmail.com","username":"ildikov"},"change_message_id":"01b070aa2d96f16318b526a13293ba2f793dfb13","unresolved":false,"context_lines":[{"line_number":71,"context_line":"reject the attach request in case the hypervisor does not support it, but"},{"line_number":72,"context_line":"with the current API it is not possible. This could probably be solved with"},{"line_number":73,"context_line":"policies later on but as a first we will leave it for the computes to fail in"},{"line_number":74,"context_line":"case of not running libvirt."},{"line_number":75,"context_line":""},{"line_number":76,"context_line":"Due to the need to add the \u0027shareable\u0027 flag to the guest xml and further"},{"line_number":77,"context_line":"possible changes in the computes for detach we need to check whether the min"}],"source_content_type":"text/x-rst","patch_set":3,"id":"7f515b1d_9cdf8fd5","line":74,"in_reply_to":"7f515b1d_2e62b23f","updated":"2017-09-25 20:14:07.000000000","message":"Done","commit_id":"82aae4c2bb44554ae83c71f9124e3a661d4d2d6c"},{"author":{"_account_id":782,"name":"John Garbutt","email":"john@johngarbutt.com","username":"johngarbutt"},"change_message_id":"61456ddf5d247a4d669eee039bc31e7b667a289b","unresolved":false,"context_lines":[{"line_number":75,"context_line":""},{"line_number":76,"context_line":"Due to the need to add the \u0027shareable\u0027 flag to the guest xml and further"},{"line_number":77,"context_line":"possible changes in the computes for detach we need to check whether the min"},{"line_number":78,"context_line":"version is high enough to enable multi-attach."},{"line_number":79,"context_line":""},{"line_number":80,"context_line":""},{"line_number":81,"context_line":"Alternatives"}],"source_content_type":"text/x-rst","patch_set":3,"id":"7f515b1d_4e4ccec6","line":78,"updated":"2017-09-25 10:46:31.000000000","message":"Maybe this repeats line 42, not sure. I would add more detail to line 42 rather than mention this twice.","commit_id":"82aae4c2bb44554ae83c71f9124e3a661d4d2d6c"},{"author":{"_account_id":9562,"name":"Ildiko Vancsa","email":"ildiko.vancsa@gmail.com","username":"ildikov"},"change_message_id":"01b070aa2d96f16318b526a13293ba2f793dfb13","unresolved":false,"context_lines":[{"line_number":75,"context_line":""},{"line_number":76,"context_line":"Due to the need to add the \u0027shareable\u0027 flag to the guest xml and further"},{"line_number":77,"context_line":"possible changes in the computes for detach we need to check whether the min"},{"line_number":78,"context_line":"version is high enough to enable multi-attach."},{"line_number":79,"context_line":""},{"line_number":80,"context_line":""},{"line_number":81,"context_line":"Alternatives"}],"source_content_type":"text/x-rst","patch_set":3,"id":"7f515b1d_7cf05368","line":78,"in_reply_to":"7f515b1d_4e4ccec6","updated":"2017-09-25 20:14:07.000000000","message":"Done","commit_id":"82aae4c2bb44554ae83c71f9124e3a661d4d2d6c"},{"author":{"_account_id":782,"name":"John Garbutt","email":"john@johngarbutt.com","username":"johngarbutt"},"change_message_id":"61456ddf5d247a4d669eee039bc31e7b667a289b","unresolved":false,"context_lines":[{"line_number":120,"context_line":"instance termination process. According to this this flag will be allowed to"},{"line_number":121,"context_line":"use along with multi-attach, no changes are necessary when the volume provided"},{"line_number":122,"context_line":"has multiattach\u003dTrue and the delete_on_termination\u003dTrue flag is passed in for"},{"line_number":123,"context_line":"BFV."},{"line_number":124,"context_line":""},{"line_number":125,"context_line":"The second aspect of BFV is the boot process. In this case the only issue"},{"line_number":126,"context_line":"comes with the bootable volumes, which are specified in the boot request as"}],"source_content_type":"text/x-rst","patch_set":3,"id":"7f515b1d_4ee16e9a","line":123,"updated":"2017-09-25 10:46:31.000000000","message":"I would say delete_on_termination\u003dTrue for a multi-attach volume should fail on the boot server API? Would that be easy enough to do?","commit_id":"82aae4c2bb44554ae83c71f9124e3a661d4d2d6c"},{"author":{"_account_id":9562,"name":"Ildiko Vancsa","email":"ildiko.vancsa@gmail.com","username":"ildikov"},"change_message_id":"01b070aa2d96f16318b526a13293ba2f793dfb13","unresolved":false,"context_lines":[{"line_number":120,"context_line":"instance termination process. According to this this flag will be allowed to"},{"line_number":121,"context_line":"use along with multi-attach, no changes are necessary when the volume provided"},{"line_number":122,"context_line":"has multiattach\u003dTrue and the delete_on_termination\u003dTrue flag is passed in for"},{"line_number":123,"context_line":"BFV."},{"line_number":124,"context_line":""},{"line_number":125,"context_line":"The second aspect of BFV is the boot process. In this case the only issue"},{"line_number":126,"context_line":"comes with the bootable volumes, which are specified in the boot request as"}],"source_content_type":"text/x-rst","patch_set":3,"id":"7f515b1d_685f7772","line":123,"in_reply_to":"7f515b1d_4ee16e9a","updated":"2017-09-25 20:14:07.000000000","message":"In my understanding we will not have problem with this option therefore we can allow it safely. So this paragraph is intended to describe why we are not blocking it. I will try to put my head around how to make it more clear.","commit_id":"82aae4c2bb44554ae83c71f9124e3a661d4d2d6c"},{"author":{"_account_id":782,"name":"John Garbutt","email":"john@johngarbutt.com","username":"johngarbutt"},"change_message_id":"80665b9181ce15b0196cc49efb7137a875aa925a","unresolved":false,"context_lines":[{"line_number":32,"context_line":"over quickly and has access to the data."},{"line_number":33,"context_line":""},{"line_number":34,"context_line":"In a read-only case the purpose of multiple attachments is to efficiently share"},{"line_number":35,"context_line":"a data set between multiple nodes for example for running HPC workloads."},{"line_number":36,"context_line":""},{"line_number":37,"context_line":""},{"line_number":38,"context_line":""}],"source_content_type":"text/x-rst","patch_set":4,"id":"5f4e5783_4e9d5964","line":35,"updated":"2017-10-17 13:27:40.000000000","message":"Yeah, lets remove this bit now.","commit_id":"2394dd7ee7ad01943cd065ac721befec06b6cf68"},{"author":{"_account_id":9562,"name":"Ildiko Vancsa","email":"ildiko.vancsa@gmail.com","username":"ildikov"},"change_message_id":"bebff0752233ea0cb289ffbdc0329f9556a46183","unresolved":false,"context_lines":[{"line_number":32,"context_line":"over quickly and has access to the data."},{"line_number":33,"context_line":""},{"line_number":34,"context_line":"In a read-only case the purpose of multiple attachments is to efficiently share"},{"line_number":35,"context_line":"a data set between multiple nodes for example for running HPC workloads."},{"line_number":36,"context_line":""},{"line_number":37,"context_line":""},{"line_number":38,"context_line":""}],"source_content_type":"text/x-rst","patch_set":4,"id":"5f4e5783_b12312a7","line":35,"in_reply_to":"5f4e5783_4e9d5964","updated":"2017-10-17 13:45:41.000000000","message":"Done","commit_id":"2394dd7ee7ad01943cd065ac721befec06b6cf68"},{"author":{"_account_id":9562,"name":"Ildiko Vancsa","email":"ildiko.vancsa@gmail.com","username":"ildikov"},"change_message_id":"167f71351ce07353eeecc6687f2d8ba4a15e0bae","unresolved":false,"context_lines":[{"line_number":126,"context_line":"volume that is attached to the instance when it is deleted. This option does"},{"line_number":127,"context_line":"not cause problem as Cinder takes care of not deleting a volume if it still"},{"line_number":128,"context_line":"has active attachments. Nova will receive an error from Cinder that the volume"},{"line_number":129,"context_line":"deletion failed, which will then be logged [#]_, but will not affect the"},{"line_number":130,"context_line":"instance termination process."},{"line_number":131,"context_line":""},{"line_number":132,"context_line":"The second aspect of BFV is the boot process. In this case the only issue"},{"line_number":133,"context_line":"comes with the bootable volumes, which are specified in the boot request as"}],"source_content_type":"text/x-rst","patch_set":4,"id":"7f515b1d_0c2c7745","line":130,"range":{"start_line":129,"start_character":49,"end_line":130,"end_character":28},"updated":"2017-09-26 11:51:14.000000000","message":"TODO (ildikov): Double check whether the instance goes to \u0027ERROR\u0027 state or not. We might want to block it if it puts the instance into \u0027ERROR\u0027 state.","commit_id":"2394dd7ee7ad01943cd065ac721befec06b6cf68"},{"author":{"_account_id":782,"name":"John Garbutt","email":"john@johngarbutt.com","username":"johngarbutt"},"change_message_id":"80665b9181ce15b0196cc49efb7137a875aa925a","unresolved":false,"context_lines":[{"line_number":126,"context_line":"volume that is attached to the instance when it is deleted. This option does"},{"line_number":127,"context_line":"not cause problem as Cinder takes care of not deleting a volume if it still"},{"line_number":128,"context_line":"has active attachments. Nova will receive an error from Cinder that the volume"},{"line_number":129,"context_line":"deletion failed, which will then be logged [#]_, but will not affect the"},{"line_number":130,"context_line":"instance termination process."},{"line_number":131,"context_line":""},{"line_number":132,"context_line":"The second aspect of BFV is the boot process. In this case the only issue"},{"line_number":133,"context_line":"comes with the bootable volumes, which are specified in the boot request as"}],"source_content_type":"text/x-rst","patch_set":4,"id":"5f4e5783_cee5c9dd","line":130,"range":{"start_line":129,"start_character":49,"end_line":130,"end_character":28},"in_reply_to":"5f4e5783_4e5319e3","updated":"2017-10-17 13:27:40.000000000","message":"I still think its easier to say we don\u0027t allow delete_on_termination \u003d True when multi-attach\u003dTrue on the volume. At least for this initial version.","commit_id":"2394dd7ee7ad01943cd065ac721befec06b6cf68"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"8a44f72d379a06ff0e4b8456af12923a2383fe5c","unresolved":false,"context_lines":[{"line_number":126,"context_line":"volume that is attached to the instance when it is deleted. This option does"},{"line_number":127,"context_line":"not cause problem as Cinder takes care of not deleting a volume if it still"},{"line_number":128,"context_line":"has active attachments. Nova will receive an error from Cinder that the volume"},{"line_number":129,"context_line":"deletion failed, which will then be logged [#]_, but will not affect the"},{"line_number":130,"context_line":"instance termination process."},{"line_number":131,"context_line":""},{"line_number":132,"context_line":"The second aspect of BFV is the boot process. In this case the only issue"},{"line_number":133,"context_line":"comes with the bootable volumes, which are specified in the boot request as"}],"source_content_type":"text/x-rst","patch_set":4,"id":"5f4e5783_144f7493","line":130,"range":{"start_line":129,"start_character":49,"end_line":130,"end_character":28},"in_reply_to":"5f4e5783_91e936e7","updated":"2017-10-17 14:29:44.000000000","message":"_cleanup_running_deleted_instances won\u0027t actually fail, it will handle the exception and log a (confusing) warning:\n\nhttps://github.com/openstack/nova/blob/5e4c98a58f1afeaa903829f5e3f28cd6dc30bf4b/nova/compute/manager.py#L7045\n\nSimilar in the API if we\u0027re doing a local delete (the instance is not on a compute host or the compute host is down):\n\nhttps://github.com/openstack/nova/blob/5e4c98a58f1afeaa903829f5e3f28cd6dc30bf4b/nova/compute/api.py#L2023","commit_id":"2394dd7ee7ad01943cd065ac721befec06b6cf68"},{"author":{"_account_id":782,"name":"John Garbutt","email":"john@johngarbutt.com","username":"johngarbutt"},"change_message_id":"ce4cd27196904bf14a092324adb57a051d2aed1f","unresolved":false,"context_lines":[{"line_number":126,"context_line":"volume that is attached to the instance when it is deleted. This option does"},{"line_number":127,"context_line":"not cause problem as Cinder takes care of not deleting a volume if it still"},{"line_number":128,"context_line":"has active attachments. Nova will receive an error from Cinder that the volume"},{"line_number":129,"context_line":"deletion failed, which will then be logged [#]_, but will not affect the"},{"line_number":130,"context_line":"instance termination process."},{"line_number":131,"context_line":""},{"line_number":132,"context_line":"The second aspect of BFV is the boot process. In this case the only issue"},{"line_number":133,"context_line":"comes with the bootable volumes, which are specified in the boot request as"}],"source_content_type":"text/x-rst","patch_set":4,"id":"5f4e5783_91e936e7","line":130,"range":{"start_line":129,"start_character":49,"end_line":130,"end_character":28},"in_reply_to":"5f4e5783_cee5c9dd","updated":"2017-10-17 13:46:34.000000000","message":"Here we go, I think we can check in the bdm validate near here:\nhttps://github.com/openstack/nova/blob/5e4c98a58f1afeaa903829f5e3f28cd6dc30bf4b/nova/compute/api.py#L1346","commit_id":"2394dd7ee7ad01943cd065ac721befec06b6cf68"},{"author":{"_account_id":9562,"name":"Ildiko Vancsa","email":"ildiko.vancsa@gmail.com","username":"ildikov"},"change_message_id":"fe11ac315854709719ce1a13bc8462f2259008c5","unresolved":false,"context_lines":[{"line_number":126,"context_line":"volume that is attached to the instance when it is deleted. This option does"},{"line_number":127,"context_line":"not cause problem as Cinder takes care of not deleting a volume if it still"},{"line_number":128,"context_line":"has active attachments. Nova will receive an error from Cinder that the volume"},{"line_number":129,"context_line":"deletion failed, which will then be logged [#]_, but will not affect the"},{"line_number":130,"context_line":"instance termination process."},{"line_number":131,"context_line":""},{"line_number":132,"context_line":"The second aspect of BFV is the boot process. In this case the only issue"},{"line_number":133,"context_line":"comes with the bootable volumes, which are specified in the boot request as"}],"source_content_type":"text/x-rst","patch_set":4,"id":"5f4e5783_4e5319e3","line":130,"range":{"start_line":129,"start_character":49,"end_line":130,"end_character":28},"in_reply_to":"7f515b1d_0c2c7745","updated":"2017-10-17 13:21:57.000000000","message":"There is only one case where exception gets raised, which is: _cleanup_running_deleted_instances , that seems like a sacrifice we can take with the first version.","commit_id":"2394dd7ee7ad01943cd065ac721befec06b6cf68"},{"author":{"_account_id":9562,"name":"Ildiko Vancsa","email":"ildiko.vancsa@gmail.com","username":"ildikov"},"change_message_id":"167f71351ce07353eeecc6687f2d8ba4a15e0bae","unresolved":false,"context_lines":[{"line_number":139,"context_line":"from Cinder to check whether the \u0027multiattach\u0027 flag is set to True in which"},{"line_number":140,"context_line":"case we will rely on policies for this operation."},{"line_number":141,"context_line":""},{"line_number":142,"context_line":"The proposed policies are: \u0027multi-attach\u0027, \u0027read-only\u0027 and \u0027read/write\u0027. We"},{"line_number":143,"context_line":"will set the policies as diabled by default."},{"line_number":144,"context_line":""},{"line_number":145,"context_line":"For cases, where Nova creates the volume itself, i.e. source_type is"},{"line_number":146,"context_line":"blank/image/snapshot, it should not enable multi-attach for the volume for now."}],"source_content_type":"text/x-rst","patch_set":4,"id":"7f515b1d_ac9323fa","line":143,"range":{"start_line":142,"start_character":0,"end_line":143,"end_character":43},"updated":"2017-09-26 11:51:14.000000000","message":"How setting \u0027attach_mode\u0027 in Cinder works exactly? How \u0027read-only\u0027 works? Should this be a volume-create policy?","commit_id":"2394dd7ee7ad01943cd065ac721befec06b6cf68"},{"author":{"_account_id":782,"name":"John Garbutt","email":"john@johngarbutt.com","username":"johngarbutt"},"change_message_id":"10f2945d695b7ef1a65b7ca52cdc8414a953b643","unresolved":false,"context_lines":[{"line_number":140,"context_line":"case we will rely on policies for this operation."},{"line_number":141,"context_line":""},{"line_number":142,"context_line":"The proposed policies are: \u0027multi-attach\u0027, \u0027read-only\u0027 and \u0027read/write\u0027. We"},{"line_number":143,"context_line":"will set the policies as diabled by default."},{"line_number":144,"context_line":""},{"line_number":145,"context_line":"For cases, where Nova creates the volume itself, i.e. source_type is"},{"line_number":146,"context_line":"blank/image/snapshot, it should not enable multi-attach for the volume for now."}],"source_content_type":"text/x-rst","patch_set":4,"id":"7f515b1d_2c47337b","line":143,"updated":"2017-09-26 11:52:54.000000000","message":"So looking at the Cinder API I am confused where a read-only policy would be enforced?\n\nI was hoping we would have a policy that on creating a Cinder volume with multi_attach\u003dTrue, Cinder would check its API policy to see if the user was able to do that, given if the volume is marked as read-only or not, and the current roles the user has. Of course that all assumes that read_only is fixed on volume create, and I guess it may not be that simple?","commit_id":"2394dd7ee7ad01943cd065ac721befec06b6cf68"},{"author":{"_account_id":782,"name":"John Garbutt","email":"john@johngarbutt.com","username":"johngarbutt"},"change_message_id":"80665b9181ce15b0196cc49efb7137a875aa925a","unresolved":false,"context_lines":[{"line_number":140,"context_line":"case we will rely on policies for this operation."},{"line_number":141,"context_line":""},{"line_number":142,"context_line":"The proposed policies are: \u0027multi-attach\u0027, \u0027read-only\u0027 and \u0027read/write\u0027. We"},{"line_number":143,"context_line":"will set the policies as diabled by default."},{"line_number":144,"context_line":""},{"line_number":145,"context_line":"For cases, where Nova creates the volume itself, i.e. source_type is"},{"line_number":146,"context_line":"blank/image/snapshot, it should not enable multi-attach for the volume for now."}],"source_content_type":"text/x-rst","patch_set":4,"id":"5f4e5783_8eceb160","line":143,"in_reply_to":"5f4e5783_6ef6d5d9","updated":"2017-10-17 13:27:40.000000000","message":"++","commit_id":"2394dd7ee7ad01943cd065ac721befec06b6cf68"},{"author":{"_account_id":9562,"name":"Ildiko Vancsa","email":"ildiko.vancsa@gmail.com","username":"ildikov"},"change_message_id":"bebff0752233ea0cb289ffbdc0329f9556a46183","unresolved":false,"context_lines":[{"line_number":140,"context_line":"case we will rely on policies for this operation."},{"line_number":141,"context_line":""},{"line_number":142,"context_line":"The proposed policies are: \u0027multi-attach\u0027, \u0027read-only\u0027 and \u0027read/write\u0027. We"},{"line_number":143,"context_line":"will set the policies as diabled by default."},{"line_number":144,"context_line":""},{"line_number":145,"context_line":"For cases, where Nova creates the volume itself, i.e. source_type is"},{"line_number":146,"context_line":"blank/image/snapshot, it should not enable multi-attach for the volume for now."}],"source_content_type":"text/x-rst","patch_set":4,"id":"5f4e5783_d1ca0e9f","line":143,"in_reply_to":"5f4e5783_8eceb160","updated":"2017-10-17 13:45:41.000000000","message":"Done","commit_id":"2394dd7ee7ad01943cd065ac721befec06b6cf68"},{"author":{"_account_id":9562,"name":"Ildiko Vancsa","email":"ildiko.vancsa@gmail.com","username":"ildikov"},"change_message_id":"fe11ac315854709719ce1a13bc8462f2259008c5","unresolved":false,"context_lines":[{"line_number":140,"context_line":"case we will rely on policies for this operation."},{"line_number":141,"context_line":""},{"line_number":142,"context_line":"The proposed policies are: \u0027multi-attach\u0027, \u0027read-only\u0027 and \u0027read/write\u0027. We"},{"line_number":143,"context_line":"will set the policies as diabled by default."},{"line_number":144,"context_line":""},{"line_number":145,"context_line":"For cases, where Nova creates the volume itself, i.e. source_type is"},{"line_number":146,"context_line":"blank/image/snapshot, it should not enable multi-attach for the volume for now."}],"source_content_type":"text/x-rst","patch_set":4,"id":"5f4e5783_6ef6d5d9","line":143,"in_reply_to":"7f515b1d_2c47337b","updated":"2017-10-17 13:21:57.000000000","message":"We agreed on the weekly meeting on nit taking on the R/O case for now and will do that in follow-up enhancements.","commit_id":"2394dd7ee7ad01943cd065ac721befec06b6cf68"},{"author":{"_account_id":11604,"name":"sean mooney","email":"smooney@redhat.com","username":"sean-k-mooney"},"change_message_id":"9121289ba1fae22d62ceac7eee55b64c1f928861","unresolved":false,"context_lines":[{"line_number":25,"context_line":"Use Cases"},{"line_number":26,"context_line":"---------"},{"line_number":27,"context_line":""},{"line_number":28,"context_line":"Allow users to share volumes between multiple guests using read-write"},{"line_number":29,"context_line":"attachments like clustered applications with two nodes where one is active and"},{"line_number":30,"context_line":"one is passive. Both require access to the same volume although only one"},{"line_number":31,"context_line":"accesses actively. When the active one goes down, the passive one can take"},{"line_number":32,"context_line":"over quickly and has access to the data."},{"line_number":33,"context_line":""},{"line_number":34,"context_line":""},{"line_number":35,"context_line":"Proposed change"}],"source_content_type":"text/x-rst","patch_set":5,"id":"5f4e5783_657b9027","line":32,"range":{"start_line":28,"start_character":0,"end_line":32,"end_character":40},"updated":"2017-10-17 16:29:18.000000000","message":"if the volume had a clustered filesystem on it then you could technically support the active active case two. is the intent to allow faster ha fail over in the event the primary vm goes down in cases where the blockdevice is not use a clustered filesystem where concurrent acess could lead to data corruption.\n\nis the intent to support both active active attachment and active passive with this proposal or just the later.","commit_id":"cb8cce4a69fbb79f78b75409f3fffe31ec6744b3"},{"author":{"_account_id":2243,"name":"John Griffith","email":"john.griffith8@gmail.com","username":"john-griffith"},"change_message_id":"be7c463480fe25ee0e372a1d419cd9a7972cde40","unresolved":false,"context_lines":[{"line_number":25,"context_line":"Use Cases"},{"line_number":26,"context_line":"---------"},{"line_number":27,"context_line":""},{"line_number":28,"context_line":"Allow users to share volumes between multiple guests using read-write"},{"line_number":29,"context_line":"attachments like clustered applications with two nodes where one is active and"},{"line_number":30,"context_line":"one is passive. Both require access to the same volume although only one"},{"line_number":31,"context_line":"accesses actively. When the active one goes down, the passive one can take"},{"line_number":32,"context_line":"over quickly and has access to the data."},{"line_number":33,"context_line":""},{"line_number":34,"context_line":""},{"line_number":35,"context_line":"Proposed change"}],"source_content_type":"text/x-rst","patch_set":5,"id":"3f4b6375_7d8e2d79","line":32,"range":{"start_line":28,"start_character":0,"end_line":32,"end_character":40},"in_reply_to":"5f4e5783_657b9027","updated":"2017-10-19 15:58:06.000000000","message":"Done","commit_id":"cb8cce4a69fbb79f78b75409f3fffe31ec6744b3"},{"author":{"_account_id":9562,"name":"Ildiko Vancsa","email":"ildiko.vancsa@gmail.com","username":"ildikov"},"change_message_id":"f9eca5c7bb130959a030a02ca7a9fc6ebaa860d3","unresolved":false,"context_lines":[{"line_number":25,"context_line":"Use Cases"},{"line_number":26,"context_line":"---------"},{"line_number":27,"context_line":""},{"line_number":28,"context_line":"Allow users to share volumes between multiple guests using read-write"},{"line_number":29,"context_line":"attachments like clustered applications with two nodes where one is active and"},{"line_number":30,"context_line":"one is passive. Both require access to the same volume although only one"},{"line_number":31,"context_line":"accesses actively. When the active one goes down, the passive one can take"},{"line_number":32,"context_line":"over quickly and has access to the data."},{"line_number":33,"context_line":""},{"line_number":34,"context_line":""},{"line_number":35,"context_line":"Proposed change"}],"source_content_type":"text/x-rst","patch_set":5,"id":"3f4b6375_88609034","line":32,"range":{"start_line":28,"start_character":0,"end_line":32,"end_character":40},"in_reply_to":"5f4e5783_657b9027","updated":"2017-10-19 17:59:38.000000000","message":"The above use case is more of an example than the only scenario the feature could be used in. I would say that by the end of the day it\u0027s on the user side to decide whether or not they take on the active-active case.\n\nAt least from Cinder/Nova perspective nothing will stop that at this point. It also assumes the user\u0027s responsibility to use a filesystem that suits their intent.","commit_id":"cb8cce4a69fbb79f78b75409f3fffe31ec6744b3"},{"author":{"_account_id":782,"name":"John Garbutt","email":"john@johngarbutt.com","username":"johngarbutt"},"change_message_id":"f51ded907d010d0100ac60000519c342d56450ab","unresolved":false,"context_lines":[{"line_number":36,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"},{"line_number":37,"context_line":""},{"line_number":38,"context_line":"The new \u0027multi-attach\u0027 functionality will be enabled by using the new Cinder"},{"line_number":39,"context_line":"attach/detach API which is available from the API microversion \u003cTODO add the"},{"line_number":40,"context_line":"latest\u003e."},{"line_number":41,"context_line":""},{"line_number":42,"context_line":"Cinder will only allow a volume to be attached more than once if its"},{"line_number":43,"context_line":"\u0027multiattach\u0027 flag is set on the volume at create time. Nova is expected to"}],"source_content_type":"text/x-rst","patch_set":5,"id":"5f4e5783_bfcab99d","line":40,"range":{"start_line":39,"start_character":62,"end_line":40,"end_character":8},"updated":"2017-10-17 14:48:02.000000000","message":"Maybe we just put here the microversion that added the shared_targets flag? and leave that for the cinder code to define once it lands?","commit_id":"cb8cce4a69fbb79f78b75409f3fffe31ec6744b3"},{"author":{"_account_id":2243,"name":"John Griffith","email":"john.griffith8@gmail.com","username":"john-griffith"},"change_message_id":"be7c463480fe25ee0e372a1d419cd9a7972cde40","unresolved":false,"context_lines":[{"line_number":36,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"},{"line_number":37,"context_line":""},{"line_number":38,"context_line":"The new \u0027multi-attach\u0027 functionality will be enabled by using the new Cinder"},{"line_number":39,"context_line":"attach/detach API which is available from the API microversion \u003cTODO add the"},{"line_number":40,"context_line":"latest\u003e."},{"line_number":41,"context_line":""},{"line_number":42,"context_line":"Cinder will only allow a volume to be attached more than once if its"},{"line_number":43,"context_line":"\u0027multiattach\u0027 flag is set on the volume at create time. Nova is expected to"}],"source_content_type":"text/x-rst","patch_set":5,"id":"3f4b6375_1df3b90c","line":40,"range":{"start_line":39,"start_character":62,"end_line":40,"end_character":8},"in_reply_to":"5f4e5783_9feb7d76","updated":"2017-10-19 15:58:06.000000000","message":"Done","commit_id":"cb8cce4a69fbb79f78b75409f3fffe31ec6744b3"},{"author":{"_account_id":9562,"name":"Ildiko Vancsa","email":"ildiko.vancsa@gmail.com","username":"ildikov"},"change_message_id":"f9eca5c7bb130959a030a02ca7a9fc6ebaa860d3","unresolved":false,"context_lines":[{"line_number":36,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"},{"line_number":37,"context_line":""},{"line_number":38,"context_line":"The new \u0027multi-attach\u0027 functionality will be enabled by using the new Cinder"},{"line_number":39,"context_line":"attach/detach API which is available from the API microversion \u003cTODO add the"},{"line_number":40,"context_line":"latest\u003e."},{"line_number":41,"context_line":""},{"line_number":42,"context_line":"Cinder will only allow a volume to be attached more than once if its"},{"line_number":43,"context_line":"\u0027multiattach\u0027 flag is set on the volume at create time. Nova is expected to"}],"source_content_type":"text/x-rst","patch_set":5,"id":"3f4b6375_e894a4ea","line":40,"range":{"start_line":39,"start_character":62,"end_line":40,"end_character":8},"in_reply_to":"5f4e5783_9feb7d76","updated":"2017-10-19 17:59:38.000000000","message":"Done","commit_id":"cb8cce4a69fbb79f78b75409f3fffe31ec6744b3"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"42e6e9a33ecefb89a701abf3050495fbfc87ea7f","unresolved":false,"context_lines":[{"line_number":36,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"},{"line_number":37,"context_line":""},{"line_number":38,"context_line":"The new \u0027multi-attach\u0027 functionality will be enabled by using the new Cinder"},{"line_number":39,"context_line":"attach/detach API which is available from the API microversion \u003cTODO add the"},{"line_number":40,"context_line":"latest\u003e."},{"line_number":41,"context_line":""},{"line_number":42,"context_line":"Cinder will only allow a volume to be attached more than once if its"},{"line_number":43,"context_line":"\u0027multiattach\u0027 flag is set on the volume at create time. Nova is expected to"}],"source_content_type":"text/x-rst","patch_set":5,"id":"5f4e5783_9feb7d76","line":40,"range":{"start_line":39,"start_character":62,"end_line":40,"end_character":8},"in_reply_to":"5f4e5783_bfcab99d","updated":"2017-10-17 15:28:46.000000000","message":"Agree with John.","commit_id":"cb8cce4a69fbb79f78b75409f3fffe31ec6744b3"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"42e6e9a33ecefb89a701abf3050495fbfc87ea7f","unresolved":false,"context_lines":[{"line_number":45,"context_line":"volume on the API level by calling attachment_create."},{"line_number":46,"context_line":""},{"line_number":47,"context_line":"There are problems today when multiple volume attachments share a single"},{"line_number":48,"context_line":"target to the volume backend. If we do not take case, multi-attach would"},{"line_number":49,"context_line":"make these problems much worse. The simplest fix is to serialize all attach and"},{"line_number":50,"context_line":"detach operations involving a shared target. To do this Cinder will expose"},{"line_number":51,"context_line":"a volume info property of \u0027shared_targets\u0027, when True a lock will be"}],"source_content_type":"text/x-rst","patch_set":5,"id":"5f4e5783_7f8b21c2","line":48,"range":{"start_line":48,"start_character":48,"end_line":48,"end_character":52},"updated":"2017-10-17 15:28:46.000000000","message":"care","commit_id":"cb8cce4a69fbb79f78b75409f3fffe31ec6744b3"},{"author":{"_account_id":9562,"name":"Ildiko Vancsa","email":"ildiko.vancsa@gmail.com","username":"ildikov"},"change_message_id":"f9eca5c7bb130959a030a02ca7a9fc6ebaa860d3","unresolved":false,"context_lines":[{"line_number":45,"context_line":"volume on the API level by calling attachment_create."},{"line_number":46,"context_line":""},{"line_number":47,"context_line":"There are problems today when multiple volume attachments share a single"},{"line_number":48,"context_line":"target to the volume backend. If we do not take case, multi-attach would"},{"line_number":49,"context_line":"make these problems much worse. The simplest fix is to serialize all attach and"},{"line_number":50,"context_line":"detach operations involving a shared target. To do this Cinder will expose"},{"line_number":51,"context_line":"a volume info property of \u0027shared_targets\u0027, when True a lock will be"}],"source_content_type":"text/x-rst","patch_set":5,"id":"3f4b6375_a89eac04","line":48,"range":{"start_line":48,"start_character":48,"end_line":48,"end_character":52},"in_reply_to":"5f4e5783_7f8b21c2","updated":"2017-10-19 17:59:38.000000000","message":"Done","commit_id":"cb8cce4a69fbb79f78b75409f3fffe31ec6744b3"},{"author":{"_account_id":2243,"name":"John Griffith","email":"john.griffith8@gmail.com","username":"john-griffith"},"change_message_id":"be7c463480fe25ee0e372a1d419cd9a7972cde40","unresolved":false,"context_lines":[{"line_number":45,"context_line":"volume on the API level by calling attachment_create."},{"line_number":46,"context_line":""},{"line_number":47,"context_line":"There are problems today when multiple volume attachments share a single"},{"line_number":48,"context_line":"target to the volume backend. If we do not take case, multi-attach would"},{"line_number":49,"context_line":"make these problems much worse. The simplest fix is to serialize all attach and"},{"line_number":50,"context_line":"detach operations involving a shared target. To do this Cinder will expose"},{"line_number":51,"context_line":"a volume info property of \u0027shared_targets\u0027, when True a lock will be"}],"source_content_type":"text/x-rst","patch_set":5,"id":"3f4b6375_fdf7bdf4","line":48,"range":{"start_line":48,"start_character":48,"end_line":48,"end_character":52},"in_reply_to":"5f4e5783_7f8b21c2","updated":"2017-10-19 15:58:06.000000000","message":"Done","commit_id":"cb8cce4a69fbb79f78b75409f3fffe31ec6744b3"},{"author":{"_account_id":782,"name":"John Garbutt","email":"john@johngarbutt.com","username":"johngarbutt"},"change_message_id":"f51ded907d010d0100ac60000519c342d56450ab","unresolved":false,"context_lines":[{"line_number":50,"context_line":"detach operations involving a shared target. To do this Cinder will expose"},{"line_number":51,"context_line":"a volume info property of \u0027shared_targets\u0027, when True a lock will be"},{"line_number":52,"context_line":"placed around all attachment_update and attachment_delete calls, and the"},{"line_number":53,"context_line":"associated calls to os-brick."},{"line_number":54,"context_line":""},{"line_number":55,"context_line":".. note::"},{"line_number":56,"context_line":""}],"source_content_type":"text/x-rst","patch_set":5,"id":"5f4e5783_14f3f475","line":53,"updated":"2017-10-17 14:48:02.000000000","message":"Nit: I know what you mean here, but we need a touch more clarity.\n\nMaybe add some pseudo code is the best way forward here, something like this (but with less non-sense names for function calls, ideally):\n\n with optional_host_local_lock(acquire\u003dvolume.shared_target):\n   connector \u003d os-brick.get_connector()\n   conn_info \u003d attachment.update(connector).conn_info\n   os-brick.attach(conn_info)\n   attachment.attach_complete()\n\n with optional_host_local_lock(acquire\u003dvolume.shared_target):\n   conn_info \u003d attachment.start_detaching()\n   os-brick.detach(conn_info)\n   attachment.delete()","commit_id":"cb8cce4a69fbb79f78b75409f3fffe31ec6744b3"},{"author":{"_account_id":9562,"name":"Ildiko Vancsa","email":"ildiko.vancsa@gmail.com","username":"ildikov"},"change_message_id":"f9eca5c7bb130959a030a02ca7a9fc6ebaa860d3","unresolved":false,"context_lines":[{"line_number":50,"context_line":"detach operations involving a shared target. To do this Cinder will expose"},{"line_number":51,"context_line":"a volume info property of \u0027shared_targets\u0027, when True a lock will be"},{"line_number":52,"context_line":"placed around all attachment_update and attachment_delete calls, and the"},{"line_number":53,"context_line":"associated calls to os-brick."},{"line_number":54,"context_line":""},{"line_number":55,"context_line":".. note::"},{"line_number":56,"context_line":""}],"source_content_type":"text/x-rst","patch_set":5,"id":"3f4b6375_b9403c09","line":53,"in_reply_to":"3f4b6375_5748faca","updated":"2017-10-19 17:59:38.000000000","message":"There is already a race condition there which gets I believe almost fixed by this change and it will also sort out our multi-attach needs. As far as I understand when the two attachments are on different hosts we will not have an issue.","commit_id":"cb8cce4a69fbb79f78b75409f3fffe31ec6744b3"},{"author":{"_account_id":782,"name":"John Garbutt","email":"john@johngarbutt.com","username":"johngarbutt"},"change_message_id":"8b1d05b8f4242f244ada2606eb8a1ec330b3ef5a","unresolved":false,"context_lines":[{"line_number":50,"context_line":"detach operations involving a shared target. To do this Cinder will expose"},{"line_number":51,"context_line":"a volume info property of \u0027shared_targets\u0027, when True a lock will be"},{"line_number":52,"context_line":"placed around all attachment_update and attachment_delete calls, and the"},{"line_number":53,"context_line":"associated calls to os-brick."},{"line_number":54,"context_line":""},{"line_number":55,"context_line":".. note::"},{"line_number":56,"context_line":""}],"source_content_type":"text/x-rst","patch_set":5,"id":"5f4e5783_5f6aa519","line":53,"in_reply_to":"5f4e5783_14f3f475","updated":"2017-10-17 14:54:56.000000000","message":"Thinking about it more, not so sure who does the start_detaching, that is probably the API, if so that bit should probably go.","commit_id":"cb8cce4a69fbb79f78b75409f3fffe31ec6744b3"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"42e6e9a33ecefb89a701abf3050495fbfc87ea7f","unresolved":false,"context_lines":[{"line_number":50,"context_line":"detach operations involving a shared target. To do this Cinder will expose"},{"line_number":51,"context_line":"a volume info property of \u0027shared_targets\u0027, when True a lock will be"},{"line_number":52,"context_line":"placed around all attachment_update and attachment_delete calls, and the"},{"line_number":53,"context_line":"associated calls to os-brick."},{"line_number":54,"context_line":""},{"line_number":55,"context_line":".. note::"},{"line_number":56,"context_line":""}],"source_content_type":"text/x-rst","patch_set":5,"id":"5f4e5783_bf44f974","line":53,"in_reply_to":"5f4e5783_5f6aa519","updated":"2017-10-17 15:28:46.000000000","message":"Yeah begin_detaching happens in the API:\n\nhttps://github.com/openstack/nova/blob/master/nova/compute/api.py#L3621\n\nAs for the lock, I know this has come up before, but if the volume is attached to two instances on different compute hosts, the locks aren\u0027t going to do anything, right? Because we\u0027re not using a distributed lock manager.","commit_id":"cb8cce4a69fbb79f78b75409f3fffe31ec6744b3"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"080a6d00eb8323186c8fc1ecadb93df4fcf7b710","unresolved":false,"context_lines":[{"line_number":50,"context_line":"detach operations involving a shared target. To do this Cinder will expose"},{"line_number":51,"context_line":"a volume info property of \u0027shared_targets\u0027, when True a lock will be"},{"line_number":52,"context_line":"placed around all attachment_update and attachment_delete calls, and the"},{"line_number":53,"context_line":"associated calls to os-brick."},{"line_number":54,"context_line":""},{"line_number":55,"context_line":".. note::"},{"line_number":56,"context_line":""}],"source_content_type":"text/x-rst","patch_set":5,"id":"5f4e5783_a86f1829","line":53,"in_reply_to":"5f4e5783_85d32c04","updated":"2017-10-17 19:05:15.000000000","message":"\u003e I believe this host local lock would at least stop us disconnecting a shared connection that just started getting used by some other volume on the same host (be it multi-attach or not). So you do an attach and detach of two volumes that share the same export, before they might race and we end up deleting the shared export just after some other volume started using it.\n\nOK so I\u0027m reading latent potential bug and not something introduced by volume multiattach.","commit_id":"cb8cce4a69fbb79f78b75409f3fffe31ec6744b3"},{"author":{"_account_id":2243,"name":"John Griffith","email":"john.griffith8@gmail.com","username":"john-griffith"},"change_message_id":"be7c463480fe25ee0e372a1d419cd9a7972cde40","unresolved":false,"context_lines":[{"line_number":50,"context_line":"detach operations involving a shared target. To do this Cinder will expose"},{"line_number":51,"context_line":"a volume info property of \u0027shared_targets\u0027, when True a lock will be"},{"line_number":52,"context_line":"placed around all attachment_update and attachment_delete calls, and the"},{"line_number":53,"context_line":"associated calls to os-brick."},{"line_number":54,"context_line":""},{"line_number":55,"context_line":".. note::"},{"line_number":56,"context_line":""}],"source_content_type":"text/x-rst","patch_set":5,"id":"3f4b6375_bdd94584","line":53,"in_reply_to":"5f4e5783_a86f1829","updated":"2017-10-19 15:58:06.000000000","message":"I have another idea to cleaning this up, let me know what you think.","commit_id":"cb8cce4a69fbb79f78b75409f3fffe31ec6744b3"},{"author":{"_account_id":782,"name":"John Garbutt","email":"john@johngarbutt.com","username":"johngarbutt"},"change_message_id":"2686617d68c0778b24ea1d2d03abe7536ed9c3b6","unresolved":false,"context_lines":[{"line_number":50,"context_line":"detach operations involving a shared target. To do this Cinder will expose"},{"line_number":51,"context_line":"a volume info property of \u0027shared_targets\u0027, when True a lock will be"},{"line_number":52,"context_line":"placed around all attachment_update and attachment_delete calls, and the"},{"line_number":53,"context_line":"associated calls to os-brick."},{"line_number":54,"context_line":""},{"line_number":55,"context_line":".. note::"},{"line_number":56,"context_line":""}],"source_content_type":"text/x-rst","patch_set":5,"id":"3f4b6375_5748faca","line":53,"in_reply_to":"5f4e5783_a86f1829","updated":"2017-10-19 08:31:08.000000000","message":"Yeah, I believe that is latent bug before we has all the attachments. Its probably a cinder backend bug at this point.","commit_id":"cb8cce4a69fbb79f78b75409f3fffe31ec6744b3"},{"author":{"_account_id":782,"name":"John Garbutt","email":"john@johngarbutt.com","username":"johngarbutt"},"change_message_id":"e94c7b746fd5b9426ea72104801c81c6c378031d","unresolved":false,"context_lines":[{"line_number":50,"context_line":"detach operations involving a shared target. To do this Cinder will expose"},{"line_number":51,"context_line":"a volume info property of \u0027shared_targets\u0027, when True a lock will be"},{"line_number":52,"context_line":"placed around all attachment_update and attachment_delete calls, and the"},{"line_number":53,"context_line":"associated calls to os-brick."},{"line_number":54,"context_line":""},{"line_number":55,"context_line":".. note::"},{"line_number":56,"context_line":""}],"source_content_type":"text/x-rst","patch_set":5,"id":"5f4e5783_85d32c04","line":53,"in_reply_to":"5f4e5783_bf44f974","updated":"2017-10-17 16:23:34.000000000","message":"This local lock is focusing on a connection shared between multiple volumes to a single host. Not really relating to multi-attach as such.\n\nI believe this host local lock would at least stop us disconnecting a shared connection that just started getting used by some other volume on the same host (be it multi-attach or not). So you do an attach and detach of two volumes that share the same export, before they might race and we end up deleting the shared export just after some other volume started using it.\n\nI think it would also stop us from removing the export in the cinder backend code, as the lock covers the update and remove attachment, that needs to create / remove the shared export as required. Although I think the cinder backend should now have all the info it needs in the attachment records to avoid that problem.\n\nThe only case I can think of in the multi-host attach case is where you have two attachments on different hosts to the same volume, so on some backends they share a single export. Care is needed on when that export is deleted (consider attach, attach, delete, attach, delete, etc). Cinder now has all the state from the attachment records to ensure the export is only removed where there are no other attachment records, and is created again when the next update attachment is sent with a connector, etc. I am not sure though, I asked jungleboyj and smcginnis to explain why this isn\u0027t a problem.","commit_id":"cb8cce4a69fbb79f78b75409f3fffe31ec6744b3"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"42e6e9a33ecefb89a701abf3050495fbfc87ea7f","unresolved":false,"context_lines":[{"line_number":77,"context_line":"version."},{"line_number":78,"context_line":""},{"line_number":79,"context_line":"Nova should reject the attach request in case the hypervisor does not support"},{"line_number":80,"context_line":"it, but with the current API it is not possible. This could probably be solved"},{"line_number":81,"context_line":"with policies later on but as a first we will leave it for the computes to fail"},{"line_number":82,"context_line":"in case of not running libvirt."},{"line_number":83,"context_line":""},{"line_number":84,"context_line":""},{"line_number":85,"context_line":"Alternatives"}],"source_content_type":"text/x-rst","patch_set":5,"id":"5f4e5783_fa2a7f7c","line":82,"range":{"start_line":80,"start_character":49,"end_line":82,"end_character":31},"updated":"2017-10-17 15:28:46.000000000","message":"Cinder should at least have a policy for allowing or blocking creating volumes with multiattach\u003dTrue. That\u0027s something that should happen regardless of this change. Then the block storage admin is the one that is configuring whether or not a user can even get into a mess.","commit_id":"cb8cce4a69fbb79f78b75409f3fffe31ec6744b3"},{"author":{"_account_id":9562,"name":"Ildiko Vancsa","email":"ildiko.vancsa@gmail.com","username":"ildikov"},"change_message_id":"f9eca5c7bb130959a030a02ca7a9fc6ebaa860d3","unresolved":false,"context_lines":[{"line_number":77,"context_line":"version."},{"line_number":78,"context_line":""},{"line_number":79,"context_line":"Nova should reject the attach request in case the hypervisor does not support"},{"line_number":80,"context_line":"it, but with the current API it is not possible. This could probably be solved"},{"line_number":81,"context_line":"with policies later on but as a first we will leave it for the computes to fail"},{"line_number":82,"context_line":"in case of not running libvirt."},{"line_number":83,"context_line":""},{"line_number":84,"context_line":""},{"line_number":85,"context_line":"Alternatives"}],"source_content_type":"text/x-rst","patch_set":5,"id":"3f4b6375_0ea5c104","line":82,"range":{"start_line":80,"start_character":49,"end_line":82,"end_character":31},"in_reply_to":"5f4e5783_00316aa3","updated":"2017-10-19 17:59:38.000000000","message":"Done","commit_id":"cb8cce4a69fbb79f78b75409f3fffe31ec6744b3"},{"author":{"_account_id":782,"name":"John Garbutt","email":"john@johngarbutt.com","username":"johngarbutt"},"change_message_id":"e94c7b746fd5b9426ea72104801c81c6c378031d","unresolved":false,"context_lines":[{"line_number":77,"context_line":"version."},{"line_number":78,"context_line":""},{"line_number":79,"context_line":"Nova should reject the attach request in case the hypervisor does not support"},{"line_number":80,"context_line":"it, but with the current API it is not possible. This could probably be solved"},{"line_number":81,"context_line":"with policies later on but as a first we will leave it for the computes to fail"},{"line_number":82,"context_line":"in case of not running libvirt."},{"line_number":83,"context_line":""},{"line_number":84,"context_line":""},{"line_number":85,"context_line":"Alternatives"}],"source_content_type":"text/x-rst","patch_set":5,"id":"5f4e5783_00316aa3","line":82,"range":{"start_line":80,"start_character":49,"end_line":82,"end_character":31},"in_reply_to":"5f4e5783_fa2a7f7c","updated":"2017-10-17 16:23:34.000000000","message":"Good point, in a cloud where your hypervisior would reject all multi-attach attaches, the operator can simple disable creating multi-attach volumes in Cinder. We should note that here.","commit_id":"cb8cce4a69fbb79f78b75409f3fffe31ec6744b3"},{"author":{"_account_id":782,"name":"John Garbutt","email":"john@johngarbutt.com","username":"johngarbutt"},"change_message_id":"e94c7b746fd5b9426ea72104801c81c6c378031d","unresolved":false,"context_lines":[{"line_number":84,"context_line":""},{"line_number":85,"context_line":"Alternatives"},{"line_number":86,"context_line":"------------"},{"line_number":87,"context_line":""},{"line_number":88,"context_line":"For the use case described above the failover scenario can be handled by"},{"line_number":89,"context_line":"attaching the volume to the passive/standby instance. This means that the"},{"line_number":90,"context_line":"standby instance is not a hot standby anymore as the volume attachment"}],"source_content_type":"text/x-rst","patch_set":5,"id":"5f4e5783_60cede5a","line":87,"updated":"2017-10-17 16:23:34.000000000","message":"Maybe we should add a note that in the future we plan to add read-only volumes in a way you can have multi-attach only with read-only volumes.\n\nAnd as a follow on, once that feature is in we could allow BFV with a multi-attach volume only when the root disk is a read-only volume (or rather Cinder can change its restrictions, once we have the read-only multi-attach concept sorted in Cinder).","commit_id":"cb8cce4a69fbb79f78b75409f3fffe31ec6744b3"},{"author":{"_account_id":9562,"name":"Ildiko Vancsa","email":"ildiko.vancsa@gmail.com","username":"ildikov"},"change_message_id":"f9eca5c7bb130959a030a02ca7a9fc6ebaa860d3","unresolved":false,"context_lines":[{"line_number":84,"context_line":""},{"line_number":85,"context_line":"Alternatives"},{"line_number":86,"context_line":"------------"},{"line_number":87,"context_line":""},{"line_number":88,"context_line":"For the use case described above the failover scenario can be handled by"},{"line_number":89,"context_line":"attaching the volume to the passive/standby instance. This means that the"},{"line_number":90,"context_line":"standby instance is not a hot standby anymore as the volume attachment"}],"source_content_type":"text/x-rst","patch_set":5,"id":"3f4b6375_39e44cac","line":87,"in_reply_to":"5f4e5783_60cede5a","updated":"2017-10-19 17:59:38.000000000","message":"It\u0027s added in the REST API section as it\u0027s not an alternative.","commit_id":"cb8cce4a69fbb79f78b75409f3fffe31ec6744b3"},{"author":{"_account_id":782,"name":"John Garbutt","email":"john@johngarbutt.com","username":"johngarbutt"},"change_message_id":"f51ded907d010d0100ac60000519c342d56450ab","unresolved":false,"context_lines":[{"line_number":112,"context_line":"\u0027volume_snapshot_create\u0027 where we don\u0027t have the instance_uuid to retrieve the"},{"line_number":113,"context_line":"right BDM, therefore we need to disable this call for multi-attach. The API"},{"line_number":114,"context_line":"format for this request is not changed, it is only a protection until the"},{"line_number":115,"context_line":"required API changes to support this request with multi-attach."},{"line_number":116,"context_line":""},{"line_number":117,"context_line":"Another feature that needs further investigation is \u0027boot from volume\u0027 (BFV)."},{"line_number":118,"context_line":"The first aspect of the feature is the \u0027delete_on_termination\u0027 flag, which will"}],"source_content_type":"text/x-rst","patch_set":5,"id":"5f4e5783_bf82b9fe","line":115,"updated":"2017-10-17 14:48:02.000000000","message":"Did we say the swap_volume API is also not supported with multi-attach?","commit_id":"cb8cce4a69fbb79f78b75409f3fffe31ec6744b3"},{"author":{"_account_id":9562,"name":"Ildiko Vancsa","email":"ildiko.vancsa@gmail.com","username":"ildikov"},"change_message_id":"f9eca5c7bb130959a030a02ca7a9fc6ebaa860d3","unresolved":false,"context_lines":[{"line_number":112,"context_line":"\u0027volume_snapshot_create\u0027 where we don\u0027t have the instance_uuid to retrieve the"},{"line_number":113,"context_line":"right BDM, therefore we need to disable this call for multi-attach. The API"},{"line_number":114,"context_line":"format for this request is not changed, it is only a protection until the"},{"line_number":115,"context_line":"required API changes to support this request with multi-attach."},{"line_number":116,"context_line":""},{"line_number":117,"context_line":"Another feature that needs further investigation is \u0027boot from volume\u0027 (BFV)."},{"line_number":118,"context_line":"The first aspect of the feature is the \u0027delete_on_termination\u0027 flag, which will"}],"source_content_type":"text/x-rst","patch_set":5,"id":"3f4b6375_b99efc04","line":115,"in_reply_to":"5f4e5783_7a6c4fcd","updated":"2017-10-19 17:59:38.000000000","message":"I agree with Matt, I think it should be fine, I can hopefully get volunteers to test it out when we have the functionality.","commit_id":"cb8cce4a69fbb79f78b75409f3fffe31ec6744b3"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"42e6e9a33ecefb89a701abf3050495fbfc87ea7f","unresolved":false,"context_lines":[{"line_number":112,"context_line":"\u0027volume_snapshot_create\u0027 where we don\u0027t have the instance_uuid to retrieve the"},{"line_number":113,"context_line":"right BDM, therefore we need to disable this call for multi-attach. The API"},{"line_number":114,"context_line":"format for this request is not changed, it is only a protection until the"},{"line_number":115,"context_line":"required API changes to support this request with multi-attach."},{"line_number":116,"context_line":""},{"line_number":117,"context_line":"Another feature that needs further investigation is \u0027boot from volume\u0027 (BFV)."},{"line_number":118,"context_line":"The first aspect of the feature is the \u0027delete_on_termination\u0027 flag, which will"}],"source_content_type":"text/x-rst","patch_set":5,"id":"5f4e5783_7a6c4fcd","line":115,"in_reply_to":"5f4e5783_bf82b9fe","updated":"2017-10-17 15:28:46.000000000","message":"Hmm, it\u0027s been so long I can\u0027t remember. You\u0027d have to consider:\n\n1. Swapping from a multi-attach volume to a non-multi-attach volume.\n\n2. Swapping from a non-multi-attach volume to a multi-attach volume.\n\nIf we allow attaching a multi-attach volume, which we will, I\u0027m not sure there is much difference in supporting swap volume. And if you don\u0027t support swap volume, you effectively break volume migration / retype for multi-attach volumes on the Cinder side, since Cinder calls Nova to complete the migration / retype.\n\nSo my first inclination is to allow it. There could be dragons I\u0027m not thinking of though.","commit_id":"cb8cce4a69fbb79f78b75409f3fffe31ec6744b3"},{"author":{"_account_id":782,"name":"John Garbutt","email":"john@johngarbutt.com","username":"johngarbutt"},"change_message_id":"f51ded907d010d0100ac60000519c342d56450ab","unresolved":false,"context_lines":[{"line_number":115,"context_line":"required API changes to support this request with multi-attach."},{"line_number":116,"context_line":""},{"line_number":117,"context_line":"Another feature that needs further investigation is \u0027boot from volume\u0027 (BFV)."},{"line_number":118,"context_line":"The first aspect of the feature is the \u0027delete_on_termination\u0027 flag, which will"},{"line_number":119,"context_line":"be allowed to use along with multi-attach, no changes are necessary when the"},{"line_number":120,"context_line":"volume provided has multiattach\u003dTrue and the delete_on_termination\u003dTrue flag is"},{"line_number":121,"context_line":"passed in for BFV. When this flag is set to True it is intended to remove the"}],"source_content_type":"text/x-rst","patch_set":5,"id":"5f4e5783_7fb1e146","line":118,"range":{"start_line":118,"start_character":0,"end_line":118,"end_character":67},"updated":"2017-10-17 14:48:02.000000000","message":"Nit: I think that flag is for any volume, but it makes the most sense for BFV.","commit_id":"cb8cce4a69fbb79f78b75409f3fffe31ec6744b3"},{"author":{"_account_id":9562,"name":"Ildiko Vancsa","email":"ildiko.vancsa@gmail.com","username":"ildikov"},"change_message_id":"f9eca5c7bb130959a030a02ca7a9fc6ebaa860d3","unresolved":false,"context_lines":[{"line_number":115,"context_line":"required API changes to support this request with multi-attach."},{"line_number":116,"context_line":""},{"line_number":117,"context_line":"Another feature that needs further investigation is \u0027boot from volume\u0027 (BFV)."},{"line_number":118,"context_line":"The first aspect of the feature is the \u0027delete_on_termination\u0027 flag, which will"},{"line_number":119,"context_line":"be allowed to use along with multi-attach, no changes are necessary when the"},{"line_number":120,"context_line":"volume provided has multiattach\u003dTrue and the delete_on_termination\u003dTrue flag is"},{"line_number":121,"context_line":"passed in for BFV. When this flag is set to True it is intended to remove the"}],"source_content_type":"text/x-rst","patch_set":5,"id":"3f4b6375_ceae89e6","line":118,"range":{"start_line":118,"start_character":0,"end_line":118,"end_character":67},"in_reply_to":"5f4e5783_401be271","updated":"2017-10-19 17:59:38.000000000","message":"Done","commit_id":"cb8cce4a69fbb79f78b75409f3fffe31ec6744b3"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"42e6e9a33ecefb89a701abf3050495fbfc87ea7f","unresolved":false,"context_lines":[{"line_number":115,"context_line":"required API changes to support this request with multi-attach."},{"line_number":116,"context_line":""},{"line_number":117,"context_line":"Another feature that needs further investigation is \u0027boot from volume\u0027 (BFV)."},{"line_number":118,"context_line":"The first aspect of the feature is the \u0027delete_on_termination\u0027 flag, which will"},{"line_number":119,"context_line":"be allowed to use along with multi-attach, no changes are necessary when the"},{"line_number":120,"context_line":"volume provided has multiattach\u003dTrue and the delete_on_termination\u003dTrue flag is"},{"line_number":121,"context_line":"passed in for BFV. When this flag is set to True it is intended to remove the"}],"source_content_type":"text/x-rst","patch_set":5,"id":"5f4e5783_ba7c0772","line":118,"range":{"start_line":118,"start_character":0,"end_line":118,"end_character":67},"in_reply_to":"5f4e5783_7fb1e146","updated":"2017-10-17 15:28:46.000000000","message":"\u003e Nit: I think that flag is for any volume, but it makes the most\n \u003e sense for BFV.\n\nWhat do you mean the flag is for any volume? You can only pass delete_on_termination when booting from volume in the server create request. It\u0027s in the BDM modeling yes, but defaults to False. So it wouldn\u0027t affect just attaching a volume to a server.","commit_id":"cb8cce4a69fbb79f78b75409f3fffe31ec6744b3"},{"author":{"_account_id":11604,"name":"sean mooney","email":"smooney@redhat.com","username":"sean-k-mooney"},"change_message_id":"9121289ba1fae22d62ceac7eee55b64c1f928861","unresolved":false,"context_lines":[{"line_number":115,"context_line":"required API changes to support this request with multi-attach."},{"line_number":116,"context_line":""},{"line_number":117,"context_line":"Another feature that needs further investigation is \u0027boot from volume\u0027 (BFV)."},{"line_number":118,"context_line":"The first aspect of the feature is the \u0027delete_on_termination\u0027 flag, which will"},{"line_number":119,"context_line":"be allowed to use along with multi-attach, no changes are necessary when the"},{"line_number":120,"context_line":"volume provided has multiattach\u003dTrue and the delete_on_termination\u003dTrue flag is"},{"line_number":121,"context_line":"passed in for BFV. When this flag is set to True it is intended to remove the"}],"source_content_type":"text/x-rst","patch_set":5,"id":"5f4e5783_401be271","line":118,"range":{"start_line":118,"start_character":0,"end_line":118,"end_character":67},"in_reply_to":"5f4e5783_ba7c0772","updated":"2017-10-17 16:29:18.000000000","message":"john were you asserting if you boot form volume and also pass an addtional volume on the commandline that the delete_on_terminate flag should apply to both?\n\ntoday i belive it should apply to root,swap and ephemerial volumes that are created as part of a boot form volumn request but i did not think it applied to addtional volumns that were precreated","commit_id":"cb8cce4a69fbb79f78b75409f3fffe31ec6744b3"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"42e6e9a33ecefb89a701abf3050495fbfc87ea7f","unresolved":false,"context_lines":[{"line_number":122,"context_line":"volume that is attached to the instance when it is deleted. This option does"},{"line_number":123,"context_line":"not cause problem as Cinder takes care of not deleting a volume if it still"},{"line_number":124,"context_line":"has active attachments. Nova will receive an error from Cinder that the volume"},{"line_number":125,"context_line":"deletion failed, which will then be logged [#]_, but will not affect the"},{"line_number":126,"context_line":"instance termination process."},{"line_number":127,"context_line":""},{"line_number":128,"context_line":"The second aspect of BFV is the boot process. In the first iteration we will"}],"source_content_type":"text/x-rst","patch_set":5,"id":"5f4e5783_5adab377","line":125,"range":{"start_line":125,"start_character":42,"end_line":125,"end_character":47},"updated":"2017-10-17 15:28:46.000000000","message":"And in the API here for \"local delete\":\n\nhttps://github.com/openstack/nova/blob/295224c41e7da07c5ddbdafc72ac5abf2d708c69/nova/compute/api.py#L1834","commit_id":"cb8cce4a69fbb79f78b75409f3fffe31ec6744b3"},{"author":{"_account_id":9562,"name":"Ildiko Vancsa","email":"ildiko.vancsa@gmail.com","username":"ildikov"},"change_message_id":"f9eca5c7bb130959a030a02ca7a9fc6ebaa860d3","unresolved":false,"context_lines":[{"line_number":122,"context_line":"volume that is attached to the instance when it is deleted. This option does"},{"line_number":123,"context_line":"not cause problem as Cinder takes care of not deleting a volume if it still"},{"line_number":124,"context_line":"has active attachments. Nova will receive an error from Cinder that the volume"},{"line_number":125,"context_line":"deletion failed, which will then be logged [#]_, but will not affect the"},{"line_number":126,"context_line":"instance termination process."},{"line_number":127,"context_line":""},{"line_number":128,"context_line":"The second aspect of BFV is the boot process. In the first iteration we will"}],"source_content_type":"text/x-rst","patch_set":5,"id":"3f4b6375_994a207e","line":125,"range":{"start_line":125,"start_character":42,"end_line":125,"end_character":47},"in_reply_to":"5f4e5783_5adab377","updated":"2017-10-19 17:59:38.000000000","message":"Done","commit_id":"cb8cce4a69fbb79f78b75409f3fffe31ec6744b3"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"42e6e9a33ecefb89a701abf3050495fbfc87ea7f","unresolved":false,"context_lines":[{"line_number":126,"context_line":"instance termination process."},{"line_number":127,"context_line":""},{"line_number":128,"context_line":"The second aspect of BFV is the boot process. In the first iteration we will"},{"line_number":129,"context_line":"have Read/Write volumes and will not control that through policies. Due to this"},{"line_number":130,"context_line":"aspect we will not enable to multi-attach a bootable volume for which the check"},{"line_number":131,"context_line":"will be on the Cinder side."},{"line_number":132,"context_line":""},{"line_number":133,"context_line":"For cases, where Nova creates the volume itself, i.e. source_type is"},{"line_number":134,"context_line":"blank/image/snapshot, it should not enable multi-attach for the volume for now."}],"source_content_type":"text/x-rst","patch_set":5,"id":"5f4e5783_5ac3d389","line":131,"range":{"start_line":129,"start_character":68,"end_line":131,"end_character":27},"updated":"2017-10-17 15:28:46.000000000","message":"This is a bit unclear to me. Do you mean there will be a policy rule on the cinder side and if so, what is the rule controlling exactly? Just multi-attach volume support in general? Or read/write multi-attach volumes?\n\nOr is this not a policy rule and rather something Cinder checks when nova tries to attach an instance to the volume? Failing there would be bad as that\u0027s down in the compute and is less intuitive for the user that something failed and why.","commit_id":"cb8cce4a69fbb79f78b75409f3fffe31ec6744b3"},{"author":{"_account_id":782,"name":"John Garbutt","email":"john@johngarbutt.com","username":"johngarbutt"},"change_message_id":"e94c7b746fd5b9426ea72104801c81c6c378031d","unresolved":false,"context_lines":[{"line_number":126,"context_line":"instance termination process."},{"line_number":127,"context_line":""},{"line_number":128,"context_line":"The second aspect of BFV is the boot process. In the first iteration we will"},{"line_number":129,"context_line":"have Read/Write volumes and will not control that through policies. Due to this"},{"line_number":130,"context_line":"aspect we will not enable to multi-attach a bootable volume for which the check"},{"line_number":131,"context_line":"will be on the Cinder side."},{"line_number":132,"context_line":""},{"line_number":133,"context_line":"For cases, where Nova creates the volume itself, i.e. source_type is"},{"line_number":134,"context_line":"blank/image/snapshot, it should not enable multi-attach for the volume for now."}],"source_content_type":"text/x-rst","patch_set":5,"id":"5f4e5783_a0de163b","line":131,"range":{"start_line":129,"start_character":68,"end_line":131,"end_character":27},"in_reply_to":"5f4e5783_5ac3d389","updated":"2017-10-17 16:23:34.000000000","message":"I think that is the policy rule in cinder for bootable\u003dTrue being set on a volume, and Nova rejects using a volume as the root disk unless it has that flag set.\n\nWe should make that clearer here.","commit_id":"cb8cce4a69fbb79f78b75409f3fffe31ec6744b3"},{"author":{"_account_id":9562,"name":"Ildiko Vancsa","email":"ildiko.vancsa@gmail.com","username":"ildikov"},"change_message_id":"f9eca5c7bb130959a030a02ca7a9fc6ebaa860d3","unresolved":false,"context_lines":[{"line_number":126,"context_line":"instance termination process."},{"line_number":127,"context_line":""},{"line_number":128,"context_line":"The second aspect of BFV is the boot process. In the first iteration we will"},{"line_number":129,"context_line":"have Read/Write volumes and will not control that through policies. Due to this"},{"line_number":130,"context_line":"aspect we will not enable to multi-attach a bootable volume for which the check"},{"line_number":131,"context_line":"will be on the Cinder side."},{"line_number":132,"context_line":""},{"line_number":133,"context_line":"For cases, where Nova creates the volume itself, i.e. source_type is"},{"line_number":134,"context_line":"blank/image/snapshot, it should not enable multi-attach for the volume for now."}],"source_content_type":"text/x-rst","patch_set":5,"id":"3f4b6375_2e899d74","line":131,"range":{"start_line":129,"start_character":68,"end_line":131,"end_character":27},"in_reply_to":"5f4e5783_a0de163b","updated":"2017-10-19 17:59:38.000000000","message":"Done","commit_id":"cb8cce4a69fbb79f78b75409f3fffe31ec6744b3"},{"author":{"_account_id":782,"name":"John Garbutt","email":"john@johngarbutt.com","username":"johngarbutt"},"change_message_id":"f51ded907d010d0100ac60000519c342d56450ab","unresolved":false,"context_lines":[{"line_number":131,"context_line":"will be on the Cinder side."},{"line_number":132,"context_line":""},{"line_number":133,"context_line":"For cases, where Nova creates the volume itself, i.e. source_type is"},{"line_number":134,"context_line":"blank/image/snapshot, it should not enable multi-attach for the volume for now."},{"line_number":135,"context_line":""},{"line_number":136,"context_line":"When we attach a volume at boot time (BFV with source\u003dvolume,dest\u003dvolume)"},{"line_number":137,"context_line":"scheduling will retry in case of selecting computes that do not support"}],"source_content_type":"text/x-rst","patch_set":5,"id":"5f4e5783_7f768172","line":134,"updated":"2017-10-17 14:48:02.000000000","message":"Nit: i.e. no change to the existing code for now.","commit_id":"cb8cce4a69fbb79f78b75409f3fffe31ec6744b3"},{"author":{"_account_id":9562,"name":"Ildiko Vancsa","email":"ildiko.vancsa@gmail.com","username":"ildikov"},"change_message_id":"f9eca5c7bb130959a030a02ca7a9fc6ebaa860d3","unresolved":false,"context_lines":[{"line_number":131,"context_line":"will be on the Cinder side."},{"line_number":132,"context_line":""},{"line_number":133,"context_line":"For cases, where Nova creates the volume itself, i.e. source_type is"},{"line_number":134,"context_line":"blank/image/snapshot, it should not enable multi-attach for the volume for now."},{"line_number":135,"context_line":""},{"line_number":136,"context_line":"When we attach a volume at boot time (BFV with source\u003dvolume,dest\u003dvolume)"},{"line_number":137,"context_line":"scheduling will retry in case of selecting computes that do not support"}],"source_content_type":"text/x-rst","patch_set":5,"id":"3f4b6375_ae946d8c","line":134,"in_reply_to":"5f4e5783_7f768172","updated":"2017-10-19 17:59:38.000000000","message":"Done","commit_id":"cb8cce4a69fbb79f78b75409f3fffe31ec6744b3"},{"author":{"_account_id":782,"name":"John Garbutt","email":"john@johngarbutt.com","username":"johngarbutt"},"change_message_id":"f51ded907d010d0100ac60000519c342d56450ab","unresolved":false,"context_lines":[{"line_number":134,"context_line":"blank/image/snapshot, it should not enable multi-attach for the volume for now."},{"line_number":135,"context_line":""},{"line_number":136,"context_line":"When we attach a volume at boot time (BFV with source\u003dvolume,dest\u003dvolume)"},{"line_number":137,"context_line":"scheduling will retry in case of selecting computes that do not support"},{"line_number":138,"context_line":"multi-attach. To make it more efficient, later on we can add a new scheduler"},{"line_number":139,"context_line":"filter to avoid the overhead of repeating the scheduling until a valid host is"},{"line_number":140,"context_line":"found. The filter would check the compute capabilities. This step is considered"},{"line_number":141,"context_line":"to be a future improvement."}],"source_content_type":"text/x-rst","patch_set":5,"id":"5f4e5783_dfc1f555","line":138,"range":{"start_line":137,"start_character":33,"end_line":138,"end_character":13},"updated":"2017-10-17 14:48:02.000000000","message":"Lets go for simple here. 80%+ of the time, everything will support multi-attach or everything will not support it.\n\nWe only support multi-attach if all hosts have been upgraded, so that shouldn\u0027t come into it here.\n\nIn the only remaining case, we hit a hypervisor that doesn\u0027t support it, but some other ones do. But in that case, 80% of the time the flavor and/or image will route you to the same hypervisor on the retry.\n\nI would just fail fast here if its not supported. Totally worth noting that we will fail though, but the instance going into the ERROR state.","commit_id":"cb8cce4a69fbb79f78b75409f3fffe31ec6744b3"},{"author":{"_account_id":9562,"name":"Ildiko Vancsa","email":"ildiko.vancsa@gmail.com","username":"ildikov"},"change_message_id":"f9eca5c7bb130959a030a02ca7a9fc6ebaa860d3","unresolved":false,"context_lines":[{"line_number":134,"context_line":"blank/image/snapshot, it should not enable multi-attach for the volume for now."},{"line_number":135,"context_line":""},{"line_number":136,"context_line":"When we attach a volume at boot time (BFV with source\u003dvolume,dest\u003dvolume)"},{"line_number":137,"context_line":"scheduling will retry in case of selecting computes that do not support"},{"line_number":138,"context_line":"multi-attach. To make it more efficient, later on we can add a new scheduler"},{"line_number":139,"context_line":"filter to avoid the overhead of repeating the scheduling until a valid host is"},{"line_number":140,"context_line":"found. The filter would check the compute capabilities. This step is considered"},{"line_number":141,"context_line":"to be a future improvement."}],"source_content_type":"text/x-rst","patch_set":5,"id":"3f4b6375_2e3cfd91","line":138,"range":{"start_line":137,"start_character":33,"end_line":138,"end_character":13},"in_reply_to":"5f4e5783_1a4e7b37","updated":"2017-10-19 17:59:38.000000000","message":"Done","commit_id":"cb8cce4a69fbb79f78b75409f3fffe31ec6744b3"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"42e6e9a33ecefb89a701abf3050495fbfc87ea7f","unresolved":false,"context_lines":[{"line_number":134,"context_line":"blank/image/snapshot, it should not enable multi-attach for the volume for now."},{"line_number":135,"context_line":""},{"line_number":136,"context_line":"When we attach a volume at boot time (BFV with source\u003dvolume,dest\u003dvolume)"},{"line_number":137,"context_line":"scheduling will retry in case of selecting computes that do not support"},{"line_number":138,"context_line":"multi-attach. To make it more efficient, later on we can add a new scheduler"},{"line_number":139,"context_line":"filter to avoid the overhead of repeating the scheduling until a valid host is"},{"line_number":140,"context_line":"found. The filter would check the compute capabilities. This step is considered"},{"line_number":141,"context_line":"to be a future improvement."}],"source_content_type":"text/x-rst","patch_set":5,"id":"5f4e5783_1a4e7b37","line":138,"range":{"start_line":137,"start_character":33,"end_line":138,"end_character":13},"in_reply_to":"5f4e5783_dfc1f555","updated":"2017-10-17 15:28:46.000000000","message":"Agree with John. I think we\u0027d have to consider better scheduling capabilities, maybe with traits, later. For now if you end up on a host that doesn\u0027t support multiattach during boot from volume you fail.\n\nI don\u0027t know how many multi-hypervisor clouds there are out there, but admins could segregate the hosts that support multi-attach using flavor extra specs and host aggregates and then have their users only use certain flavors if they are doing BFV with a multi-attach volume.","commit_id":"cb8cce4a69fbb79f78b75409f3fffe31ec6744b3"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"42e6e9a33ecefb89a701abf3050495fbfc87ea7f","unresolved":false,"context_lines":[{"line_number":137,"context_line":"scheduling will retry in case of selecting computes that do not support"},{"line_number":138,"context_line":"multi-attach. To make it more efficient, later on we can add a new scheduler"},{"line_number":139,"context_line":"filter to avoid the overhead of repeating the scheduling until a valid host is"},{"line_number":140,"context_line":"found. The filter would check the compute capabilities. This step is considered"},{"line_number":141,"context_line":"to be a future improvement."},{"line_number":142,"context_line":""},{"line_number":143,"context_line":"When we enable the feature we will have a \u0027multiattach\u0027 policy to enable or"}],"source_content_type":"text/x-rst","patch_set":5,"id":"5f4e5783_fad5df1c","line":140,"range":{"start_line":140,"start_character":7,"end_line":140,"end_character":55},"updated":"2017-10-17 15:28:46.000000000","message":"I think we can report traits for this from the compute service to Placement, but you still get into a bind with tying the user request to that via flavors, as noted above.\n\nHonestly a scheduler filter is probably easiest and would be easier to implement than retries, plus retries don\u0027t help you with things like evacuate or unshelve where we don\u0027t retry on build failures.","commit_id":"cb8cce4a69fbb79f78b75409f3fffe31ec6744b3"},{"author":{"_account_id":782,"name":"John Garbutt","email":"john@johngarbutt.com","username":"johngarbutt"},"change_message_id":"e94c7b746fd5b9426ea72104801c81c6c378031d","unresolved":false,"context_lines":[{"line_number":137,"context_line":"scheduling will retry in case of selecting computes that do not support"},{"line_number":138,"context_line":"multi-attach. To make it more efficient, later on we can add a new scheduler"},{"line_number":139,"context_line":"filter to avoid the overhead of repeating the scheduling until a valid host is"},{"line_number":140,"context_line":"found. The filter would check the compute capabilities. This step is considered"},{"line_number":141,"context_line":"to be a future improvement."},{"line_number":142,"context_line":""},{"line_number":143,"context_line":"When we enable the feature we will have a \u0027multiattach\u0027 policy to enable or"}],"source_content_type":"text/x-rst","patch_set":5,"id":"5f4e5783_20ea26d7","line":140,"range":{"start_line":140,"start_character":7,"end_line":140,"end_character":55},"in_reply_to":"5f4e5783_fad5df1c","updated":"2017-10-17 16:23:34.000000000","message":"Yeah, lets just fail hard for now.","commit_id":"cb8cce4a69fbb79f78b75409f3fffe31ec6744b3"},{"author":{"_account_id":11604,"name":"sean mooney","email":"smooney@redhat.com","username":"sean-k-mooney"},"change_message_id":"9121289ba1fae22d62ceac7eee55b64c1f928861","unresolved":false,"context_lines":[{"line_number":137,"context_line":"scheduling will retry in case of selecting computes that do not support"},{"line_number":138,"context_line":"multi-attach. To make it more efficient, later on we can add a new scheduler"},{"line_number":139,"context_line":"filter to avoid the overhead of repeating the scheduling until a valid host is"},{"line_number":140,"context_line":"found. The filter would check the compute capabilities. This step is considered"},{"line_number":141,"context_line":"to be a future improvement."},{"line_number":142,"context_line":""},{"line_number":143,"context_line":"When we enable the feature we will have a \u0027multiattach\u0027 policy to enable or"}],"source_content_type":"text/x-rst","patch_set":5,"id":"5f4e5783_c00c7295","line":140,"range":{"start_line":140,"start_character":7,"end_line":140,"end_character":55},"in_reply_to":"5f4e5783_fad5df1c","updated":"2017-10-17 16:29:18.000000000","message":"long term yes traits on the host and request for those traits on the volumn would be the most general approach to this but as you said simple filter is likely the quickest option and most viable in the queens cycle.\n\nat the ptg we talked about being able to pass resource and trait requests in the future form a neutron port but it likely also makes sense to do the same via the glance image or cinder volumn. the inverse is also usefull e.g. describing volumnes and images with traits but again something to consider in the future.","commit_id":"cb8cce4a69fbb79f78b75409f3fffe31ec6744b3"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"42e6e9a33ecefb89a701abf3050495fbfc87ea7f","unresolved":false,"context_lines":[{"line_number":140,"context_line":"found. The filter would check the compute capabilities. This step is considered"},{"line_number":141,"context_line":"to be a future improvement."},{"line_number":142,"context_line":""},{"line_number":143,"context_line":"When we enable the feature we will have a \u0027multiattach\u0027 policy to enable or"},{"line_number":144,"context_line":"disable the operation entirely. Read/Only policy is a future work item and out"},{"line_number":145,"context_line":"of the scope of this spec."},{"line_number":146,"context_line":""},{"line_number":147,"context_line":""}],"source_content_type":"text/x-rst","patch_set":5,"id":"5f4e5783_bab407f3","line":144,"range":{"start_line":143,"start_character":0,"end_line":144,"end_character":32},"updated":"2017-10-17 15:28:46.000000000","message":"Will this be in nova or in cinder? Maybe this belongs in the paragraph on L128 above?","commit_id":"cb8cce4a69fbb79f78b75409f3fffe31ec6744b3"},{"author":{"_account_id":782,"name":"John Garbutt","email":"john@johngarbutt.com","username":"johngarbutt"},"change_message_id":"e94c7b746fd5b9426ea72104801c81c6c378031d","unresolved":false,"context_lines":[{"line_number":140,"context_line":"found. The filter would check the compute capabilities. This step is considered"},{"line_number":141,"context_line":"to be a future improvement."},{"line_number":142,"context_line":""},{"line_number":143,"context_line":"When we enable the feature we will have a \u0027multiattach\u0027 policy to enable or"},{"line_number":144,"context_line":"disable the operation entirely. Read/Only policy is a future work item and out"},{"line_number":145,"context_line":"of the scope of this spec."},{"line_number":146,"context_line":""},{"line_number":147,"context_line":""}],"source_content_type":"text/x-rst","patch_set":5,"id":"5f4e5783_e0c1ae55","line":144,"range":{"start_line":143,"start_character":0,"end_line":144,"end_character":32},"in_reply_to":"5f4e5783_bab407f3","updated":"2017-10-17 16:23:34.000000000","message":"Good point, I assumed that policy is in Cinder, we should define that.","commit_id":"cb8cce4a69fbb79f78b75409f3fffe31ec6744b3"},{"author":{"_account_id":11604,"name":"sean mooney","email":"smooney@redhat.com","username":"sean-k-mooney"},"change_message_id":"9121289ba1fae22d62ceac7eee55b64c1f928861","unresolved":false,"context_lines":[{"line_number":140,"context_line":"found. The filter would check the compute capabilities. This step is considered"},{"line_number":141,"context_line":"to be a future improvement."},{"line_number":142,"context_line":""},{"line_number":143,"context_line":"When we enable the feature we will have a \u0027multiattach\u0027 policy to enable or"},{"line_number":144,"context_line":"disable the operation entirely. Read/Only policy is a future work item and out"},{"line_number":145,"context_line":"of the scope of this spec."},{"line_number":146,"context_line":""},{"line_number":147,"context_line":""}],"source_content_type":"text/x-rst","patch_set":5,"id":"5f4e5783_808b1ac1","line":144,"range":{"start_line":143,"start_character":0,"end_line":144,"end_character":32},"in_reply_to":"5f4e5783_bab407f3","updated":"2017-10-17 16:29:18.000000000","message":"perhaps this is where my first question should have been placed.  this spec covers active/backup read/write sharing.\n\nwill it be possible to add active/active in a singel writer multi reader or multi writer multi reader model in the future.\n\nif so is that a cinder change, a nova change or both.","commit_id":"cb8cce4a69fbb79f78b75409f3fffe31ec6744b3"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"42e6e9a33ecefb89a701abf3050495fbfc87ea7f","unresolved":false,"context_lines":[{"line_number":205,"context_line":"   above listed cases"},{"line_number":206,"context_line":"3. Add Tempest test cases and documentation"},{"line_number":207,"context_line":""},{"line_number":208,"context_line":"Dependencies"},{"line_number":209,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"},{"line_number":210,"context_line":""},{"line_number":211,"context_line":"* This requires the version 3.2.0 or above of the python-cinderclient."}],"source_content_type":"text/x-rst","patch_set":5,"id":"5f4e5783_9a85cb09","line":208,"updated":"2017-10-17 15:28:46.000000000","message":"This is a new dependency:\n\nhttps://specs.openstack.org/openstack/cinder-specs/specs/queens/add-shared-targets-to-volume-ref.html","commit_id":"cb8cce4a69fbb79f78b75409f3fffe31ec6744b3"},{"author":{"_account_id":782,"name":"John Garbutt","email":"john@johngarbutt.com","username":"johngarbutt"},"change_message_id":"e94c7b746fd5b9426ea72104801c81c6c378031d","unresolved":false,"context_lines":[{"line_number":205,"context_line":"   above listed cases"},{"line_number":206,"context_line":"3. Add Tempest test cases and documentation"},{"line_number":207,"context_line":""},{"line_number":208,"context_line":"Dependencies"},{"line_number":209,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"},{"line_number":210,"context_line":""},{"line_number":211,"context_line":"* This requires the version 3.2.0 or above of the python-cinderclient."}],"source_content_type":"text/x-rst","patch_set":5,"id":"5f4e5783_c0b472f4","line":208,"in_reply_to":"5f4e5783_9a85cb09","updated":"2017-10-17 16:23:34.000000000","message":"+1 I forgot about adding that","commit_id":"cb8cce4a69fbb79f78b75409f3fffe31ec6744b3"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"42e6e9a33ecefb89a701abf3050495fbfc87ea7f","unresolved":false,"context_lines":[{"line_number":231,"context_line":"* Encrypted volume with multiattach"},{"line_number":232,"context_line":"* Negative testing:"},{"line_number":233,"context_line":""},{"line_number":234,"context_line":" * Boot from multi-attachable volume with boot_index\u003d0"},{"line_number":235,"context_line":" * Tying to attach a non-multiattach volume to multiple instances"},{"line_number":236,"context_line":""},{"line_number":237,"context_line":"Additionally to the above, Cinder migrate needs to be tested on the gate, as it"}],"source_content_type":"text/x-rst","patch_set":5,"id":"5f4e5783_bab087de","line":234,"range":{"start_line":234,"start_character":3,"end_line":234,"end_character":54},"updated":"2017-10-17 15:28:46.000000000","message":"Why is this not supported? I don\u0027t remember reading about that above.","commit_id":"cb8cce4a69fbb79f78b75409f3fffe31ec6744b3"},{"author":{"_account_id":782,"name":"John Garbutt","email":"john@johngarbutt.com","username":"johngarbutt"},"change_message_id":"e94c7b746fd5b9426ea72104801c81c6c378031d","unresolved":false,"context_lines":[{"line_number":231,"context_line":"* Encrypted volume with multiattach"},{"line_number":232,"context_line":"* Negative testing:"},{"line_number":233,"context_line":""},{"line_number":234,"context_line":" * Boot from multi-attachable volume with boot_index\u003d0"},{"line_number":235,"context_line":" * Tying to attach a non-multiattach volume to multiple instances"},{"line_number":236,"context_line":""},{"line_number":237,"context_line":"Additionally to the above, Cinder migrate needs to be tested on the gate, as it"}],"source_content_type":"text/x-rst","patch_set":5,"id":"5f4e5783_80d2fa26","line":234,"range":{"start_line":234,"start_character":3,"end_line":234,"end_character":54},"in_reply_to":"5f4e5783_bab087de","updated":"2017-10-17 16:23:34.000000000","message":"We agreed not allowing this until we do the read_only volumes.","commit_id":"cb8cce4a69fbb79f78b75409f3fffe31ec6744b3"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"42e6e9a33ecefb89a701abf3050495fbfc87ea7f","unresolved":false,"context_lines":[{"line_number":235,"context_line":" * Tying to attach a non-multiattach volume to multiple instances"},{"line_number":236,"context_line":""},{"line_number":237,"context_line":"Additionally to the above, Cinder migrate needs to be tested on the gate, as it"},{"line_number":238,"context_line":"triggres swap_volume in Nova that is not tested today at all."},{"line_number":239,"context_line":""},{"line_number":240,"context_line":"Documentation Impact"},{"line_number":241,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"}],"source_content_type":"text/x-rst","patch_set":5,"id":"5f4e5783_5aa07390","line":238,"range":{"start_line":238,"start_character":29,"end_line":238,"end_character":61},"updated":"2017-10-17 15:28:46.000000000","message":"This is no longer true. There are some retype tests now in tempest and they run in the gate-tempest-dsvm-neutron-scenario-multinode-lvm-multibackend-ubuntu-xenial-nv job in the experimental queue.","commit_id":"cb8cce4a69fbb79f78b75409f3fffe31ec6744b3"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"42e6e9a33ecefb89a701abf3050495fbfc87ea7f","unresolved":false,"context_lines":[{"line_number":285,"context_line":"     - Re-approved"},{"line_number":286,"context_line":"   * - Mitaka-2"},{"line_number":287,"context_line":"     - Updated with API limitations and testing scenarios"},{"line_number":288,"context_line":"   * - Newton"},{"line_number":289,"context_line":"     - Re-approved"}],"source_content_type":"text/x-rst","patch_set":5,"id":"5f4e5783_fa083f92","line":289,"range":{"start_line":288,"start_character":2,"end_line":289,"end_character":18},"updated":"2017-10-17 15:28:46.000000000","message":"* - Queens\n  - Re-proposed","commit_id":"cb8cce4a69fbb79f78b75409f3fffe31ec6744b3"},{"author":{"_account_id":782,"name":"John Garbutt","email":"john@johngarbutt.com","username":"johngarbutt"},"change_message_id":"ae7889674fc987e6caee5d61a0661d1bb685c46c","unresolved":false,"context_lines":[{"line_number":29,"context_line":"attachments like clustered applications with two nodes where one is active and"},{"line_number":30,"context_line":"one is passive. Both require access to the same volume although only one"},{"line_number":31,"context_line":"accesses actively. When the active one goes down, the passive one can take"},{"line_number":32,"context_line":"over quickly and has access to the data.  Active/Active use cases are also"},{"line_number":33,"context_line":"possible however they are out of scope for the current proposal."},{"line_number":34,"context_line":""},{"line_number":35,"context_line":""},{"line_number":36,"context_line":"Proposed change"}],"source_content_type":"text/x-rst","patch_set":6,"id":"3f4b6375_fd045d7d","line":33,"range":{"start_line":32,"start_character":40,"end_line":33,"end_character":64},"updated":"2017-10-19 16:29:20.000000000","message":"I thought we were just adding one volume in two places, and its up to the application to play nice. It could do Active/Active if it knows how to do that?\n\nAll we know is either could be writing and reading, it does whatever on top?","commit_id":"a9a1641c37b0a4ddd95f512a63dc1c8d9faf2162"},{"author":{"_account_id":17386,"name":"Steve Noyes","email":"steve.noyes@oracle.com","username":"ssn1ssn"},"change_message_id":"79dfb2395915476082715f058d7e8059fca04a80","unresolved":false,"context_lines":[{"line_number":29,"context_line":"attachments like clustered applications with two nodes where one is active and"},{"line_number":30,"context_line":"one is passive. Both require access to the same volume although only one"},{"line_number":31,"context_line":"accesses actively. When the active one goes down, the passive one can take"},{"line_number":32,"context_line":"over quickly and has access to the data.  Active/Active use cases are also"},{"line_number":33,"context_line":"possible however they are out of scope for the current proposal."},{"line_number":34,"context_line":""},{"line_number":35,"context_line":""},{"line_number":36,"context_line":"Proposed change"}],"source_content_type":"text/x-rst","patch_set":6,"id":"3f4b6375_aec8ed54","line":33,"range":{"start_line":32,"start_character":40,"end_line":33,"end_character":64},"in_reply_to":"3f4b6375_604c449e","updated":"2017-10-19 18:16:59.000000000","message":"For Oracle RAC, active/active is required.\n\nWhy does the spec need to distinguish between a/a and a/p? Is the code any different?","commit_id":"a9a1641c37b0a4ddd95f512a63dc1c8d9faf2162"},{"author":{"_account_id":2243,"name":"John Griffith","email":"john.griffith8@gmail.com","username":"john-griffith"},"change_message_id":"a890a1c95eaba488a5dd73cc96903a927ca02064","unresolved":false,"context_lines":[{"line_number":29,"context_line":"attachments like clustered applications with two nodes where one is active and"},{"line_number":30,"context_line":"one is passive. Both require access to the same volume although only one"},{"line_number":31,"context_line":"accesses actively. When the active one goes down, the passive one can take"},{"line_number":32,"context_line":"over quickly and has access to the data.  Active/Active use cases are also"},{"line_number":33,"context_line":"possible however they are out of scope for the current proposal."},{"line_number":34,"context_line":""},{"line_number":35,"context_line":""},{"line_number":36,"context_line":"Proposed change"}],"source_content_type":"text/x-rst","patch_set":6,"id":"3f4b6375_604c449e","line":33,"range":{"start_line":32,"start_character":40,"end_line":33,"end_character":64},"in_reply_to":"3f4b6375_fd045d7d","updated":"2017-10-19 17:02:50.000000000","message":"V5 specifically called out ONLY active/passive and a question was raised about active/active, if that\u0027s in scope awesome, but needs to be mentioned then, my bad.","commit_id":"a9a1641c37b0a4ddd95f512a63dc1c8d9faf2162"},{"author":{"_account_id":782,"name":"John Garbutt","email":"john@johngarbutt.com","username":"johngarbutt"},"change_message_id":"ae7889674fc987e6caee5d61a0661d1bb685c46c","unresolved":false,"context_lines":[{"line_number":39,"context_line":"The new \u0027multi-attach\u0027 functionality will be enabled by using the new Cinder"},{"line_number":40,"context_line":"attach/detach API which is available from the API microversion 3.45."},{"line_number":41,"context_line":""},{"line_number":42,"context_line":"Cinder will only allow a volume to be attached more than once if:"},{"line_number":43,"context_line":"    1. Policy allows it"},{"line_number":44,"context_line":"    2. The backend providing the volume supports it"},{"line_number":45,"context_line":"    3. The volume is NOT already attached to an Instance without being marked"}],"source_content_type":"text/x-rst","patch_set":6,"id":"3f4b6375_fdeb3dad","line":42,"updated":"2017-10-19 16:29:20.000000000","message":"Nit: I think you need a space here for the rst formatting.","commit_id":"a9a1641c37b0a4ddd95f512a63dc1c8d9faf2162"},{"author":{"_account_id":2243,"name":"John Griffith","email":"john.griffith8@gmail.com","username":"john-griffith"},"change_message_id":"a890a1c95eaba488a5dd73cc96903a927ca02064","unresolved":false,"context_lines":[{"line_number":39,"context_line":"The new \u0027multi-attach\u0027 functionality will be enabled by using the new Cinder"},{"line_number":40,"context_line":"attach/detach API which is available from the API microversion 3.45."},{"line_number":41,"context_line":""},{"line_number":42,"context_line":"Cinder will only allow a volume to be attached more than once if:"},{"line_number":43,"context_line":"    1. Policy allows it"},{"line_number":44,"context_line":"    2. The backend providing the volume supports it"},{"line_number":45,"context_line":"    3. The volume is NOT already attached to an Instance without being marked"}],"source_content_type":"text/x-rst","patch_set":6,"id":"3f4b6375_004b5084","line":42,"in_reply_to":"3f4b6375_fdeb3dad","updated":"2017-10-19 17:02:50.000000000","message":"Yep, sorry...  Note I also missed the 4\u0027th check here \n\n4.  The volume does NOT have the bootable flag set to true","commit_id":"a9a1641c37b0a4ddd95f512a63dc1c8d9faf2162"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"feb2bb27da894507531062c9622578c8ff66b68c","unresolved":false,"context_lines":[{"line_number":40,"context_line":"attach/detach API which is available from the API microversion 3.45."},{"line_number":41,"context_line":""},{"line_number":42,"context_line":"Cinder will only allow a volume to be attached more than once if:"},{"line_number":43,"context_line":"    1. Policy allows it"},{"line_number":44,"context_line":"    2. The backend providing the volume supports it"},{"line_number":45,"context_line":"    3. The volume is NOT already attached to an Instance without being marked"},{"line_number":46,"context_line":"       as multi-attach."}],"source_content_type":"text/x-rst","patch_set":6,"id":"3f4b6375_fd9d7d59","line":43,"updated":"2017-10-19 16:28:27.000000000","message":"need a blank line before the list for the numbered list to render properly","commit_id":"a9a1641c37b0a4ddd95f512a63dc1c8d9faf2162"},{"author":{"_account_id":2243,"name":"John Griffith","email":"john.griffith8@gmail.com","username":"john-griffith"},"change_message_id":"a890a1c95eaba488a5dd73cc96903a927ca02064","unresolved":false,"context_lines":[{"line_number":40,"context_line":"attach/detach API which is available from the API microversion 3.45."},{"line_number":41,"context_line":""},{"line_number":42,"context_line":"Cinder will only allow a volume to be attached more than once if:"},{"line_number":43,"context_line":"    1. Policy allows it"},{"line_number":44,"context_line":"    2. The backend providing the volume supports it"},{"line_number":45,"context_line":"    3. The volume is NOT already attached to an Instance without being marked"},{"line_number":46,"context_line":"       as multi-attach."}],"source_content_type":"text/x-rst","patch_set":6,"id":"3f4b6375_a0591c5b","line":43,"in_reply_to":"3f4b6375_fd9d7d59","updated":"2017-10-19 17:02:50.000000000","message":"Yes, sorry, and also see my response to the same comment from John above and the missing 4\u0027th requiremnent of the bootable flag.","commit_id":"a9a1641c37b0a4ddd95f512a63dc1c8d9faf2162"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"feb2bb27da894507531062c9622578c8ff66b68c","unresolved":false,"context_lines":[{"line_number":45,"context_line":"    3. The volume is NOT already attached to an Instance without being marked"},{"line_number":46,"context_line":"       as multi-attach."},{"line_number":47,"context_line":""},{"line_number":48,"context_line":"NOTE:  multi-attach checks are all done at attachment-create time, there is no"},{"line_number":49,"context_line":"longer a requirement to create a volume with multi-attach flags."},{"line_number":50,"context_line":""},{"line_number":51,"context_line":"There are existing challenges with devices that use shared targets,"},{"line_number":52,"context_line":"specifically if a backend provides multiple volumes to Instances on the same"}],"source_content_type":"text/x-rst","patch_set":6,"id":"3f4b6375_3d68b5f3","line":49,"range":{"start_line":48,"start_character":0,"end_line":49,"end_character":64},"updated":"2017-10-19 16:28:27.000000000","message":"This is a big new change in how we\u0027ve always thought this was going to work. In the IRC meeting jgriffith said the volume multiattach flag was going to go away and the attachment record would have the multiattach flag on it, but I don\u0027t get that - the attachment record is per volume/instance combo, the attachment can\u0027t be shared with another instance for that volume. Only the volume can have multiple attachments to different instances.\n\nPlus nova doesn\u0027t know if it should tell cinder to create an attachment in multiattach mode or whatever, especially for the first attachment - we don\u0027t have the user passing anything into the compute API for this, nor do we want to.","commit_id":"a9a1641c37b0a4ddd95f512a63dc1c8d9faf2162"},{"author":{"_account_id":2243,"name":"John Griffith","email":"john.griffith8@gmail.com","username":"john-griffith"},"change_message_id":"a890a1c95eaba488a5dd73cc96903a927ca02064","unresolved":false,"context_lines":[{"line_number":45,"context_line":"    3. The volume is NOT already attached to an Instance without being marked"},{"line_number":46,"context_line":"       as multi-attach."},{"line_number":47,"context_line":""},{"line_number":48,"context_line":"NOTE:  multi-attach checks are all done at attachment-create time, there is no"},{"line_number":49,"context_line":"longer a requirement to create a volume with multi-attach flags."},{"line_number":50,"context_line":""},{"line_number":51,"context_line":"There are existing challenges with devices that use shared targets,"},{"line_number":52,"context_line":"specifically if a backend provides multiple volumes to Instances on the same"}],"source_content_type":"text/x-rst","patch_set":6,"id":"3f4b6375_6031e40b","line":49,"range":{"start_line":48,"start_character":0,"end_line":49,"end_character":64},"in_reply_to":"3f4b6375_3d68b5f3","updated":"2017-10-19 17:02:50.000000000","message":"This is apparantly all on me, this is what I intended with the attachment api\u0027s from the very beginning; it was sort of the entire point.  I thought we were all on the same page, but I obviously blew it because everybody had somethign completely different in mind. :(","commit_id":"a9a1641c37b0a4ddd95f512a63dc1c8d9faf2162"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"feb2bb27da894507531062c9622578c8ff66b68c","unresolved":false,"context_lines":[{"line_number":58,"context_line":"attach/detach processes on a single node for a backend.  The lock is based on"},{"line_number":59,"context_line":"the backend device UUID, so it will serialize operations only on a single"},{"line_number":60,"context_line":"compute node for a specific backend-device.  It\u0027s not necessary to do this"},{"line_number":61,"context_line":"across nodes becuase the targets across nodes are always unique and not shared."},{"line_number":62,"context_line":""},{"line_number":63,"context_line":"Cinder will provide two new attributes to every volume:"},{"line_number":64,"context_line":"   1. shared_targets (boolean)"}],"source_content_type":"text/x-rst","patch_set":6,"id":"3f4b6375_5d4c919d","line":61,"range":{"start_line":61,"start_character":13,"end_line":61,"end_character":20},"updated":"2017-10-19 16:28:27.000000000","message":"because","commit_id":"a9a1641c37b0a4ddd95f512a63dc1c8d9faf2162"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"feb2bb27da894507531062c9622578c8ff66b68c","unresolved":false,"context_lines":[{"line_number":61,"context_line":"across nodes becuase the targets across nodes are always unique and not shared."},{"line_number":62,"context_line":""},{"line_number":63,"context_line":"Cinder will provide two new attributes to every volume:"},{"line_number":64,"context_line":"   1. shared_targets (boolean)"},{"line_number":65,"context_line":"   2. backend_uuid (UUID)"},{"line_number":66,"context_line":""},{"line_number":67,"context_line":"If shared_tagets is True, then that should be a notification to the consumer"}],"source_content_type":"text/x-rst","patch_set":6,"id":"3f4b6375_3d51d537","line":64,"updated":"2017-10-19 16:28:27.000000000","message":"need a blank line for formatting the list","commit_id":"a9a1641c37b0a4ddd95f512a63dc1c8d9faf2162"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"feb2bb27da894507531062c9622578c8ff66b68c","unresolved":false,"context_lines":[{"line_number":94,"context_line":"version."},{"line_number":95,"context_line":""},{"line_number":96,"context_line":"To use multi-attach the only thing that should change in the users workflow is"},{"line_number":97,"context_line":"the addition of a new option to cinder\u0027s attachment-create.  Cinder will accept"},{"line_number":98,"context_line":"a \"--multi-attach\" option (`attachment-create ... --multiattach \u003cvolID\u003e`)."},{"line_number":99,"context_line":""},{"line_number":100,"context_line":"The determination of whether multi-attach should be allowed or not on the"},{"line_number":101,"context_line":"Cinder side should be done at attachment-create time.  When attachment-create"}],"source_content_type":"text/x-rst","patch_set":6,"id":"3f4b6375_5d31310b","line":98,"range":{"start_line":97,"start_character":61,"end_line":98,"end_character":74},"updated":"2017-10-19 16:28:27.000000000","message":"see above - nova doesn\u0027t know when to provide this when creating the attachment, especially the first attachment to an instance","commit_id":"a9a1641c37b0a4ddd95f512a63dc1c8d9faf2162"},{"author":{"_account_id":2243,"name":"John Griffith","email":"john.griffith8@gmail.com","username":"john-griffith"},"change_message_id":"a890a1c95eaba488a5dd73cc96903a927ca02064","unresolved":false,"context_lines":[{"line_number":94,"context_line":"version."},{"line_number":95,"context_line":""},{"line_number":96,"context_line":"To use multi-attach the only thing that should change in the users workflow is"},{"line_number":97,"context_line":"the addition of a new option to cinder\u0027s attachment-create.  Cinder will accept"},{"line_number":98,"context_line":"a \"--multi-attach\" option (`attachment-create ... --multiattach \u003cvolID\u003e`)."},{"line_number":99,"context_line":""},{"line_number":100,"context_line":"The determination of whether multi-attach should be allowed or not on the"},{"line_number":101,"context_line":"Cinder side should be done at attachment-create time.  When attachment-create"}],"source_content_type":"text/x-rst","patch_set":6,"id":"3f4b6375_203beceb","line":98,"range":{"start_line":97,"start_character":61,"end_line":98,"end_character":74},"in_reply_to":"3f4b6375_5d31310b","updated":"2017-10-19 17:02:50.000000000","message":"I\u0027d assumed we\u0027d add an optional param to \"nova volume-attach...\"  I didn\u0027t realize that was prohibitively complex to do.","commit_id":"a9a1641c37b0a4ddd95f512a63dc1c8d9faf2162"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"feb2bb27da894507531062c9622578c8ff66b68c","unresolved":false,"context_lines":[{"line_number":100,"context_line":"The determination of whether multi-attach should be allowed or not on the"},{"line_number":101,"context_line":"Cinder side should be done at attachment-create time.  When attachment-create"},{"line_number":102,"context_line":"is called, Cinder should check the following:"},{"line_number":103,"context_line":"    1. Policy allows multi-attach"},{"line_number":104,"context_line":"    2. Backend supports multi-attach"},{"line_number":105,"context_line":"    3. Volume bootable flag is False"},{"line_number":106,"context_line":"    4. There are NO current attachments for this volume that are NOT marked as"}],"source_content_type":"text/x-rst","patch_set":6,"id":"3f4b6375_1d3bb9eb","line":103,"updated":"2017-10-19 16:28:27.000000000","message":"blank line","commit_id":"a9a1641c37b0a4ddd95f512a63dc1c8d9faf2162"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"feb2bb27da894507531062c9622578c8ff66b68c","unresolved":false,"context_lines":[{"line_number":102,"context_line":"is called, Cinder should check the following:"},{"line_number":103,"context_line":"    1. Policy allows multi-attach"},{"line_number":104,"context_line":"    2. Backend supports multi-attach"},{"line_number":105,"context_line":"    3. Volume bootable flag is False"},{"line_number":106,"context_line":"    4. There are NO current attachments for this volume that are NOT marked as"},{"line_number":107,"context_line":"       multi-attach"},{"line_number":108,"context_line":""}],"source_content_type":"text/x-rst","patch_set":6,"id":"3f4b6375_dd2401c9","line":105,"updated":"2017-10-19 16:28:27.000000000","message":"what about boot from volume with multiattach?","commit_id":"a9a1641c37b0a4ddd95f512a63dc1c8d9faf2162"},{"author":{"_account_id":17386,"name":"Steve Noyes","email":"steve.noyes@oracle.com","username":"ssn1ssn"},"change_message_id":"79dfb2395915476082715f058d7e8059fca04a80","unresolved":false,"context_lines":[{"line_number":102,"context_line":"is called, Cinder should check the following:"},{"line_number":103,"context_line":"    1. Policy allows multi-attach"},{"line_number":104,"context_line":"    2. Backend supports multi-attach"},{"line_number":105,"context_line":"    3. Volume bootable flag is False"},{"line_number":106,"context_line":"    4. There are NO current attachments for this volume that are NOT marked as"},{"line_number":107,"context_line":"       multi-attach"},{"line_number":108,"context_line":""}],"source_content_type":"text/x-rst","patch_set":6,"id":"3f4b6375_8ee411ae","line":105,"in_reply_to":"3f4b6375_c021b8b6","updated":"2017-10-19 18:16:59.000000000","message":"I seem to remember from the PTG that bfv + m-a would eventually be supported, but was a non-goal for queens.","commit_id":"a9a1641c37b0a4ddd95f512a63dc1c8d9faf2162"},{"author":{"_account_id":2243,"name":"John Griffith","email":"john.griffith8@gmail.com","username":"john-griffith"},"change_message_id":"a890a1c95eaba488a5dd73cc96903a927ca02064","unresolved":false,"context_lines":[{"line_number":102,"context_line":"is called, Cinder should check the following:"},{"line_number":103,"context_line":"    1. Policy allows multi-attach"},{"line_number":104,"context_line":"    2. Backend supports multi-attach"},{"line_number":105,"context_line":"    3. Volume bootable flag is False"},{"line_number":106,"context_line":"    4. There are NO current attachments for this volume that are NOT marked as"},{"line_number":107,"context_line":"       multi-attach"},{"line_number":108,"context_line":""}],"source_content_type":"text/x-rst","patch_set":6,"id":"3f4b6375_c021b8b6","line":105,"in_reply_to":"3f4b6375_dd2401c9","updated":"2017-10-19 17:02:50.000000000","message":"We initially said we weren\u0027t going to support that I thought, so that\u0027s what that is (check number 3), but apparantly I\u0027m behind and that\u0027s not the case now?","commit_id":"a9a1641c37b0a4ddd95f512a63dc1c8d9faf2162"},{"author":{"_account_id":17386,"name":"Steve Noyes","email":"steve.noyes@oracle.com","username":"ssn1ssn"},"change_message_id":"79dfb2395915476082715f058d7e8059fca04a80","unresolved":false,"context_lines":[{"line_number":105,"context_line":"    3. Volume bootable flag is False"},{"line_number":106,"context_line":"    4. There are NO current attachments for this volume that are NOT marked as"},{"line_number":107,"context_line":"       multi-attach"},{"line_number":108,"context_line":""},{"line_number":109,"context_line":"If any of the above 4 checks on the Cinder side are not satisfied during the"},{"line_number":110,"context_line":"attachment-create request, the result is a failed attachment-create with the"},{"line_number":111,"context_line":"reason `multi-attach invalid for this volume`.  If all of these conditions are"}],"source_content_type":"text/x-rst","patch_set":6,"id":"3f4b6375_ee882520","line":108,"updated":"2017-10-19 18:16:59.000000000","message":"I think we need another check to ensure that we can\u0027t attach a volume to an instance on which that volume is already attached, regardless of multi-attach.","commit_id":"a9a1641c37b0a4ddd95f512a63dc1c8d9faf2162"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"feb2bb27da894507531062c9622578c8ff66b68c","unresolved":false,"context_lines":[{"line_number":109,"context_line":"If any of the above 4 checks on the Cinder side are not satisfied during the"},{"line_number":110,"context_line":"attachment-create request, the result is a failed attachment-create with the"},{"line_number":111,"context_line":"reason `multi-attach invalid for this volume`.  If all of these conditions are"},{"line_number":112,"context_line":"met, the attachment will be created and returned.  "},{"line_number":113,"context_line":""},{"line_number":114,"context_line":"On subsequent attachments, even if the multi-attach options is not provided"},{"line_number":115,"context_line":"during attachment-create, if Cinder detects that the volume is already attached"}],"source_content_type":"text/x-rst","patch_set":6,"id":"3f4b6375_bd5ca54b","line":112,"range":{"start_line":112,"start_character":49,"end_line":112,"end_character":51},"updated":"2017-10-19 16:28:27.000000000","message":"oops!","commit_id":"a9a1641c37b0a4ddd95f512a63dc1c8d9faf2162"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"feb2bb27da894507531062c9622578c8ff66b68c","unresolved":false,"context_lines":[{"line_number":111,"context_line":"reason `multi-attach invalid for this volume`.  If all of these conditions are"},{"line_number":112,"context_line":"met, the attachment will be created and returned.  "},{"line_number":113,"context_line":""},{"line_number":114,"context_line":"On subsequent attachments, even if the multi-attach options is not provided"},{"line_number":115,"context_line":"during attachment-create, if Cinder detects that the volume is already attached"},{"line_number":116,"context_line":"and multi-attach has been set, it will automatically peform these checks and"},{"line_number":117,"context_line":"set the appropriate settings on the volume and instruct Nova tha this MUST be"}],"source_content_type":"text/x-rst","patch_set":6,"id":"3f4b6375_9d07a91d","line":114,"updated":"2017-10-19 16:28:27.000000000","message":"I don\u0027t see why the user isn\u0027t responsible for creating a volume with the multiattach flag like they are today. If the backend doesn\u0027t support it, fail the volume create (or retype) request. Nova needs to key off the volume.multiattach flag to determine if it should set the shared flag in the libvirt guest xml.","commit_id":"a9a1641c37b0a4ddd95f512a63dc1c8d9faf2162"},{"author":{"_account_id":2243,"name":"John Griffith","email":"john.griffith8@gmail.com","username":"john-griffith"},"change_message_id":"a890a1c95eaba488a5dd73cc96903a927ca02064","unresolved":false,"context_lines":[{"line_number":111,"context_line":"reason `multi-attach invalid for this volume`.  If all of these conditions are"},{"line_number":112,"context_line":"met, the attachment will be created and returned.  "},{"line_number":113,"context_line":""},{"line_number":114,"context_line":"On subsequent attachments, even if the multi-attach options is not provided"},{"line_number":115,"context_line":"during attachment-create, if Cinder detects that the volume is already attached"},{"line_number":116,"context_line":"and multi-attach has been set, it will automatically peform these checks and"},{"line_number":117,"context_line":"set the appropriate settings on the volume and instruct Nova tha this MUST be"}],"source_content_type":"text/x-rst","patch_set":6,"id":"3f4b6375_00191084","line":114,"in_reply_to":"3f4b6375_9d07a91d","updated":"2017-10-19 17:02:50.000000000","message":"IMO it\u0027s better, more flexible and easier to maintain and track by doing this at attach time.  It also alleviates problems with things like retype, live-migrate of volumes etc.","commit_id":"a9a1641c37b0a4ddd95f512a63dc1c8d9faf2162"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"feb2bb27da894507531062c9622578c8ff66b68c","unresolved":false,"context_lines":[{"line_number":113,"context_line":""},{"line_number":114,"context_line":"On subsequent attachments, even if the multi-attach options is not provided"},{"line_number":115,"context_line":"during attachment-create, if Cinder detects that the volume is already attached"},{"line_number":116,"context_line":"and multi-attach has been set, it will automatically peform these checks and"},{"line_number":117,"context_line":"set the appropriate settings on the volume and instruct Nova tha this MUST be"},{"line_number":118,"context_line":"a multi-attach volume for libvirt."},{"line_number":119,"context_line":""}],"source_content_type":"text/x-rst","patch_set":6,"id":"3f4b6375_5d1a517d","line":116,"range":{"start_line":116,"start_character":53,"end_line":116,"end_character":59},"updated":"2017-10-19 16:28:27.000000000","message":"perform","commit_id":"a9a1641c37b0a4ddd95f512a63dc1c8d9faf2162"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"feb2bb27da894507531062c9622578c8ff66b68c","unresolved":false,"context_lines":[{"line_number":114,"context_line":"On subsequent attachments, even if the multi-attach options is not provided"},{"line_number":115,"context_line":"during attachment-create, if Cinder detects that the volume is already attached"},{"line_number":116,"context_line":"and multi-attach has been set, it will automatically peform these checks and"},{"line_number":117,"context_line":"set the appropriate settings on the volume and instruct Nova tha this MUST be"},{"line_number":118,"context_line":"a multi-attach volume for libvirt."},{"line_number":119,"context_line":""},{"line_number":120,"context_line":"The attachment response from Cinder for EVERY attachment-create call will have"}],"source_content_type":"text/x-rst","patch_set":6,"id":"3f4b6375_3d1f956d","line":117,"range":{"start_line":117,"start_character":61,"end_line":117,"end_character":64},"updated":"2017-10-19 16:28:27.000000000","message":"that","commit_id":"a9a1641c37b0a4ddd95f512a63dc1c8d9faf2162"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"feb2bb27da894507531062c9622578c8ff66b68c","unresolved":false,"context_lines":[{"line_number":171,"context_line":"instance termination process."},{"line_number":172,"context_line":""},{"line_number":173,"context_line":"The second aspect of BFV is the boot process. In the first iteration we will"},{"line_number":174,"context_line":"have Read/Write volumes and will not control that through policies. Due to this"},{"line_number":175,"context_line":"aspect we will not enable to multi-attach a bootable volume for which the check"},{"line_number":176,"context_line":"will be on the Cinder side. "},{"line_number":177,"context_line":""},{"line_number":178,"context_line":"When we attach a volume at boot time (BFV with source\u003dvolume,dest\u003dvolume)"},{"line_number":179,"context_line":"scheduling will retry in case of selecting computes that do not support"}],"source_content_type":"text/x-rst","patch_set":6,"id":"3f4b6375_806060d0","line":176,"range":{"start_line":174,"start_character":68,"end_line":176,"end_character":26},"updated":"2017-10-19 16:28:27.000000000","message":"See PS5, should probably clarify this is a policy rule on the cinder side.","commit_id":"a9a1641c37b0a4ddd95f512a63dc1c8d9faf2162"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"feb2bb27da894507531062c9622578c8ff66b68c","unresolved":false,"context_lines":[{"line_number":173,"context_line":"The second aspect of BFV is the boot process. In the first iteration we will"},{"line_number":174,"context_line":"have Read/Write volumes and will not control that through policies. Due to this"},{"line_number":175,"context_line":"aspect we will not enable to multi-attach a bootable volume for which the check"},{"line_number":176,"context_line":"will be on the Cinder side. "},{"line_number":177,"context_line":""},{"line_number":178,"context_line":"When we attach a volume at boot time (BFV with source\u003dvolume,dest\u003dvolume)"},{"line_number":179,"context_line":"scheduling will retry in case of selecting computes that do not support"}],"source_content_type":"text/x-rst","patch_set":6,"id":"3f4b6375_c03d38bc","line":176,"range":{"start_line":176,"start_character":27,"end_line":176,"end_character":28},"updated":"2017-10-19 16:28:27.000000000","message":"whitespace","commit_id":"a9a1641c37b0a4ddd95f512a63dc1c8d9faf2162"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"feb2bb27da894507531062c9622578c8ff66b68c","unresolved":false,"context_lines":[{"line_number":175,"context_line":"aspect we will not enable to multi-attach a bootable volume for which the check"},{"line_number":176,"context_line":"will be on the Cinder side. "},{"line_number":177,"context_line":""},{"line_number":178,"context_line":"When we attach a volume at boot time (BFV with source\u003dvolume,dest\u003dvolume)"},{"line_number":179,"context_line":"scheduling will retry in case of selecting computes that do not support"},{"line_number":180,"context_line":"multi-attach. To make it more efficient, later on we can add a new scheduler"},{"line_number":181,"context_line":"filter to avoid the overhead of repeating the scheduling until a valid host is"},{"line_number":182,"context_line":"found. The filter would check the compute capabilities. This step is considered"},{"line_number":183,"context_line":"to be a future improvement."}],"source_content_type":"text/x-rst","patch_set":6,"id":"3f4b6375_e0aef4e5","line":180,"range":{"start_line":178,"start_character":0,"end_line":180,"end_character":14},"updated":"2017-10-19 16:28:27.000000000","message":"In PS5 we said we wouldn\u0027t retry on a failed boot from volume with multiattach.","commit_id":"a9a1641c37b0a4ddd95f512a63dc1c8d9faf2162"},{"author":{"_account_id":782,"name":"John Garbutt","email":"john@johngarbutt.com","username":"johngarbutt"},"change_message_id":"ae7889674fc987e6caee5d61a0661d1bb685c46c","unresolved":false,"context_lines":[{"line_number":185,"context_line":"When we enable the feature we will have a \u0027multiattach\u0027 policy to enable or"},{"line_number":186,"context_line":"disable the operation entirely. Read/Only policy is a future work item and out"},{"line_number":187,"context_line":"of the scope of this spec."},{"line_number":188,"context_line":""},{"line_number":189,"context_line":""},{"line_number":190,"context_line":"Security impact"},{"line_number":191,"context_line":"---------------"}],"source_content_type":"text/x-rst","patch_set":6,"id":"3f4b6375_1d14d94d","line":188,"updated":"2017-10-19 16:29:20.000000000","message":"Above you say we pass a multi-attach flag into cinder, that means we change BDM on boot and attach volume APIs to have that additional flag. That all needs detailing here.","commit_id":"a9a1641c37b0a4ddd95f512a63dc1c8d9faf2162"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"feb2bb27da894507531062c9622578c8ff66b68c","unresolved":false,"context_lines":[{"line_number":247,"context_line":"   above listed cases"},{"line_number":248,"context_line":"3. Add Tempest test cases and documentation"},{"line_number":249,"context_line":""},{"line_number":250,"context_line":"Dependencies"},{"line_number":251,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"},{"line_number":252,"context_line":""},{"line_number":253,"context_line":"* This requires the version 3.2.0 or above of the python-cinderclient."}],"source_content_type":"text/x-rst","patch_set":6,"id":"3f4b6375_207e4c6b","line":250,"updated":"2017-10-19 16:28:27.000000000","message":"This is a new dependency:\n\nhttps://specs.openstack.org/openstack/cinder-specs/specs/queens/add-shared-targets-to-volume-ref.html","commit_id":"a9a1641c37b0a4ddd95f512a63dc1c8d9faf2162"},{"author":{"_account_id":782,"name":"John Garbutt","email":"john@johngarbutt.com","username":"johngarbutt"},"change_message_id":"ae7889674fc987e6caee5d61a0661d1bb685c46c","unresolved":false,"context_lines":[{"line_number":248,"context_line":"3. Add Tempest test cases and documentation"},{"line_number":249,"context_line":""},{"line_number":250,"context_line":"Dependencies"},{"line_number":251,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"},{"line_number":252,"context_line":""},{"line_number":253,"context_line":"* This requires the version 3.2.0 or above of the python-cinderclient."},{"line_number":254,"context_line":"  Corresponding blueprint:"}],"source_content_type":"text/x-rst","patch_set":6,"id":"3f4b6375_e0403438","line":251,"updated":"2017-10-19 16:29:20.000000000","message":"matt\u0027s comments need fixing here still.","commit_id":"a9a1641c37b0a4ddd95f512a63dc1c8d9faf2162"},{"author":{"_account_id":782,"name":"John Garbutt","email":"john@johngarbutt.com","username":"johngarbutt"},"change_message_id":"ae7889674fc987e6caee5d61a0661d1bb685c46c","unresolved":false,"context_lines":[{"line_number":273,"context_line":"* Encrypted volume with multiattach"},{"line_number":274,"context_line":"* Negative testing:"},{"line_number":275,"context_line":""},{"line_number":276,"context_line":" * Boot from multi-attachable volume with boot_index\u003d0"},{"line_number":277,"context_line":" * Tying to attach a non-multiattach volume to multiple instances"},{"line_number":278,"context_line":""},{"line_number":279,"context_line":"Additionally to the above, Cinder migrate needs to be tested on the gate, as it"}],"source_content_type":"text/x-rst","patch_set":6,"id":"3f4b6375_8047404c","line":276,"updated":"2017-10-19 16:29:20.000000000","message":"need this fixing still.","commit_id":"a9a1641c37b0a4ddd95f512a63dc1c8d9faf2162"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"feb2bb27da894507531062c9622578c8ff66b68c","unresolved":false,"context_lines":[{"line_number":277,"context_line":" * Tying to attach a non-multiattach volume to multiple instances"},{"line_number":278,"context_line":""},{"line_number":279,"context_line":"Additionally to the above, Cinder migrate needs to be tested on the gate, as it"},{"line_number":280,"context_line":"triggres swap_volume in Nova that is not tested today at all."},{"line_number":281,"context_line":""},{"line_number":282,"context_line":"Documentation Impact"},{"line_number":283,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"}],"source_content_type":"text/x-rst","patch_set":6,"id":"3f4b6375_00835055","line":280,"range":{"start_line":280,"start_character":34,"end_line":280,"end_character":61},"updated":"2017-10-19 16:28:27.000000000","message":"This is no longer true. There are some retype tests now in tempest and they run in the gate-tempest-dsvm-neutron-scenario-multinode-lvm-multibackend-ubuntu-xenial-nv job in the experimental queue.","commit_id":"a9a1641c37b0a4ddd95f512a63dc1c8d9faf2162"},{"author":{"_account_id":782,"name":"John Garbutt","email":"john@johngarbutt.com","username":"johngarbutt"},"change_message_id":"ae7889674fc987e6caee5d61a0661d1bb685c46c","unresolved":false,"context_lines":[{"line_number":277,"context_line":" * Tying to attach a non-multiattach volume to multiple instances"},{"line_number":278,"context_line":""},{"line_number":279,"context_line":"Additionally to the above, Cinder migrate needs to be tested on the gate, as it"},{"line_number":280,"context_line":"triggres swap_volume in Nova that is not tested today at all."},{"line_number":281,"context_line":""},{"line_number":282,"context_line":"Documentation Impact"},{"line_number":283,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"}],"source_content_type":"text/x-rst","patch_set":6,"id":"3f4b6375_00759095","line":280,"updated":"2017-10-19 16:29:20.000000000","message":"need this fixing still.","commit_id":"a9a1641c37b0a4ddd95f512a63dc1c8d9faf2162"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"feb2bb27da894507531062c9622578c8ff66b68c","unresolved":false,"context_lines":[{"line_number":327,"context_line":"     - Re-approved"},{"line_number":328,"context_line":"   * - Mitaka-2"},{"line_number":329,"context_line":"     - Updated with API limitations and testing scenarios"},{"line_number":330,"context_line":"   * - Newton"},{"line_number":331,"context_line":"     - Re-approved"}],"source_content_type":"text/x-rst","patch_set":6,"id":"3f4b6375_c094188e","line":330,"range":{"start_line":330,"start_character":7,"end_line":330,"end_character":13},"updated":"2017-10-19 16:28:27.000000000","message":"Append:\n\n* - Queens\n  - Re-proposed","commit_id":"a9a1641c37b0a4ddd95f512a63dc1c8d9faf2162"},{"author":{"_account_id":782,"name":"John Garbutt","email":"john@johngarbutt.com","username":"johngarbutt"},"change_message_id":"ae7889674fc987e6caee5d61a0661d1bb685c46c","unresolved":false,"context_lines":[{"line_number":328,"context_line":"   * - Mitaka-2"},{"line_number":329,"context_line":"     - Updated with API limitations and testing scenarios"},{"line_number":330,"context_line":"   * - Newton"},{"line_number":331,"context_line":"     - Re-approved"}],"source_content_type":"text/x-rst","patch_set":6,"id":"3f4b6375_40526815","line":331,"updated":"2017-10-19 16:29:20.000000000","message":"Matt\u0027s comment needs fixing here.","commit_id":"a9a1641c37b0a4ddd95f512a63dc1c8d9faf2162"},{"author":{"_account_id":17386,"name":"Steve Noyes","email":"steve.noyes@oracle.com","username":"ssn1ssn"},"change_message_id":"28f4dc86a84dc98f96fb7038fc05368231a903e0","unresolved":false,"context_lines":[{"line_number":44,"context_line":"Cinder will only allow a volume to be attached more than once if its"},{"line_number":45,"context_line":"\u0027multiattach\u0027 flag is set on the volume at create time. Nova is expected to"},{"line_number":46,"context_line":"rely on Cinder to do the check on the volume state when it\u0027s reserving the"},{"line_number":47,"context_line":"volume on the API level by calling attachment_create."},{"line_number":48,"context_line":""},{"line_number":49,"context_line":"There are problems today when multiple volume attachments share a single"},{"line_number":50,"context_line":"target to the volume backend. If we do not take care, multi-attach would"}],"source_content_type":"text/x-rst","patch_set":7,"id":"3f4b6375_69e30f35","line":47,"updated":"2017-10-19 18:27:42.000000000","message":"We should also disallow connecting a volume more than once to the same instance even if its multiattach flag is set. It creates subtle issues in libvirt when this happens.","commit_id":"d287d4504bad22a8fa1f8a29b81a86376bfebde6"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"37d8959d2ecadb7c4fa09b46bd56a18fbb004cf1","unresolved":false,"context_lines":[{"line_number":44,"context_line":"Cinder will only allow a volume to be attached more than once if its"},{"line_number":45,"context_line":"\u0027multiattach\u0027 flag is set on the volume at create time. Nova is expected to"},{"line_number":46,"context_line":"rely on Cinder to do the check on the volume state when it\u0027s reserving the"},{"line_number":47,"context_line":"volume on the API level by calling attachment_create."},{"line_number":48,"context_line":""},{"line_number":49,"context_line":"There are problems today when multiple volume attachments share a single"},{"line_number":50,"context_line":"target to the volume backend. If we do not take care, multi-attach would"}],"source_content_type":"text/x-rst","patch_set":7,"id":"3f4b6375_b4527a66","line":47,"in_reply_to":"3f4b6375_69e30f35","updated":"2017-10-19 19:43:27.000000000","message":"Hmm, good point, I wonder what happens with this today? There is no unique constraint on the nova.block_device_mappings table so you could have more than one BDM record for the same volume and instance. What happens on the Cinder side if you try to create an attachment for the same volume and instance? There is no unique constraint between volume and instance in the cinder.attachment table either...","commit_id":"d287d4504bad22a8fa1f8a29b81a86376bfebde6"},{"author":{"_account_id":17386,"name":"Steve Noyes","email":"steve.noyes@oracle.com","username":"ssn1ssn"},"change_message_id":"27d1f63d4dfaa268e439faf3f819e3cae539c041","unresolved":false,"context_lines":[{"line_number":44,"context_line":"Cinder will only allow a volume to be attached more than once if its"},{"line_number":45,"context_line":"\u0027multiattach\u0027 flag is set on the volume at create time. Nova is expected to"},{"line_number":46,"context_line":"rely on Cinder to do the check on the volume state when it\u0027s reserving the"},{"line_number":47,"context_line":"volume on the API level by calling attachment_create."},{"line_number":48,"context_line":""},{"line_number":49,"context_line":"There are problems today when multiple volume attachments share a single"},{"line_number":50,"context_line":"target to the volume backend. If we do not take care, multi-attach would"}],"source_content_type":"text/x-rst","patch_set":7,"id":"3f4b6375_ef9dbf59","line":47,"in_reply_to":"3f4b6375_b4527a66","updated":"2017-10-19 20:08:49.000000000","message":"In master, Cinder handles it today and throws \"Invalid volume: Volume status must be available to reserve\". With multi-attach, this just needs to be kept in mind. (Tho with the new api code I was able to do a volume_attach of the bfv boot volume as a separate volume to the same instance; so something may not be quite right there, I\u0027ll look into that)","commit_id":"d287d4504bad22a8fa1f8a29b81a86376bfebde6"},{"author":{"_account_id":2243,"name":"John Griffith","email":"john.griffith8@gmail.com","username":"john-griffith"},"change_message_id":"13042fd3e744e7362e576bff316b03bdcdf5b01a","unresolved":false,"context_lines":[{"line_number":49,"context_line":"There are problems today when multiple volume attachments share a single"},{"line_number":50,"context_line":"target to the volume backend. If we do not take care, multi-attach would"},{"line_number":51,"context_line":"make these problems much worse. The simplest fix is to serialize all attach and"},{"line_number":52,"context_line":"detach operations involving a shared target. To do this Cinder will expose"},{"line_number":53,"context_line":"a volume info property of \u0027shared_targets\u0027, when True a lock will be"},{"line_number":54,"context_line":"placed around all attachment_update and attachment_delete calls, and the"},{"line_number":55,"context_line":"associated calls to os-brick."},{"line_number":56,"context_line":""},{"line_number":57,"context_line":" with optional_host_local_lock(acquire\u003dvolume.shared_target):"},{"line_number":58,"context_line":"   connector \u003d os-brick.get_connector()"}],"source_content_type":"text/x-rst","patch_set":7,"id":"3f4b6375_0eb261e7","line":55,"range":{"start_line":52,"start_character":45,"end_line":55,"end_character":29},"updated":"2017-10-19 18:12:13.000000000","message":"Should the backend_uuid attribute of the volume be mentioned here as well?","commit_id":"d287d4504bad22a8fa1f8a29b81a86376bfebde6"},{"author":{"_account_id":9562,"name":"Ildiko Vancsa","email":"ildiko.vancsa@gmail.com","username":"ildikov"},"change_message_id":"0dd78f27809bf083339a601655abf8641639c3cc","unresolved":false,"context_lines":[{"line_number":49,"context_line":"There are problems today when multiple volume attachments share a single"},{"line_number":50,"context_line":"target to the volume backend. If we do not take care, multi-attach would"},{"line_number":51,"context_line":"make these problems much worse. The simplest fix is to serialize all attach and"},{"line_number":52,"context_line":"detach operations involving a shared target. To do this Cinder will expose"},{"line_number":53,"context_line":"a volume info property of \u0027shared_targets\u0027, when True a lock will be"},{"line_number":54,"context_line":"placed around all attachment_update and attachment_delete calls, and the"},{"line_number":55,"context_line":"associated calls to os-brick."},{"line_number":56,"context_line":""},{"line_number":57,"context_line":" with optional_host_local_lock(acquire\u003dvolume.shared_target):"},{"line_number":58,"context_line":"   connector \u003d os-brick.get_connector()"}],"source_content_type":"text/x-rst","patch_set":7,"id":"3f4b6375_e9d01f53","line":55,"range":{"start_line":52,"start_character":45,"end_line":55,"end_character":29},"in_reply_to":"3f4b6375_0eb261e7","updated":"2017-10-19 18:27:39.000000000","message":"I think it\u0027s ok as implementation detail?","commit_id":"d287d4504bad22a8fa1f8a29b81a86376bfebde6"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"37d8959d2ecadb7c4fa09b46bd56a18fbb004cf1","unresolved":false,"context_lines":[{"line_number":53,"context_line":"a volume info property of \u0027shared_targets\u0027, when True a lock will be"},{"line_number":54,"context_line":"placed around all attachment_update and attachment_delete calls, and the"},{"line_number":55,"context_line":"associated calls to os-brick."},{"line_number":56,"context_line":""},{"line_number":57,"context_line":" with optional_host_local_lock(acquire\u003dvolume.shared_target):"},{"line_number":58,"context_line":"   connector \u003d os-brick.get_connector()"},{"line_number":59,"context_line":"   conn_info \u003d attachment.update(connector).conn_info"}],"source_content_type":"text/x-rst","patch_set":7,"id":"3f4b6375_b4d71ac0","line":56,"updated":"2017-10-19 19:43:27.000000000","message":"nit: pseudo code should got in a ..code:: block.","commit_id":"d287d4504bad22a8fa1f8a29b81a86376bfebde6"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"37d8959d2ecadb7c4fa09b46bd56a18fbb004cf1","unresolved":false,"context_lines":[{"line_number":57,"context_line":" with optional_host_local_lock(acquire\u003dvolume.shared_target):"},{"line_number":58,"context_line":"   connector \u003d os-brick.get_connector()"},{"line_number":59,"context_line":"   conn_info \u003d attachment.update(connector).conn_info"},{"line_number":60,"context_line":"   os-brick.attach(conn_info)"},{"line_number":61,"context_line":"   attachment.attach_complete()"},{"line_number":62,"context_line":""},{"line_number":63,"context_line":" with optional_host_local_lock(acquire\u003dvolume.shared_target):"}],"source_content_type":"text/x-rst","patch_set":7,"id":"3f4b6375_94e41eae","line":60,"range":{"start_line":60,"start_character":12,"end_line":60,"end_character":18},"updated":"2017-10-19 19:43:27.000000000","message":"connect_volume","commit_id":"d287d4504bad22a8fa1f8a29b81a86376bfebde6"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"37d8959d2ecadb7c4fa09b46bd56a18fbb004cf1","unresolved":false,"context_lines":[{"line_number":61,"context_line":"   attachment.attach_complete()"},{"line_number":62,"context_line":""},{"line_number":63,"context_line":" with optional_host_local_lock(acquire\u003dvolume.shared_target):"},{"line_number":64,"context_line":"   os-brick.detach(conn_info)"},{"line_number":65,"context_line":"   attachment.delete()"},{"line_number":66,"context_line":""},{"line_number":67,"context_line":".. note::"}],"source_content_type":"text/x-rst","patch_set":7,"id":"3f4b6375_74e1a29a","line":64,"range":{"start_line":64,"start_character":12,"end_line":64,"end_character":18},"updated":"2017-10-19 19:43:27.000000000","message":"disconnect_volume","commit_id":"d287d4504bad22a8fa1f8a29b81a86376bfebde6"},{"author":{"_account_id":2243,"name":"John Griffith","email":"john.griffith8@gmail.com","username":"john-griffith"},"change_message_id":"13042fd3e744e7362e576bff316b03bdcdf5b01a","unresolved":false,"context_lines":[{"line_number":68,"context_line":""},{"line_number":69,"context_line":"  We assume the detach and attach related calls to Cinder are synchronous so"},{"line_number":70,"context_line":"  there will be no races between os-brick operations on the host and cinder"},{"line_number":71,"context_line":"  operations on the backend. Any driver deviation from this pattern will be"},{"line_number":72,"context_line":"  considered a bug."},{"line_number":73,"context_line":""},{"line_number":74,"context_line":"By default libvirt assumes all disks are exclusively used by a single guest."},{"line_number":75,"context_line":"If you want to share disks between instances, you need to tell libvirt"}],"source_content_type":"text/x-rst","patch_set":7,"id":"3f4b6375_aecd8d5e","line":72,"range":{"start_line":71,"start_character":29,"end_line":72,"end_character":19},"updated":"2017-10-19 18:12:13.000000000","message":"driver deviation?  Whether a call is sync or async on the cinder side is up to the rpc API no?  In other words I agree that anything different is a bug but it\u0027s not necessarily specfiic to multi-attach or this spec is it?","commit_id":"d287d4504bad22a8fa1f8a29b81a86376bfebde6"},{"author":{"_account_id":2243,"name":"John Griffith","email":"john.griffith8@gmail.com","username":"john-griffith"},"change_message_id":"22bd2f123e859f17d04ffdad8e6f49b5905220aa","unresolved":false,"context_lines":[{"line_number":68,"context_line":""},{"line_number":69,"context_line":"  We assume the detach and attach related calls to Cinder are synchronous so"},{"line_number":70,"context_line":"  there will be no races between os-brick operations on the host and cinder"},{"line_number":71,"context_line":"  operations on the backend. Any driver deviation from this pattern will be"},{"line_number":72,"context_line":"  considered a bug."},{"line_number":73,"context_line":""},{"line_number":74,"context_line":"By default libvirt assumes all disks are exclusively used by a single guest."},{"line_number":75,"context_line":"If you want to share disks between instances, you need to tell libvirt"}],"source_content_type":"text/x-rst","patch_set":7,"id":"3f4b6375_095a5bf2","line":72,"range":{"start_line":71,"start_character":29,"end_line":72,"end_character":19},"in_reply_to":"3f4b6375_894acb52","updated":"2017-10-19 18:29:57.000000000","message":"Ok, your call","commit_id":"d287d4504bad22a8fa1f8a29b81a86376bfebde6"},{"author":{"_account_id":9562,"name":"Ildiko Vancsa","email":"ildiko.vancsa@gmail.com","username":"ildikov"},"change_message_id":"0dd78f27809bf083339a601655abf8641639c3cc","unresolved":false,"context_lines":[{"line_number":68,"context_line":""},{"line_number":69,"context_line":"  We assume the detach and attach related calls to Cinder are synchronous so"},{"line_number":70,"context_line":"  there will be no races between os-brick operations on the host and cinder"},{"line_number":71,"context_line":"  operations on the backend. Any driver deviation from this pattern will be"},{"line_number":72,"context_line":"  considered a bug."},{"line_number":73,"context_line":""},{"line_number":74,"context_line":"By default libvirt assumes all disks are exclusively used by a single guest."},{"line_number":75,"context_line":"If you want to share disks between instances, you need to tell libvirt"}],"source_content_type":"text/x-rst","patch_set":7,"id":"3f4b6375_894acb52","line":72,"range":{"start_line":71,"start_character":29,"end_line":72,"end_character":19},"in_reply_to":"3f4b6375_aecd8d5e","updated":"2017-10-19 18:27:39.000000000","message":"A little with the background knowledge about the detach issues earlier when testing. For now it\u0027s a bit of I reserved the right to say whatever goes wrong I will care about that later. :)","commit_id":"d287d4504bad22a8fa1f8a29b81a86376bfebde6"},{"author":{"_account_id":2243,"name":"John Griffith","email":"john.griffith8@gmail.com","username":"john-griffith"},"change_message_id":"13042fd3e744e7362e576bff316b03bdcdf5b01a","unresolved":false,"context_lines":[{"line_number":88,"context_line":"introduce the usage of the flag we will also need to bump the minimum compute"},{"line_number":89,"context_line":"version."},{"line_number":90,"context_line":""},{"line_number":91,"context_line":"Nova should reject the attach request in case the hypervisor does not support"},{"line_number":92,"context_line":"it, but with the current API it is not possible. This could probably be solved"},{"line_number":93,"context_line":"with policies later on but as a first we will leave it for the computes to fail"},{"line_number":94,"context_line":"in case of not running libvirt."},{"line_number":95,"context_line":""},{"line_number":96,"context_line":"Policies to add to Cinder::"},{"line_number":97,"context_line":""}],"source_content_type":"text/x-rst","patch_set":7,"id":"3f4b6375_4e7979fa","line":94,"range":{"start_line":91,"start_character":0,"end_line":94,"end_character":31},"updated":"2017-10-19 18:12:13.000000000","message":"seems like we\u0027re glossing over how Nova gets this info.  \n\nThis of course means they take the performance hit even if it\u0027s not multi-attached but oh well.","commit_id":"d287d4504bad22a8fa1f8a29b81a86376bfebde6"},{"author":{"_account_id":17386,"name":"Steve Noyes","email":"steve.noyes@oracle.com","username":"ssn1ssn"},"change_message_id":"dc469b21b357ba272d18f66daefba42f5511d608","unresolved":false,"context_lines":[{"line_number":88,"context_line":"introduce the usage of the flag we will also need to bump the minimum compute"},{"line_number":89,"context_line":"version."},{"line_number":90,"context_line":""},{"line_number":91,"context_line":"Nova should reject the attach request in case the hypervisor does not support"},{"line_number":92,"context_line":"it, but with the current API it is not possible. This could probably be solved"},{"line_number":93,"context_line":"with policies later on but as a first we will leave it for the computes to fail"},{"line_number":94,"context_line":"in case of not running libvirt."},{"line_number":95,"context_line":""},{"line_number":96,"context_line":"Policies to add to Cinder::"},{"line_number":97,"context_line":""}],"source_content_type":"text/x-rst","patch_set":7,"id":"3f4b6375_a98587af","line":94,"range":{"start_line":91,"start_character":0,"end_line":94,"end_character":31},"in_reply_to":"3f4b6375_0e46e17d","updated":"2017-10-19 19:13:31.000000000","message":"I\u0027m curious about why only libvirt? Xen also supports multi-attach. Is this so we can avoid testing xen/ma for queens?","commit_id":"d287d4504bad22a8fa1f8a29b81a86376bfebde6"},{"author":{"_account_id":2243,"name":"John Griffith","email":"john.griffith8@gmail.com","username":"john-griffith"},"change_message_id":"22bd2f123e859f17d04ffdad8e6f49b5905220aa","unresolved":false,"context_lines":[{"line_number":88,"context_line":"introduce the usage of the flag we will also need to bump the minimum compute"},{"line_number":89,"context_line":"version."},{"line_number":90,"context_line":""},{"line_number":91,"context_line":"Nova should reject the attach request in case the hypervisor does not support"},{"line_number":92,"context_line":"it, but with the current API it is not possible. This could probably be solved"},{"line_number":93,"context_line":"with policies later on but as a first we will leave it for the computes to fail"},{"line_number":94,"context_line":"in case of not running libvirt."},{"line_number":95,"context_line":""},{"line_number":96,"context_line":"Policies to add to Cinder::"},{"line_number":97,"context_line":""}],"source_content_type":"text/x-rst","patch_set":7,"id":"3f4b6375_c9732380","line":94,"range":{"start_line":91,"start_character":0,"end_line":94,"end_character":31},"in_reply_to":"3f4b6375_0e46e17d","updated":"2017-10-19 18:29:57.000000000","message":"what I mean is \"how\" does Nova know what to do here and when?","commit_id":"d287d4504bad22a8fa1f8a29b81a86376bfebde6"},{"author":{"_account_id":9562,"name":"Ildiko Vancsa","email":"ildiko.vancsa@gmail.com","username":"ildikov"},"change_message_id":"0dd78f27809bf083339a601655abf8641639c3cc","unresolved":false,"context_lines":[{"line_number":88,"context_line":"introduce the usage of the flag we will also need to bump the minimum compute"},{"line_number":89,"context_line":"version."},{"line_number":90,"context_line":""},{"line_number":91,"context_line":"Nova should reject the attach request in case the hypervisor does not support"},{"line_number":92,"context_line":"it, but with the current API it is not possible. This could probably be solved"},{"line_number":93,"context_line":"with policies later on but as a first we will leave it for the computes to fail"},{"line_number":94,"context_line":"in case of not running libvirt."},{"line_number":95,"context_line":""},{"line_number":96,"context_line":"Policies to add to Cinder::"},{"line_number":97,"context_line":""}],"source_content_type":"text/x-rst","patch_set":7,"id":"3f4b6375_0e46e17d","line":94,"range":{"start_line":91,"start_character":0,"end_line":94,"end_character":31},"in_reply_to":"3f4b6375_4e7979fa","updated":"2017-10-19 18:27:39.000000000","message":"You\u0027re right, it\u0027s a capability at this point that specifies it for the back end and will only be true for libvirt.","commit_id":"d287d4504bad22a8fa1f8a29b81a86376bfebde6"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"37d8959d2ecadb7c4fa09b46bd56a18fbb004cf1","unresolved":false,"context_lines":[{"line_number":88,"context_line":"introduce the usage of the flag we will also need to bump the minimum compute"},{"line_number":89,"context_line":"version."},{"line_number":90,"context_line":""},{"line_number":91,"context_line":"Nova should reject the attach request in case the hypervisor does not support"},{"line_number":92,"context_line":"it, but with the current API it is not possible. This could probably be solved"},{"line_number":93,"context_line":"with policies later on but as a first we will leave it for the computes to fail"},{"line_number":94,"context_line":"in case of not running libvirt."},{"line_number":95,"context_line":""},{"line_number":96,"context_line":"Policies to add to Cinder::"},{"line_number":97,"context_line":""}],"source_content_type":"text/x-rst","patch_set":7,"id":"3f4b6375_5431060b","line":94,"range":{"start_line":91,"start_character":0,"end_line":94,"end_character":31},"in_reply_to":"3f4b6375_a98587af","updated":"2017-10-19 19:43:27.000000000","message":"My comment from PS5:\n\n\u003e Cinder should at least have a policy for allowing or blocking creating volumes with multiattach\u003dTrue. That\u0027s something that should happen regardless of this change. Then the block storage admin is the one that is configuring whether or not a user can even get into a mess.\n\nSo if you\u0027re running a cloud with computes that don\u0027t support multiattach, let\u0027s say it\u0027s all vmware, then the operator can configure policy to disable multiattach volumes on the cinder side. We should probably clarify that here.\n\nIf you\u0027ve got a mixed hypervisor cloud and the user tries to attach a multiattach volume to an instance on a compute where the virt driver doesn\u0027t support multiattach, then the attach request fails on the compute and nova-compute calls attachment_delete to delete the attachment created in nova-api\u0027s attach_volume code.\n\nIf nova-api exposed backend compute driver capabilities then we could check and fail fast in the API, but nova doesn\u0027t have that yet so we\u0027re just left with policy rules and checks on the backend.","commit_id":"d287d4504bad22a8fa1f8a29b81a86376bfebde6"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"37d8959d2ecadb7c4fa09b46bd56a18fbb004cf1","unresolved":false,"context_lines":[{"line_number":286,"context_line":""},{"line_number":287,"context_line":".. [#] https://github.com/openstack/nova/blob/295224c41e7da07c5ddbdafc72ac5abf2d708c69/nova/compute/manager.py#L2369"},{"line_number":288,"context_line":""},{"line_number":289,"context_line":".. [#] https://github.com/openstack/nova/blob/295224c41e7da07c5ddbdafc72ac5abf2d708c69/nova/compute/api.py#L"},{"line_number":290,"context_line":""},{"line_number":291,"context_line":"History"},{"line_number":292,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d"}],"source_content_type":"text/x-rst","patch_set":7,"id":"3f4b6375_541a267d","line":289,"range":{"start_line":289,"start_character":105,"end_line":289,"end_character":108},"updated":"2017-10-19 19:43:27.000000000","message":"busted link","commit_id":"d287d4504bad22a8fa1f8a29b81a86376bfebde6"},{"author":{"_account_id":782,"name":"John Garbutt","email":"john@johngarbutt.com","username":"johngarbutt"},"change_message_id":"cfd4b91cac7d5d0bb290d256652c0c450c24d634","unresolved":false,"context_lines":[{"line_number":142,"context_line":"\u0027volume_snapshot_create\u0027 where we don\u0027t have the instance_uuid to retrieve the"},{"line_number":143,"context_line":"right BDM, therefore we need to disable this call for multi-attach. The API"},{"line_number":144,"context_line":"format for this request is not changed, it is only a protection until the"},{"line_number":145,"context_line":"required API changes to support this request with multi-attach."},{"line_number":146,"context_line":""},{"line_number":147,"context_line":"Another feature that needs further investigation is \u0027boot from volume\u0027 (BFV)."},{"line_number":148,"context_line":"The first aspect of the feature is the \u0027delete_on_termination\u0027 flag, which will"}],"source_content_type":"text/x-rst","patch_set":8,"id":"3f4b6375_95c1aee7","line":145,"updated":"2017-10-20 07:50:56.000000000","message":"I think we said the same for swap volume, but we can do that in code review.","commit_id":"7b6f5329eceabcdfa640b572c1f9a4bfc44ec001"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"6bba3f2c5cd7e939aceae8fadeac4c50a7cd7ac0","unresolved":false,"context_lines":[{"line_number":142,"context_line":"\u0027volume_snapshot_create\u0027 where we don\u0027t have the instance_uuid to retrieve the"},{"line_number":143,"context_line":"right BDM, therefore we need to disable this call for multi-attach. The API"},{"line_number":144,"context_line":"format for this request is not changed, it is only a protection until the"},{"line_number":145,"context_line":"required API changes to support this request with multi-attach."},{"line_number":146,"context_line":""},{"line_number":147,"context_line":"Another feature that needs further investigation is \u0027boot from volume\u0027 (BFV)."},{"line_number":148,"context_line":"The first aspect of the feature is the \u0027delete_on_termination\u0027 flag, which will"}],"source_content_type":"text/x-rst","patch_set":8,"id":"3f4b6375_cce9dc1a","line":145,"in_reply_to":"3f4b6375_95c1aee7","updated":"2017-10-20 13:13:27.000000000","message":"To not support swap volume with multiattach volumes? As noted in an earlier patch set in here where you asked about swap volume, I don\u0027t see why we can\u0027t support it.","commit_id":"7b6f5329eceabcdfa640b572c1f9a4bfc44ec001"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"08c92077781c31976766b03ffb5b3b1a32f6d2ef","unresolved":false,"context_lines":[{"line_number":142,"context_line":"\u0027volume_snapshot_create\u0027 where we don\u0027t have the instance_uuid to retrieve the"},{"line_number":143,"context_line":"right BDM, therefore we need to disable this call for multi-attach. The API"},{"line_number":144,"context_line":"format for this request is not changed, it is only a protection until the"},{"line_number":145,"context_line":"required API changes to support this request with multi-attach."},{"line_number":146,"context_line":""},{"line_number":147,"context_line":"Another feature that needs further investigation is \u0027boot from volume\u0027 (BFV)."},{"line_number":148,"context_line":"The first aspect of the feature is the \u0027delete_on_termination\u0027 flag, which will"}],"source_content_type":"text/x-rst","patch_set":8,"id":"3f4b6375_0ce7f442","line":145,"in_reply_to":"3f4b6375_cce9dc1a","updated":"2017-10-20 13:13:56.000000000","message":"https://review.openstack.org/#/c/499777/5/specs/queens/approved/cinder-volume-multi-attach.rst@115","commit_id":"7b6f5329eceabcdfa640b572c1f9a4bfc44ec001"}]}