)]}'
{"/COMMIT_MSG":[{"author":{"_account_id":14119,"name":"Sam Yaple","email":"sam+git@yaple.net","username":"SamYaple"},"change_message_id":"b61b90a844e07c5bb6f01b1aa2882da91b8f8895","unresolved":false,"context_lines":[{"line_number":6,"context_line":""},{"line_number":7,"context_line":"Support HA Active/Active configurations"},{"line_number":8,"context_line":""},{"line_number":9,"context_line":"This spec outlines issues needed to achive Cinder High Availability"},{"line_number":10,"context_line":"Active/Active configurations."},{"line_number":11,"context_line":""},{"line_number":12,"context_line":"Implements: blueprint cinder-volume-active-active-support"}],"source_content_type":"text/x-gerrit-commit-message","patch_set":7,"id":"ba8a016a_c075f131","line":9,"updated":"2015-11-19 18:16:19.000000000","message":"achieve\n\nnitpick","commit_id":"6b166f03500fc1971dc30f27d162893208ac1362"},{"author":{"_account_id":9535,"name":"Gorka Eguileor","email":"geguileo@redhat.com","username":"Gorka"},"change_message_id":"a65a37afbad0806edee06b0e4baf8554a035a891","unresolved":false,"context_lines":[{"line_number":6,"context_line":""},{"line_number":7,"context_line":"Support HA Active/Active configurations"},{"line_number":8,"context_line":""},{"line_number":9,"context_line":"This spec outlines issues needed to achive Cinder High Availability"},{"line_number":10,"context_line":"Active/Active configurations."},{"line_number":11,"context_line":""},{"line_number":12,"context_line":"Implements: blueprint cinder-volume-active-active-support"}],"source_content_type":"text/x-gerrit-commit-message","patch_set":7,"id":"ba8a016a_c8e28c01","line":9,"in_reply_to":"ba8a016a_c075f131","updated":"2015-11-23 11:07:36.000000000","message":"Done","commit_id":"6b166f03500fc1971dc30f27d162893208ac1362"}],"specs/mitaka/cinder-volume-active-active-support.rst":[{"author":{"_account_id":11600,"name":"Michał Dulko","email":"michal.dulko@gmail.com","username":"dulek"},"change_message_id":"1d42aa77fccdd6c33aa7923f844dc420aece5ac2","unresolved":false,"context_lines":[{"line_number":48,"context_line":"Job distribution"},{"line_number":49,"context_line":"----------------"},{"line_number":50,"context_line":""},{"line_number":51,"context_line":"Cinder has no concept of clusters, only has the concept of hosts and each host"},{"line_number":52,"context_line":"implements a specific backend/service."},{"line_number":53,"context_line":""},{"line_number":54,"context_line":"Cleanup"}],"source_content_type":"text/x-rst","patch_set":2,"id":"7a2fa921_dcb88c13","line":51,"updated":"2015-10-09 14:44:30.000000000","message":"I don\u0027t think that this is a problem by itself. Currently when running c-vol A/A, you would use host field to cluster nodes. The only problem caused by that is cleanup phase you\u0027re describing further, so clustering isn\u0027t an issue preventing A/A.","commit_id":"3f93805aa4127501b87115a4a484dcec495e6ac6"},{"author":{"_account_id":11600,"name":"Michał Dulko","email":"michal.dulko@gmail.com","username":"dulek"},"change_message_id":"24f90919edb5d78922c640a11cb68c4633913395","unresolved":false,"context_lines":[{"line_number":48,"context_line":"Job distribution"},{"line_number":49,"context_line":"----------------"},{"line_number":50,"context_line":""},{"line_number":51,"context_line":"Cinder has no concept of clusters, only has the concept of hosts and each host"},{"line_number":52,"context_line":"implements a specific backend/service."},{"line_number":53,"context_line":""},{"line_number":54,"context_line":"Cleanup"}],"source_content_type":"text/x-rst","patch_set":2,"id":"7a2fa921_edcd9066","line":51,"in_reply_to":"7a2fa921_52a7830a","updated":"2015-10-09 15:39:04.000000000","message":"Semantics argument gets me, it\u0027s like service-disable, which name is counter-intuitive. I agree we shouldn\u0027t introduce more of that.\n\nSo maybe I\u0027m nit-picking, but I think it would be beneficial to explain that motivations in this section.","commit_id":"3f93805aa4127501b87115a4a484dcec495e6ac6"},{"author":{"_account_id":9535,"name":"Gorka Eguileor","email":"geguileo@redhat.com","username":"Gorka"},"change_message_id":"7551fdf1f430720d5b0aaaeaadd2f2d1dab5cafe","unresolved":false,"context_lines":[{"line_number":48,"context_line":"Job distribution"},{"line_number":49,"context_line":"----------------"},{"line_number":50,"context_line":""},{"line_number":51,"context_line":"Cinder has no concept of clusters, only has the concept of hosts and each host"},{"line_number":52,"context_line":"implements a specific backend/service."},{"line_number":53,"context_line":""},{"line_number":54,"context_line":"Cleanup"}],"source_content_type":"text/x-rst","patch_set":2,"id":"7a2fa921_52a7830a","line":51,"in_reply_to":"7a2fa921_dcb88c13","updated":"2015-10-09 15:27:59.000000000","message":"Well, that depends on how you look at it.  The way I see it \u0027host\u0027 is a word with a specific meaning, and this meaning should be kept in cinder.\n\nUsing the same host on multiple nodes is simply a hack (and a bad one, like you mentioned), you would not be describing a host with this option, but a group of hosts, and that\u0027s precisely a cluster.\n\nAnd more importantly, to solve the problem of the cleanup you mention we will need to have the concept of cluster and host.  Because if we have no way of differentiating one node from another, how are we going to be able to do the cleanup or detect when nodes are down?\n\nSo it really is a problem that we don\u0027t have such a grouping.","commit_id":"3f93805aa4127501b87115a4a484dcec495e6ac6"},{"author":{"_account_id":9535,"name":"Gorka Eguileor","email":"geguileo@redhat.com","username":"Gorka"},"change_message_id":"00176d147b6331bb5ce425fb903bf90428751594","unresolved":false,"context_lines":[{"line_number":48,"context_line":"Job distribution"},{"line_number":49,"context_line":"----------------"},{"line_number":50,"context_line":""},{"line_number":51,"context_line":"Cinder has no concept of clusters, only has the concept of hosts and each host"},{"line_number":52,"context_line":"implements a specific backend/service."},{"line_number":53,"context_line":""},{"line_number":54,"context_line":"Cleanup"}],"source_content_type":"text/x-rst","patch_set":2,"id":"7a2fa921_a85466c9","line":51,"in_reply_to":"7a2fa921_edcd9066","updated":"2015-10-09 15:54:18.000000000","message":"I can\u0027t argue with that, as a general description is too brief.  I\u0027ll copy/paste some stuff from the Job distribution specs to explain it a little bit better.","commit_id":"3f93805aa4127501b87115a4a484dcec495e6ac6"},{"author":{"_account_id":7173,"name":"Scott DAngelo","email":"scott.dangelo@gmail.com","username":"scottda"},"change_message_id":"7dffa59b64e2cb8898619e88b7dba86e9c690774","unresolved":false,"context_lines":[{"line_number":126,"context_line":"Cleanup"},{"line_number":127,"context_line":"-------"},{"line_number":128,"context_line":""},{"line_number":129,"context_line":"Cleanup will keep tracking of resources that are have undergoing operations and"},{"line_number":130,"context_line":"will have cleanup mechanisms on the Scheduler as well as the Volume nodes."},{"line_number":131,"context_line":""},{"line_number":132,"context_line":"- Specs: https://review.openstack.org/236977"}],"source_content_type":"text/x-rst","patch_set":3,"id":"3a29b11f_1b96a64c","line":129,"updated":"2015-10-19 18:44:26.000000000","message":"Cleanup will keep track of resources that have ongoing operations","commit_id":"3e28ffe522dff92a490be04f4269bfc2c6c759ff"},{"author":{"_account_id":9535,"name":"Gorka Eguileor","email":"geguileo@redhat.com","username":"Gorka"},"change_message_id":"714643eeeca7767aae6d08c821eefcc9f7276fdd","unresolved":false,"context_lines":[{"line_number":126,"context_line":"Cleanup"},{"line_number":127,"context_line":"-------"},{"line_number":128,"context_line":""},{"line_number":129,"context_line":"Cleanup will keep tracking of resources that are have undergoing operations and"},{"line_number":130,"context_line":"will have cleanup mechanisms on the Scheduler as well as the Volume nodes."},{"line_number":131,"context_line":""},{"line_number":132,"context_line":"- Specs: https://review.openstack.org/236977"}],"source_content_type":"text/x-rst","patch_set":3,"id":"3a29b11f_b82f43f5","line":129,"in_reply_to":"3a29b11f_1b96a64c","updated":"2015-10-20 16:32:25.000000000","message":"Done","commit_id":"3e28ffe522dff92a490be04f4269bfc2c6c759ff"},{"author":{"_account_id":11600,"name":"Michał Dulko","email":"michal.dulko@gmail.com","username":"dulek"},"change_message_id":"5b02c8fa5a260c9e05ea243856e03a343682788d","unresolved":false,"context_lines":[{"line_number":76,"context_line":"Since multiple nodes will be accessing the same storage back-end we have to be"},{"line_number":77,"context_line":"extra careful not to access resources that are accessed by other nodes."},{"line_number":78,"context_line":""},{"line_number":79,"context_line":"More relevant case is when we lose connection to the DB and we no longer can Service"},{"line_number":80,"context_line":"Heartbeats, since Scheduler\u0027s cleanup process will come into place and we could"},{"line_number":81,"context_line":"have 2 different nodes accessing the same resource, one because it\u0027s still"},{"line_number":82,"context_line":"working on it and the other because it is trying to do the cleanup."}],"source_content_type":"text/x-rst","patch_set":4,"id":"3a29b11f_35cedc5a","line":79,"updated":"2015-10-20 16:14:56.000000000","message":"\"can *send* Service Heartbeats\"?","commit_id":"eb5b143346bc50f74d0f56707cce47a34c8c40eb"},{"author":{"_account_id":7173,"name":"Scott DAngelo","email":"scott.dangelo@gmail.com","username":"scottda"},"change_message_id":"750d6d4621b9c5fef042f4cec22ee8f76fcde7e1","unresolved":false,"context_lines":[{"line_number":124,"context_line":"-------"},{"line_number":125,"context_line":""},{"line_number":126,"context_line":"Cleanup will keep track of resources that are have ongoing operations and will"},{"line_number":127,"context_line":"have cleanup mechanisms on the Scheduler as well as the Volume nodes."},{"line_number":128,"context_line":""},{"line_number":129,"context_line":"- Specs: https://review.openstack.org/236977"},{"line_number":130,"context_line":""}],"source_content_type":"text/x-rst","patch_set":5,"id":"3a29b11f_a3a06ba0","line":127,"updated":"2015-10-20 20:56:52.000000000","message":"I commented on this in the spec for cleanup, but it seems we could simplify to just one mechanism on the Scheduler without needing the Volume nodes to cleanup.","commit_id":"4b38baaa9c0eaf6053bc0e3362d362fd45c42bf3"},{"author":{"_account_id":9535,"name":"Gorka Eguileor","email":"geguileo@redhat.com","username":"Gorka"},"change_message_id":"5b67918408677cd75a466eeec07913b0f8f0f8af","unresolved":false,"context_lines":[{"line_number":124,"context_line":"-------"},{"line_number":125,"context_line":""},{"line_number":126,"context_line":"Cleanup will keep track of resources that are have ongoing operations and will"},{"line_number":127,"context_line":"have cleanup mechanisms on the Scheduler as well as the Volume nodes."},{"line_number":128,"context_line":""},{"line_number":129,"context_line":"- Specs: https://review.openstack.org/236977"},{"line_number":130,"context_line":""}],"source_content_type":"text/x-rst","patch_set":5,"id":"3a29b11f_337a4dbe","line":127,"in_reply_to":"3a29b11f_1fdba4ed","updated":"2015-10-21 11:50:59.000000000","message":"I have added a couple of paragraphs on that alternative to the specific spec.","commit_id":"4b38baaa9c0eaf6053bc0e3362d362fd45c42bf3"},{"author":{"_account_id":11600,"name":"Michał Dulko","email":"michal.dulko@gmail.com","username":"dulek"},"change_message_id":"fa647ab59c578cb943555e9fec72636b30dfa3c7","unresolved":false,"context_lines":[{"line_number":124,"context_line":"-------"},{"line_number":125,"context_line":""},{"line_number":126,"context_line":"Cleanup will keep track of resources that are have ongoing operations and will"},{"line_number":127,"context_line":"have cleanup mechanisms on the Scheduler as well as the Volume nodes."},{"line_number":128,"context_line":""},{"line_number":129,"context_line":"- Specs: https://review.openstack.org/236977"},{"line_number":130,"context_line":""}],"source_content_type":"text/x-rst","patch_set":5,"id":"3a29b11f_1fdba4ed","line":127,"in_reply_to":"3a29b11f_a3a06ba0","updated":"2015-10-21 06:46:02.000000000","message":"Unfortunately we need such mechanism on both. If c-vol service will be down for time lower than CONF.service_down_time, then c-sch may not notice that c-vol was down.","commit_id":"4b38baaa9c0eaf6053bc0e3362d362fd45c42bf3"},{"author":{"_account_id":6491,"name":"xing-yang","email":"xingyang105@gmail.com","username":"xing-yang"},"change_message_id":"cc5903dca18311181d1209ae19496ed3891cc3e0","unresolved":false,"context_lines":[{"line_number":77,"context_line":"extra careful not to access resources that are accessed by other nodes."},{"line_number":78,"context_line":""},{"line_number":79,"context_line":"More relevant case is when we lose connection to the DB and we no longer can"},{"line_number":80,"context_line":"send Service Heartbeats, since Scheduler\u0027s cleanup process will come into place"},{"line_number":81,"context_line":"and we could have 2 different nodes accessing the same resource, one because"},{"line_number":82,"context_line":"it\u0027s still working on it and the other because it is trying to do the cleanup."},{"line_number":83,"context_line":""}],"source_content_type":"text/x-rst","patch_set":6,"id":"da85f559_233d30c4","line":80,"updated":"2015-11-11 04:20:14.000000000","message":"I\u0027m not following you here.  What scheduler\u0027s cleanup process you are referring to?","commit_id":"f7664596a1146af34dcc68fa7912b5e2935f4083"},{"author":{"_account_id":9535,"name":"Gorka Eguileor","email":"geguileo@redhat.com","username":"Gorka"},"change_message_id":"0bc37d00b04d57769811244890a80837783c7919","unresolved":false,"context_lines":[{"line_number":77,"context_line":"extra careful not to access resources that are accessed by other nodes."},{"line_number":78,"context_line":""},{"line_number":79,"context_line":"More relevant case is when we lose connection to the DB and we no longer can"},{"line_number":80,"context_line":"send Service Heartbeats, since Scheduler\u0027s cleanup process will come into place"},{"line_number":81,"context_line":"and we could have 2 different nodes accessing the same resource, one because"},{"line_number":82,"context_line":"it\u0027s still working on it and the other because it is trying to do the cleanup."},{"line_number":83,"context_line":""}],"source_content_type":"text/x-rst","patch_set":6,"id":"ba8a016a_5239ecd5","line":80,"in_reply_to":"da85f559_233d30c4","updated":"2015-11-19 12:42:11.000000000","message":"I\u0027m referring to the cleanup the scheduler would do to cover L69-L71, but reading it now it\u0027s not explained in the Cleanup section (it is only in the specific cleanup spec).  I\u0027ll add it.","commit_id":"f7664596a1146af34dcc68fa7912b5e2935f4083"},{"author":{"_account_id":6491,"name":"xing-yang","email":"xingyang105@gmail.com","username":"xing-yang"},"change_message_id":"cc5903dca18311181d1209ae19496ed3891cc3e0","unresolved":false,"context_lines":[{"line_number":148,"context_line":"--------------"},{"line_number":149,"context_line":""},{"line_number":150,"context_line":"Create a generic locking mechanism extending the locking mechanism implemented"},{"line_number":151,"context_line":"to remove `Manager Local Locks`_."},{"line_number":152,"context_line":""},{"line_number":153,"context_line":"- Specs: https://review.openstack.org/237604"},{"line_number":154,"context_line":""}],"source_content_type":"text/x-rst","patch_set":6,"id":"da85f559_035eec11","line":151,"updated":"2015-11-11 04:20:14.000000000","message":"s/Manager/Driver","commit_id":"f7664596a1146af34dcc68fa7912b5e2935f4083"},{"author":{"_account_id":9535,"name":"Gorka Eguileor","email":"geguileo@redhat.com","username":"Gorka"},"change_message_id":"0bc37d00b04d57769811244890a80837783c7919","unresolved":false,"context_lines":[{"line_number":148,"context_line":"--------------"},{"line_number":149,"context_line":""},{"line_number":150,"context_line":"Create a generic locking mechanism extending the locking mechanism implemented"},{"line_number":151,"context_line":"to remove `Manager Local Locks`_."},{"line_number":152,"context_line":""},{"line_number":153,"context_line":"- Specs: https://review.openstack.org/237604"},{"line_number":154,"context_line":""}],"source_content_type":"text/x-rst","patch_set":6,"id":"ba8a016a_d27abc87","line":151,"in_reply_to":"da85f559_035eec11","updated":"2015-11-19 12:42:11.000000000","message":"No, this extends the one from the Manager locks.","commit_id":"f7664596a1146af34dcc68fa7912b5e2935f4083"},{"author":{"_account_id":8846,"name":"Hao Wang","display_name":"Hao Wang","email":"sxmatch1986@gmail.com","username":"sxmatch"},"change_message_id":"2c168d0ef53ab0c05508bd849e2eac2f27b1cc22","unresolved":false,"context_lines":[{"line_number":64,"context_line":""},{"line_number":65,"context_line":"It is different with an Active/Active deployment since multiple nodes are"},{"line_number":66,"context_line":"sharing the same storage back-end and a node can only do cleanup for the nodes"},{"line_number":67,"context_line":"he was working on when he died/failed."},{"line_number":68,"context_line":""},{"line_number":69,"context_line":"It is also important to do proper cleanup even when a specific node does not"},{"line_number":70,"context_line":"come back to life, since other nodes from the same cluster can still manage"}],"source_content_type":"text/x-rst","patch_set":8,"id":"9a8ffd7b_1e1fdf5b","line":67,"updated":"2015-11-28 02:38:04.000000000","message":"There confuses me a little, do you mean a node can only do cleanup for the \u0027resources\u0027 this node was working on when this node died/failed?","commit_id":"3e6e1f1b0156f2cf74175c77faead9ed65b1dfbf"},{"author":{"_account_id":170,"name":"Mike Perez","email":"thingee@gmail.com","username":"thingee"},"change_message_id":"56c6ac44e552bfc2317066e660514f4dfe86c942","unresolved":false,"context_lines":[{"line_number":64,"context_line":""},{"line_number":65,"context_line":"It is different with an Active/Active deployment since multiple nodes are"},{"line_number":66,"context_line":"sharing the same storage back-end and a node can only do cleanup for the nodes"},{"line_number":67,"context_line":"he was working on when he died/failed."},{"line_number":68,"context_line":""},{"line_number":69,"context_line":"It is also important to do proper cleanup even when a specific node does not"},{"line_number":70,"context_line":"come back to life, since other nodes from the same cluster can still manage"}],"source_content_type":"text/x-rst","patch_set":8,"id":"9a8ffd7b_c830410a","line":67,"in_reply_to":"9a8ffd7b_1e1fdf5b","updated":"2015-12-01 17:38:54.000000000","message":"It appears that way to me too.","commit_id":"3e6e1f1b0156f2cf74175c77faead9ed65b1dfbf"},{"author":{"_account_id":7173,"name":"Scott DAngelo","email":"scott.dangelo@gmail.com","username":"scottda"},"change_message_id":"cfda142ecf38ec42bf3b7ac0c4169268a971dc1a","unresolved":false,"context_lines":[{"line_number":64,"context_line":""},{"line_number":65,"context_line":"It is different with an Active/Active deployment since multiple nodes are"},{"line_number":66,"context_line":"sharing the same storage back-end and a node can only do cleanup for the nodes"},{"line_number":67,"context_line":"he was working on when he died/failed."},{"line_number":68,"context_line":""},{"line_number":69,"context_line":"It is also important to do proper cleanup even when a specific node does not"},{"line_number":70,"context_line":"come back to life, since other nodes from the same cluster can still manage"}],"source_content_type":"text/x-rst","patch_set":8,"id":"9a8ffd7b_ddc8bc24","line":67,"in_reply_to":"9a8ffd7b_c830410a","updated":"2015-12-02 20:02:42.000000000","message":"It does not seem to be a given that an AA c-vol node must only do the cleanup for the nodes it was working on when it died/failed. Below you mention the case where the node never comes back, so it seems any node could be able to do the cleanup.\nI think if the plan is to have the node that started the work do the cleanup when possible, and have a mechanism in the scheduler do the cleanup as a fallback for a node that never returns, the above sentence could be changed to reflect that this is a design decision and not something that Must be done. These details are spelled out in the Cleanup section below.","commit_id":"3e6e1f1b0156f2cf74175c77faead9ed65b1dfbf"},{"author":{"_account_id":9535,"name":"Gorka Eguileor","email":"geguileo@redhat.com","username":"Gorka"},"change_message_id":"2bde4f8d9d3e7f87ffd6ddc87c0cdb3737726627","unresolved":false,"context_lines":[{"line_number":64,"context_line":""},{"line_number":65,"context_line":"It is different with an Active/Active deployment since multiple nodes are"},{"line_number":66,"context_line":"sharing the same storage back-end and a node can only do cleanup for the nodes"},{"line_number":67,"context_line":"he was working on when he died/failed."},{"line_number":68,"context_line":""},{"line_number":69,"context_line":"It is also important to do proper cleanup even when a specific node does not"},{"line_number":70,"context_line":"come back to life, since other nodes from the same cluster can still manage"}],"source_content_type":"text/x-rst","patch_set":8,"id":"9a8ffd7b_8acf1628","line":67,"in_reply_to":"9a8ffd7b_ddc8bc24","updated":"2015-12-03 13:11:33.000000000","message":"I have to agree, that sentence was a little confusing.  I was trying to say that with current code it is only thinking of cleaning after resources he was working on.  I think the individual spec that describes this feature explains it better (it\u0027s basically what Scott said in his comment).","commit_id":"3e6e1f1b0156f2cf74175c77faead9ed65b1dfbf"}]}