)]}'
{"specs/kilo/approved/dynamic-server-groups-add.rst":[{"author":{"_account_id":6772,"name":"Stephen Gordon","email":"sgordon@redhat.com","username":"sgordon"},"change_message_id":"9a451b9b5c33356a1917ce10444077fdaa274938","unresolved":false,"context_lines":[{"line_number":142,"context_line":"Due to the 79 characters per line limit, the table for anti-affinity race"},{"line_number":143,"context_line":"conditions could not be displayed here in ASCII text. It is saved as a PDF"},{"line_number":144,"context_line":"in the following github repository."},{"line_number":145,"context_line":"https://github.com/jmulsow/nova-specs-extras/raw/master/anti-affinity.pdf"},{"line_number":146,"context_line":""},{"line_number":147,"context_line":"There are not as many race conditions to worry about when a server group has an"},{"line_number":148,"context_line":"affinity policy. Resizes to a different host, cold migrations, and live"}],"source_content_type":"text/x-rst","patch_set":1,"id":"5a890539_9e559bc4","line":145,"updated":"2014-12-05 13:19:27.000000000","message":"Not really your problem but it would be ideal if we had a space for this type of artifact alongside the specs.","commit_id":"cc359c2adc61fb1d50f5e7a8bc53389eb8edd9ff"},{"author":{"_account_id":782,"name":"John Garbutt","email":"john@johngarbutt.com","username":"johngarbutt"},"change_message_id":"a571f482e7c3939aa31efcfdade26e1fa8bdc1ec","unresolved":false,"context_lines":[{"line_number":13,"context_line":"This blueprint extends the functionality of \"Server Groups\" such that existing"},{"line_number":14,"context_line":"virtual machines can be added to a group at any time, instead of limiting it"},{"line_number":15,"context_line":"to at boot time only. By providing this added bit of functionality, server"},{"line_number":16,"context_line":"groups become orders of magnitude more operator-friendly and usable in many"},{"line_number":17,"context_line":"real-world scenarios."},{"line_number":18,"context_line":""},{"line_number":19,"context_line":"For the \u0027remove a instance from a server group\u0027 functionality, see:"}],"source_content_type":"text/x-rst","patch_set":2,"id":"7aaa499b_ec37b0e1","line":16,"updated":"2015-04-13 13:21:46.000000000","message":"honestly, this seems more about API user than cloud operator.","commit_id":"819ae6b30ebad82a8f388f5e2f8afc7fb0818b88"},{"author":{"_account_id":782,"name":"John Garbutt","email":"john@johngarbutt.com","username":"johngarbutt"},"change_message_id":"a571f482e7c3939aa31efcfdade26e1fa8bdc1ec","unresolved":false,"context_lines":[{"line_number":28,"context_line":"* As a cloud operator, I want to add an existing instance to a server group, so"},{"line_number":29,"context_line":"  that I can have more flexibility with respect to defining what instances will"},{"line_number":30,"context_line":"  have affinity rules since I may not necessarily know these rules at instance"},{"line_number":31,"context_line":"  creation time."},{"line_number":32,"context_line":""},{"line_number":33,"context_line":"Project Priority"},{"line_number":34,"context_line":"-----------------"}],"source_content_type":"text/x-rst","patch_set":2,"id":"7aaa499b_4c20e49b","line":31,"updated":"2015-04-13 13:21:46.000000000","message":"A better example might be scaling out your web tier, but wanting to keep servers away from each other to help ensure more independent failure of each server.","commit_id":"819ae6b30ebad82a8f388f5e2f8afc7fb0818b88"},{"author":{"_account_id":782,"name":"John Garbutt","email":"john@johngarbutt.com","username":"johngarbutt"},"change_message_id":"a571f482e7c3939aa31efcfdade26e1fa8bdc1ec","unresolved":false,"context_lines":[{"line_number":40,"context_line":""},{"line_number":41,"context_line":"A new API will be added to the server group APIs in"},{"line_number":42,"context_line":"nova/api/openstack/compute/contrib/server_groups.py. This API will be a POST at"},{"line_number":43,"context_line":"server group members level. It will add an existing instance to a server group."},{"line_number":44,"context_line":""},{"line_number":45,"context_line":"When adding an instance from a server group, validation will need to be"},{"line_number":46,"context_line":"performed on the server_group_members quota value. The addition of that"}],"source_content_type":"text/x-rst","patch_set":2,"id":"7aaa499b_090d2e3e","line":43,"updated":"2015-04-13 13:21:46.000000000","message":"This now needs to be a new v2.1 microversion, please update.","commit_id":"819ae6b30ebad82a8f388f5e2f8afc7fb0818b88"},{"author":{"_account_id":782,"name":"John Garbutt","email":"john@johngarbutt.com","username":"johngarbutt"},"change_message_id":"a571f482e7c3939aa31efcfdade26e1fa8bdc1ec","unresolved":false,"context_lines":[{"line_number":49,"context_line":"When a user tries to add an instance to a server group, the API will need to"},{"line_number":50,"context_line":"check the hosts of all the instances in the group to compare with the host of"},{"line_number":51,"context_line":"the instance that is being requested to join. If the instance would violate the"},{"line_number":52,"context_line":"policy of the server group, then it will not be allowed to join."},{"line_number":53,"context_line":""},{"line_number":54,"context_line":"In addition to this check, the API to add an instance to a server group will"},{"line_number":55,"context_line":"need to ensure that servers could not be added in parallel to the same group"}],"source_content_type":"text/x-rst","patch_set":2,"id":"7aaa499b_8ceeec89","line":52,"updated":"2015-04-13 13:21:46.000000000","message":"sounds like the scheduler should be doing that work, using the current logic right?\n\nYou need some way of saying \"unable to schedule\" instance when the instance goes into an ERROR state, I guess.","commit_id":"819ae6b30ebad82a8f388f5e2f8afc7fb0818b88"},{"author":{"_account_id":782,"name":"John Garbutt","email":"john@johngarbutt.com","username":"johngarbutt"},"change_message_id":"a571f482e7c3939aa31efcfdade26e1fa8bdc1ec","unresolved":false,"context_lines":[{"line_number":55,"context_line":"need to ensure that servers could not be added in parallel to the same group"},{"line_number":56,"context_line":"that could cause a policy of the group to be violated. Currently, there are"},{"line_number":57,"context_line":"ways in which instances in a server group could be moved to a different host"},{"line_number":58,"context_line":"that would cause the policy to be violated. The addition of an API to add a"},{"line_number":59,"context_line":"server to a server group would add to these possibilities if proper care is"},{"line_number":60,"context_line":"not taken. The following part of this section will describe how race conditions"},{"line_number":61,"context_line":"will be addressed, both in the existing code, and with the addition of the new"}],"source_content_type":"text/x-rst","patch_set":2,"id":"7aaa499b_ecad7032","line":58,"updated":"2015-04-13 13:21:46.000000000","message":"seems like that should be something that gets fixed separately to this spec. agreed, it needs attention.","commit_id":"819ae6b30ebad82a8f388f5e2f8afc7fb0818b88"},{"author":{"_account_id":782,"name":"John Garbutt","email":"john@johngarbutt.com","username":"johngarbutt"},"change_message_id":"a571f482e7c3939aa31efcfdade26e1fa8bdc1ec","unresolved":false,"context_lines":[{"line_number":67,"context_line":"linked to below. This table outlines scenarios involving a server group with"},{"line_number":68,"context_line":"anti-affinity policy only. Affinity is discussed after the anti-affinity table."},{"line_number":69,"context_line":""},{"line_number":70,"context_line":"The table is structured such that the first column represents the operation"},{"line_number":71,"context_line":"that the user is trying to perform on an instance in a server group, and the"},{"line_number":72,"context_line":"horizontal axis represents other actions that are going on in parallel to"},{"line_number":73,"context_line":"instances in the same server group. The last column describes how the scenario"}],"source_content_type":"text/x-rst","patch_set":2,"id":"7aaa499b_9f69708e","line":70,"updated":"2015-04-13 13:21:46.000000000","message":"I think you should only talk about the table next to the reference to the external doc. It should be clear what you mean in here without the need for the external doc.","commit_id":"819ae6b30ebad82a8f388f5e2f8afc7fb0818b88"},{"author":{"_account_id":782,"name":"John Garbutt","email":"john@johngarbutt.com","username":"johngarbutt"},"change_message_id":"a571f482e7c3939aa31efcfdade26e1fa8bdc1ec","unresolved":false,"context_lines":[{"line_number":83,"context_line":"  any of the following actions are being performed on any instances in the"},{"line_number":84,"context_line":"  server group: deploy, live migration, cold migration, and resize. This will"},{"line_number":85,"context_line":"  be checked in the \"add server to a server group API\", and a 409 HTTP error"},{"line_number":86,"context_line":"  code will be returned with a message explaining this to the user."},{"line_number":87,"context_line":""},{"line_number":88,"context_line":"  This check will prevent a timing window where the server group policy can be"},{"line_number":89,"context_line":"  violated due to an ongoing operation that places an instance on a new host"}],"source_content_type":"text/x-rst","patch_set":2,"id":"7aaa499b_7fffc401","line":86,"updated":"2015-04-13 13:21:46.000000000","message":"This check, in its-self seems racy. You can start one of those actions just after you launch the instance.\n\nI propose you instead to a secondary check just before starting the server, after updating the host field, that there are no violations.","commit_id":"819ae6b30ebad82a8f388f5e2f8afc7fb0818b88"},{"author":{"_account_id":782,"name":"John Garbutt","email":"john@johngarbutt.com","username":"johngarbutt"},"change_message_id":"a571f482e7c3939aa31efcfdade26e1fa8bdc1ec","unresolved":false,"context_lines":[{"line_number":101,"context_line":"  undergoing a cold migration, live migration, or resize. The Nova scheduler"},{"line_number":102,"context_line":"  will also filter out any target hosts of ongoing deploys of instances being"},{"line_number":103,"context_line":"  deployed to the same server group. The source and target hosts of an ongoing"},{"line_number":104,"context_line":"  resize will only be filtered out if they are not the same."},{"line_number":105,"context_line":""},{"line_number":106,"context_line":"  This additional logic will be added to the existing anti-affinity filter."},{"line_number":107,"context_line":"  This additional logic is needed because these operations can be rolled back"}],"source_content_type":"text/x-rst","patch_set":2,"id":"7aaa499b_9f3170ef","line":104,"updated":"2015-04-13 13:21:46.000000000","message":"didn\u0027t you just say you wouldn\u0027t allow any of these to happen, given the above?","commit_id":"819ae6b30ebad82a8f388f5e2f8afc7fb0818b88"},{"author":{"_account_id":782,"name":"John Garbutt","email":"john@johngarbutt.com","username":"johngarbutt"},"change_message_id":"a571f482e7c3939aa31efcfdade26e1fa8bdc1ec","unresolved":false,"context_lines":[{"line_number":121,"context_line":"  As an extra precaution, the \"add a server to a server group API\" will do an"},{"line_number":122,"context_line":"  additional policy validation check after the database has been updated. If it"},{"line_number":123,"context_line":"  finds that the policy has been violated at this point, it will back out the"},{"line_number":124,"context_line":"  addition of this instance into the server group, and return an HTTP 409 code."},{"line_number":125,"context_line":""},{"line_number":126,"context_line":"Mechanism 4::"},{"line_number":127,"context_line":""}],"source_content_type":"text/x-rst","patch_set":2,"id":"7aaa499b_5f876861","line":124,"updated":"2015-04-13 13:21:46.000000000","message":"You can\u0027t return an error code at this point, the API has already returned, assuming you mean updating the host field in the database?","commit_id":"819ae6b30ebad82a8f388f5e2f8afc7fb0818b88"},{"author":{"_account_id":782,"name":"John Garbutt","email":"john@johngarbutt.com","username":"johngarbutt"},"change_message_id":"a571f482e7c3939aa31efcfdade26e1fa8bdc1ec","unresolved":false,"context_lines":[{"line_number":133,"context_line":""},{"line_number":134,"context_line":"  This will prevent race conditions where simultaneous calls to the \"add a"},{"line_number":135,"context_line":"  server to a server group API\" could add multiple instances with conflicting"},{"line_number":136,"context_line":"  hosts to the server group\u0027s policy."},{"line_number":137,"context_line":""},{"line_number":138,"context_line":"  This will only be effective when there are not multiple API services running."},{"line_number":139,"context_line":"  However, if there are multiple API services, then Mechanism 3 should be"}],"source_content_type":"text/x-rst","patch_set":2,"id":"7aaa499b_1fcfa0b1","line":136,"updated":"2015-04-13 13:21:46.000000000","message":"Please do this using the DB. You can do a \"compare and swap\" style DB command to reduce the number of locks needed on the DB side (i.e. only update when the state of the DB is as you expect, otherwise retry).","commit_id":"819ae6b30ebad82a8f388f5e2f8afc7fb0818b88"},{"author":{"_account_id":782,"name":"John Garbutt","email":"john@johngarbutt.com","username":"johngarbutt"},"change_message_id":"a571f482e7c3939aa31efcfdade26e1fa8bdc1ec","unresolved":false,"context_lines":[{"line_number":137,"context_line":""},{"line_number":138,"context_line":"  This will only be effective when there are not multiple API services running."},{"line_number":139,"context_line":"  However, if there are multiple API services, then Mechanism 3 should be"},{"line_number":140,"context_line":"  sufficient in preventing simultaneous adds from violating the policy."},{"line_number":141,"context_line":""},{"line_number":142,"context_line":"Due to the 79 characters per line limit, the table for anti-affinity race"},{"line_number":143,"context_line":"conditions could not be displayed here in ASCII text. It is saved as a PDF"}],"source_content_type":"text/x-rst","patch_set":2,"id":"7aaa499b_bfbe6cfd","line":140,"updated":"2015-04-13 13:21:46.000000000","message":"we must assume multiple API servers, please don\u0027t do this.","commit_id":"819ae6b30ebad82a8f388f5e2f8afc7fb0818b88"},{"author":{"_account_id":782,"name":"John Garbutt","email":"john@johngarbutt.com","username":"johngarbutt"},"change_message_id":"a571f482e7c3939aa31efcfdade26e1fa8bdc1ec","unresolved":false,"context_lines":[{"line_number":149,"context_line":"migrations will not be allowed on any instances in an affinity server group"},{"line_number":150,"context_line":"since instances can only ever be on a single host. The following are the"},{"line_number":151,"context_line":"scenarios that have the possibility of violating the policy of the server"},{"line_number":152,"context_line":"group."},{"line_number":153,"context_line":""},{"line_number":154,"context_line":"Scenario 1::"},{"line_number":155,"context_line":""}],"source_content_type":"text/x-rst","patch_set":2,"id":"7aaa499b_3f26dc68","line":152,"updated":"2015-04-13 13:21:46.000000000","message":"We must assume administrators might be half way through moving a full group, but your mechanism 1 should help protect against issues there.","commit_id":"819ae6b30ebad82a8f388f5e2f8afc7fb0818b88"},{"author":{"_account_id":782,"name":"John Garbutt","email":"john@johngarbutt.com","username":"johngarbutt"},"change_message_id":"a571f482e7c3939aa31efcfdade26e1fa8bdc1ec","unresolved":false,"context_lines":[{"line_number":159,"context_line":"     finishes"},{"line_number":160,"context_line":"  3. VM-1 deploy eventually fails, so it would generally reschedule... but"},{"line_number":161,"context_line":"     since VM-2 was added to the group, rescheduling should be disabled since"},{"line_number":162,"context_line":"     the policy would be violated"},{"line_number":163,"context_line":""},{"line_number":164,"context_line":"Scenario 2::"},{"line_number":165,"context_line":""}],"source_content_type":"text/x-rst","patch_set":2,"id":"7aaa499b_1fb9e0e9","line":162,"updated":"2015-04-13 13:21:46.000000000","message":"the reschedule will already exclude all other hosts, and thus be an empty list, assuming its working correctly.","commit_id":"819ae6b30ebad82a8f388f5e2f8afc7fb0818b88"},{"author":{"_account_id":782,"name":"John Garbutt","email":"john@johngarbutt.com","username":"johngarbutt"},"change_message_id":"a571f482e7c3939aa31efcfdade26e1fa8bdc1ec","unresolved":false,"context_lines":[{"line_number":178,"context_line":"created, it cannot join a server group."},{"line_number":179,"context_line":""},{"line_number":180,"context_line":"If this route is chosen, the existing race conditions and other issues outlined"},{"line_number":181,"context_line":"in the dependencies section will still need to be addressed."},{"line_number":182,"context_line":""},{"line_number":183,"context_line":"Data model impact"},{"line_number":184,"context_line":"-----------------"}],"source_content_type":"text/x-rst","patch_set":2,"id":"7aaa499b_9f9cd06a","line":181,"updated":"2015-04-13 13:21:46.000000000","message":"I would rather we looked at fixing problems with the existing API, rather than expanding this badly.\n\nBut I can see this being useful.","commit_id":"819ae6b30ebad82a8f388f5e2f8afc7fb0818b88"}]}