)]}'
{"/COMMIT_MSG":[{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"5d2e24ff9678f02f80c511c19a846ff0aa533c44","unresolved":false,"context_lines":[{"line_number":4,"context_line":"Commit:     Sylvain Bauza \u003csbauza@redhat.com\u003e"},{"line_number":5,"context_line":"CommitDate: 2017-04-12 16:08:16 +0200"},{"line_number":6,"context_line":""},{"line_number":7,"context_line":"Placement doing claims"},{"line_number":8,"context_line":""},{"line_number":9,"context_line":"Proposal to have scheduler posting allocations instead of computes."},{"line_number":10,"context_line":""}],"source_content_type":"text/x-gerrit-commit-message","patch_set":4,"id":"7ffa3b31_193727e6","line":7,"range":{"start_line":7,"start_character":0,"end_line":7,"end_character":22},"updated":"2017-04-17 20:55:37.000000000","message":"We should probably rename this, placement doesn\u0027t do the claims, nova does using placement.\n\nMaybe call it \"Claims in the scheduler\" instead since that\u0027s what we keep calling the idea.","commit_id":"d59286ab36ccc0cebcef62371e421ddf714a694b"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"5d2e24ff9678f02f80c511c19a846ff0aa533c44","unresolved":false,"context_lines":[{"line_number":8,"context_line":""},{"line_number":9,"context_line":"Proposal to have scheduler posting allocations instead of computes."},{"line_number":10,"context_line":""},{"line_number":11,"context_line":"Partially-Implements blueprint: placement-claims"},{"line_number":12,"context_line":""},{"line_number":13,"context_line":"Change-Id: I70b6efecd36901dd059723071668e5653d393e9e"}],"source_content_type":"text/x-gerrit-commit-message","patch_set":4,"id":"7ffa3b31_79574b04","line":11,"range":{"start_line":11,"start_character":0,"end_line":11,"end_character":20},"updated":"2017-04-17 20:55:37.000000000","message":"Spec for blueprint placement-claims","commit_id":"d59286ab36ccc0cebcef62371e421ddf714a694b"}],"specs/pike/approved/placement-claims.rst":[{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"df806fbf97113af207ca3ce1b50d6fc19ec4fd21","unresolved":false,"context_lines":[{"line_number":18,"context_line":""},{"line_number":19,"context_line":"The scheduler currently calls the Placement API in order to get a list of"},{"line_number":20,"context_line":"ResourceProviders that match a specific request made of amounts of resource"},{"line_number":21,"context_line":"classes. How the Placement API corresponding resource calculates that is made"},{"line_number":22,"context_line":"by getting the list of inventories for each resource class with their total"},{"line_number":23,"context_line":"amounts and decreasing each of those values by the amounts corresponding of the"},{"line_number":24,"context_line":"allocations that are done by the ResourceTracker when creating an instance."},{"line_number":25,"context_line":""},{"line_number":26,"context_line":"While this model fits very well for a non-resource-constrainted cloud, we know"},{"line_number":27,"context_line":"that this can be suboptimal because of requests concurrency. Somehow, we would"}],"source_content_type":"text/x-rst","patch_set":1,"id":"ffe62b97_8dc10657","line":24,"range":{"start_line":21,"start_character":9,"end_line":24,"end_character":75},"updated":"2017-03-29 18:07:09.000000000","message":"This is hard to read. I get what you\u0027re saying, but we should simplify the wording. I think you\u0027re essentially trying to say something like, \"Currently, the ResourceTracker (RT) in the nova-compute service is responsible for updating inventory and allocation values for resource providers, e.g. compute nodes, in the Placement service. The RT is also responsible for making claims during server build and move operations.\"","commit_id":"6b3c00b0673fd4af670fab7e5edff48af4d3111d"},{"author":{"_account_id":7,"name":"Jay Pipes","email":"jaypipes@gmail.com","username":"jaypipes"},"change_message_id":"c54b6d006ba7570a241cdcf6298a8ad5bfd72a8c","unresolved":false,"context_lines":[{"line_number":23,"context_line":"amounts and decreasing each of those values by the amounts corresponding of the"},{"line_number":24,"context_line":"allocations that are done by the ResourceTracker when creating an instance."},{"line_number":25,"context_line":""},{"line_number":26,"context_line":"While this model fits very well for a non-resource-constrainted cloud, we know"},{"line_number":27,"context_line":"that this can be suboptimal because of requests concurrency. Somehow, we would"},{"line_number":28,"context_line":"like to make sure that what the Placement API returns is based on not only what"},{"line_number":29,"context_line":"is existing, but also what is currently *subject to exist*."}],"source_content_type":"text/x-rst","patch_set":1,"id":"ffe62b97_52e835b1","line":26,"updated":"2017-03-29 21:08:00.000000000","message":"s/contrainted/constrained/\n\nThough, I\u0027m not sure quite what you mean by \"resource-constrained\". Are you referring to a cloud that is almost at capacity? Or are you referring to a cloud\u0027s API servers that are getting hammered with lots of concurrent requests?","commit_id":"6b3c00b0673fd4af670fab7e5edff48af4d3111d"},{"author":{"_account_id":782,"name":"John Garbutt","email":"john@johngarbutt.com","username":"johngarbutt"},"change_message_id":"3bb449d7c1aa88972567d92e98d8d477d55cd3dc","unresolved":false,"context_lines":[{"line_number":26,"context_line":"While this model fits very well for a non-resource-constrainted cloud, we know"},{"line_number":27,"context_line":"that this can be suboptimal because of requests concurrency. Somehow, we would"},{"line_number":28,"context_line":"like to make sure that what the Placement API returns is based on not only what"},{"line_number":29,"context_line":"is existing, but also what is currently *subject to exist*."},{"line_number":30,"context_line":""},{"line_number":31,"context_line":"To his merit, Mr. Jay Pipes carries the idea to have the Placement API being"},{"line_number":32,"context_line":"able to give a ticket back to the conductor (as an expression of a placement"}],"source_content_type":"text/x-rst","patch_set":1,"id":"ffe62b97_f51f8c58","line":29,"updated":"2017-03-28 09:12:07.000000000","message":"We should really mention this is really important for stack, compared to spread.\n\nWhen parallel similar requests come into the system, we generally pick the same \"best\" host for all those requests. This is because there is limited sharing of state between those requests. Now consider 1000 build requests, with identical resource requests, within a 30 second period. A great may of those builds fail because three retries is not enough to find a host where the request will fit.\n\nOne current solution to this problem is to stop picking the \"best\" host, and just pick one of the top 5 or top 10 possible hosts. When you are using a spread first packing (often when you have low to medium utilisation of your cloud much of the time, and you want to allow people to burst and use more server resources than the overcommit ratio would usually allow), that can work really well, particularly when you have lots of hosts with very similar final weights. When you are trying to stack first (usually chosen so you leave space for larger instances that may be requested in the future), this \"fuzzy selection\" of the best placement tends to lead to much more spreading than is ideal.\n\nAnother alternative is to reduce the number of concurrent decisions being made, and cache data between those requests. This is done by running a single nova-scheduler process and reducing the number of greenlet workers. The new non-block DB calls mean its not as reduce in parallism as before. The HostState.consume_for_instance() method is used to cache in the host state the decision that was made. If the DB returns something that is older, the cache is kept. In the caching scheduler, this stays around longer as the information is refreshed from the DB less frequently. This has the clear down side that it limits the throughput of the scheduler, and requires an Active/Passive style HA setup.\n\nIdeally we can find a way to get a better trade off between accuracy and throughput.","commit_id":"6b3c00b0673fd4af670fab7e5edff48af4d3111d"},{"author":{"_account_id":7,"name":"Jay Pipes","email":"jaypipes@gmail.com","username":"jaypipes"},"change_message_id":"c54b6d006ba7570a241cdcf6298a8ad5bfd72a8c","unresolved":false,"context_lines":[{"line_number":26,"context_line":"While this model fits very well for a non-resource-constrainted cloud, we know"},{"line_number":27,"context_line":"that this can be suboptimal because of requests concurrency. Somehow, we would"},{"line_number":28,"context_line":"like to make sure that what the Placement API returns is based on not only what"},{"line_number":29,"context_line":"is existing, but also what is currently *subject to exist*."},{"line_number":30,"context_line":""},{"line_number":31,"context_line":"To his merit, Mr. Jay Pipes carries the idea to have the Placement API being"},{"line_number":32,"context_line":"able to give a ticket back to the conductor (as an expression of a placement"}],"source_content_type":"text/x-rst","patch_set":1,"id":"ffe62b97_322a51da","line":29,"in_reply_to":"ffe62b97_f51f8c58","updated":"2017-03-29 21:08:00.000000000","message":"It\u0027s not really that we want to return what is \"subject to exist\". Rather, what we want to do is simply consume the resources (i.e. create a set of allocation records) in the same operation that we request those resources.","commit_id":"6b3c00b0673fd4af670fab7e5edff48af4d3111d"},{"author":{"_account_id":7,"name":"Jay Pipes","email":"jaypipes@gmail.com","username":"jaypipes"},"change_message_id":"c54b6d006ba7570a241cdcf6298a8ad5bfd72a8c","unresolved":false,"context_lines":[{"line_number":30,"context_line":""},{"line_number":31,"context_line":"To his merit, Mr. Jay Pipes carries the idea to have the Placement API being"},{"line_number":32,"context_line":"able to give a ticket back to the conductor (as an expression of a placement"},{"line_number":33,"context_line":"request) so later in the boot process, the compute can acknowledge that ticket."},{"line_number":34,"context_line":""},{"line_number":35,"context_line":"Use Cases"},{"line_number":36,"context_line":"---------"}],"source_content_type":"text/x-rst","patch_set":1,"id":"ffe62b97_f24d09c4","line":33,"updated":"2017-03-29 21:08:00.000000000","message":"Actually, that\u0027s not my idea :) The original blueprint of mine was called \"allocations in scheduler\" or something like that I believe. And that\u0027s what I really think needs to happen. Just change the scheduler to call POST /allocations instead of GET /resource_providers.","commit_id":"6b3c00b0673fd4af670fab7e5edff48af4d3111d"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"df806fbf97113af207ca3ce1b50d6fc19ec4fd21","unresolved":false,"context_lines":[{"line_number":36,"context_line":"---------"},{"line_number":37,"context_line":""},{"line_number":38,"context_line":"As an operator, I\u0027d like to avoid too many reschedules if my cloud becomes full"},{"line_number":39,"context_line":"because it could lead to a snowball effect."},{"line_number":40,"context_line":""},{"line_number":41,"context_line":"Proposed change"},{"line_number":42,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"}],"source_content_type":"text/x-rst","patch_set":1,"id":"ffe62b97_ed9c126b","line":39,"range":{"start_line":39,"start_character":27,"end_line":39,"end_character":35},"updated":"2017-03-29 18:07:09.000000000","message":"For our non-English native friends, maybe just say \"cascading scheduling failures\"?","commit_id":"6b3c00b0673fd4af670fab7e5edff48af4d3111d"},{"author":{"_account_id":7,"name":"Jay Pipes","email":"jaypipes@gmail.com","username":"jaypipes"},"change_message_id":"c54b6d006ba7570a241cdcf6298a8ad5bfd72a8c","unresolved":false,"context_lines":[{"line_number":36,"context_line":"---------"},{"line_number":37,"context_line":""},{"line_number":38,"context_line":"As an operator, I\u0027d like to avoid too many reschedules if my cloud becomes full"},{"line_number":39,"context_line":"because it could lead to a snowball effect."},{"line_number":40,"context_line":""},{"line_number":41,"context_line":"Proposed change"},{"line_number":42,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"}],"source_content_type":"text/x-rst","patch_set":1,"id":"ffe62b97_3206d153","line":39,"range":{"start_line":39,"start_character":27,"end_line":39,"end_character":35},"in_reply_to":"ffe62b97_ed9c126b","updated":"2017-03-29 21:08:00.000000000","message":"Or say more simply: \"As an operator, I\u0027d like to minimize the amount of time between requesting a resource and actually consuming that resource\".","commit_id":"6b3c00b0673fd4af670fab7e5edff48af4d3111d"},{"author":{"_account_id":7,"name":"Jay Pipes","email":"jaypipes@gmail.com","username":"jaypipes"},"change_message_id":"c54b6d006ba7570a241cdcf6298a8ad5bfd72a8c","unresolved":false,"context_lines":[{"line_number":45,"context_line":"\"claim\" that would be triggered at the end of the scheduler decision-making"},{"line_number":46,"context_line":"process and which would create Allocation records for the parameterized"},{"line_number":47,"context_line":"ResourceProvider ID, with the slight difference that those allocation records"},{"line_number":48,"context_line":"would have a NULL UUID for the consumer_id field."},{"line_number":49,"context_line":"That POST would return a Claim ID that would be the record ID of a row in a"},{"line_number":50,"context_line":"new mapping table that links to the corresponding allocation entries that"},{"line_number":51,"context_line":"were created."}],"source_content_type":"text/x-rst","patch_set":1,"id":"ffe62b97_d4e4323e","line":48,"updated":"2017-03-29 21:08:00.000000000","message":"Yeah, I\u0027m not understanding why you\u0027d want to do this instead of simply calling POST /allocations with a body something like this:\n\n {\n   \"resources\": {\n     \"VCPU\": 1,\n     \"MEMORY_MB\": 1024,\n     \"DISK_GB\": 100\n   },\n   \"required\": {\n     \"HW_CPU_X86_AVX2\",\n     \"CUSTOM_FOO\"\n   },\n   \"preferred\": {\n     \"CUSTOM_BAR\"\n   },\n   \"constraints\": {\n     \"not_near\": [\n       {\n         \"uuid\": \"another instance UUID\",\n         \"distance\": 1  # Not same compute host...\n        }\n     ],\n     \"near\": [\n        {\n          \"uuid\": \"some volume UUID\",\n          \"distance\": 2  # Same volume aggregate...\n        }\n     ]\n   }\n }\n\nThe placement service would see the above body, see that the body did not contain the \u0027allocations\u0027 top-level dict key (which is the way the existing workflow creates allocations) and would understand that the request is asking to identify the resource providers to place the request to and actually consume the requested resources on those providers, returning the exact same HTTP response as the existing POST /allocations returns:\n\n {\n   \"allocations\": {\n     \u003cCOMPUTE_NODE_RP_UUID\u003e: {\n       \"generation\": 33,\n       \"resources\": {\n         \"VCPU\": 1,\n         \"MEMORY_MB\": 1024\n       }\n     },\n     \u003cSTORAGE_POOL_RP_UUID\u003e: {\n       \"generation\": 18299,\n       \"resource\": {\n         \"DISK_GB\": 100\n       }\n     }\n   }\n }","commit_id":"6b3c00b0673fd4af670fab7e5edff48af4d3111d"},{"author":{"_account_id":11564,"name":"Chris Dent","email":"cdent@anticdent.org","username":"chdent"},"change_message_id":"fb29ca2f3b74955d51a30c86dcaff7061be1a3c6","unresolved":false,"context_lines":[{"line_number":45,"context_line":"\"claim\" that would be triggered at the end of the scheduler decision-making"},{"line_number":46,"context_line":"process and which would create Allocation records for the parameterized"},{"line_number":47,"context_line":"ResourceProvider ID, with the slight difference that those allocation records"},{"line_number":48,"context_line":"would have a NULL UUID for the consumer_id field."},{"line_number":49,"context_line":"That POST would return a Claim ID that would be the record ID of a row in a"},{"line_number":50,"context_line":"new mapping table that links to the corresponding allocation entries that"},{"line_number":51,"context_line":"were created."}],"source_content_type":"text/x-rst","patch_set":1,"id":"ffe62b97_a49ef8b5","line":48,"in_reply_to":"ffe62b97_36510264","updated":"2017-03-30 15:19:19.000000000","message":"Jay, I must be missing something that\u0027s in your mental model but not mine.\n\nGiven the current state of the placement service, if:\n\n* there is a deployment with 1000 compute nodes with nothing on them, each with plenty of ram and disk and vcpu\n* if I make a POST to /allocations as you\u0027ve described for a host with some ram, disk and cpu, since I cannot express all current filters and weighersI either\n    * need to return (and thus allocate according to your model) all or many resource providers (so they can be filtered), resulting in many spurious allocations\n    * need to return one or few resource providers (to avoid spurious allocations) but risk needing to get another list of resource providers because the filters might turn this list I just got into 0\n\nSo something is missing. We need to get us all on the same page so we can figure out that is.","commit_id":"6b3c00b0673fd4af670fab7e5edff48af4d3111d"},{"author":{"_account_id":7,"name":"Jay Pipes","email":"jaypipes@gmail.com","username":"jaypipes"},"change_message_id":"3e7f780b329d0aaff310b3d17a874ed87d4c8811","unresolved":false,"context_lines":[{"line_number":45,"context_line":"\"claim\" that would be triggered at the end of the scheduler decision-making"},{"line_number":46,"context_line":"process and which would create Allocation records for the parameterized"},{"line_number":47,"context_line":"ResourceProvider ID, with the slight difference that those allocation records"},{"line_number":48,"context_line":"would have a NULL UUID for the consumer_id field."},{"line_number":49,"context_line":"That POST would return a Claim ID that would be the record ID of a row in a"},{"line_number":50,"context_line":"new mapping table that links to the corresponding allocation entries that"},{"line_number":51,"context_line":"were created."}],"source_content_type":"text/x-rst","patch_set":1,"id":"ffe62b97_36510264","line":48,"in_reply_to":"ffe62b97_b1ebeada","updated":"2017-03-30 14:50:56.000000000","message":"@cdent, @edleafe: I see no point adding in another step of getting a \"reservation ticket\" or a \"claim #\". It\u0027s just unnecessary complication IMHO. Better to just do the allocation as I\u0027ve indicated, run any filters that cannot be (yet) handled by the placement API in nova-scheduler\u0027s filters and simply DELETE /allocations/{consumer_id} if those filters end up eliminating the chosen resource provider(s) the placement API allocated against.\n\nIf anything, the flow you indicate above introduces *more* race conditions than either the existing code or my proposed \"just do the allocation\" method.","commit_id":"6b3c00b0673fd4af670fab7e5edff48af4d3111d"},{"author":{"_account_id":1063,"name":"Ed Leafe","email":"ed@leafe.com","username":"ed-leafe"},"change_message_id":"b22619229aa28a2f31006831226502e7cd75a3ef","unresolved":false,"context_lines":[{"line_number":45,"context_line":"\"claim\" that would be triggered at the end of the scheduler decision-making"},{"line_number":46,"context_line":"process and which would create Allocation records for the parameterized"},{"line_number":47,"context_line":"ResourceProvider ID, with the slight difference that those allocation records"},{"line_number":48,"context_line":"would have a NULL UUID for the consumer_id field."},{"line_number":49,"context_line":"That POST would return a Claim ID that would be the record ID of a row in a"},{"line_number":50,"context_line":"new mapping table that links to the corresponding allocation entries that"},{"line_number":51,"context_line":"were created."}],"source_content_type":"text/x-rst","patch_set":1,"id":"ffe62b97_8e8d5833","line":48,"in_reply_to":"ffe62b97_b1ebeada","updated":"2017-03-30 13:55:05.000000000","message":"Agree with Chris here. Placement isn\u0027t able (yet) to make a selection; now it only finds a groups of RPs that meet some basic criteria in the request. And once placement *can* make the full selection, the scheduler can go away: the conductor can make the request to placement directly.","commit_id":"6b3c00b0673fd4af670fab7e5edff48af4d3111d"},{"author":{"_account_id":11564,"name":"Chris Dent","email":"cdent@anticdent.org","username":"chdent"},"change_message_id":"bf4c4e6a83c4eee90425f3ebf79e44d74821cfa3","unresolved":false,"context_lines":[{"line_number":45,"context_line":"\"claim\" that would be triggered at the end of the scheduler decision-making"},{"line_number":46,"context_line":"process and which would create Allocation records for the parameterized"},{"line_number":47,"context_line":"ResourceProvider ID, with the slight difference that those allocation records"},{"line_number":48,"context_line":"would have a NULL UUID for the consumer_id field."},{"line_number":49,"context_line":"That POST would return a Claim ID that would be the record ID of a row in a"},{"line_number":50,"context_line":"new mapping table that links to the corresponding allocation entries that"},{"line_number":51,"context_line":"were created."}],"source_content_type":"text/x-rst","patch_set":1,"id":"ffe62b97_b1ebeada","line":48,"in_reply_to":"ffe62b97_d4e4323e","updated":"2017-03-30 10:32:22.000000000","message":"Jay, isn\u0027t this only possible once we have the functionality of all the filters and weighers presents on the placement side?\n\nSince we don\u0027t, there needs to be some kind of interim or evolutionary step that needs to happen which is effectively a reservation or \"pre-claim\".\n\nRather than creating yet another URI [1] in the service I would prefer for us to use a derivation of existing resources and have what could be called reserving allocation. It still counts against inventory (thus providing the \"placement is actually doing something about retries\" value that people seem to want soon) but has the capacity to be expired (either manually, automatically or both) if not verified within a TTL.\n\nThen the flow in the scheduler is:\n\n* GET a list of valid resource providers\n* filter them\n* if any are left, make a reserving allocation\n* place the instance\n* success? let the resource tracker validate the allocation (by PUTting it again, without what identifies a \"reserving allocation\")\n* fail? DELETE the allocation\n* meanwhile, periodic job is clearing reserving allocations that are out of date\n\nThere looks to be some confusing raciness in this.\n\n[1] last paragraph of this section https://docs.openstack.org/developer/nova/placement_dev.html#big-picture","commit_id":"6b3c00b0673fd4af670fab7e5edff48af4d3111d"},{"author":{"_account_id":5754,"name":"Alex Xu","email":"hejie.xu@intel.com","username":"xuhj"},"change_message_id":"2a5523d362aed40f360fb3b4be6feb32426056a9","unresolved":false,"context_lines":[{"line_number":53,"context_line":"That claim ID that the scheduler would get on that POST /claims query would"},{"line_number":54,"context_line":"be passed over the wire eventually down to the compute so that the latter would"},{"line_number":55,"context_line":"PUT or DELETE (pick your preference) /claims so it would trigger updating the"},{"line_number":56,"context_line":"corresponding allocation entries with the right consumer_id values."},{"line_number":57,"context_line":""},{"line_number":58,"context_line":"Alternatives"},{"line_number":59,"context_line":"------------"}],"source_content_type":"text/x-rst","patch_set":1,"id":"1a1ced50_116e422f","line":56,"updated":"2017-03-20 10:02:48.000000000","message":"From the definition of the \u0027POST /claims\u0027, the resource provider uuid is required in the request. That means the scheduler still needs to call \u0027GET /resource_providers?resources\u003dxxx\u0027? So the scheduler is responsible for the retry logic?","commit_id":"6b3c00b0673fd4af670fab7e5edff48af4d3111d"},{"author":{"_account_id":782,"name":"John Garbutt","email":"john@johngarbutt.com","username":"johngarbutt"},"change_message_id":"3bb449d7c1aa88972567d92e98d8d477d55cd3dc","unresolved":false,"context_lines":[{"line_number":54,"context_line":"be passed over the wire eventually down to the compute so that the latter would"},{"line_number":55,"context_line":"PUT or DELETE (pick your preference) /claims so it would trigger updating the"},{"line_number":56,"context_line":"corresponding allocation entries with the right consumer_id values."},{"line_number":57,"context_line":""},{"line_number":58,"context_line":"Alternatives"},{"line_number":59,"context_line":"------------"},{"line_number":60,"context_line":""}],"source_content_type":"text/x-rst","patch_set":1,"id":"ffe62b97_95f720d4","line":57,"updated":"2017-03-28 09:12:07.000000000","message":"Lets me less API-centric here I think. Surely the flow is something like this:\n\n* fetch candidate hosts from placement\n* apply filters and weights on Nova specific data\n* Aquire a \"claim\" for the preferred host\n* If fail to get a claim, go back to fetch new list of candidate hosts, up to 10 times?\n* double check we are not over quota now (once we use placement to check quota)\n* send build request to chosen host\n* if we trigger a retry, need to drop the old claim, and start the whole process again (which could trigger a quota error?)\n\nIf we look deeper into the \"claim\" there are some questions here:\n\n* How does a claim relate to the allocation updated by the resource tracker?\n* Presumably the resource tracker shouldn\u0027t ever update or change any of the \"incoming\" or \"transient\" allocations, until the instance is building on the host?\n* Should we fail for some reason (rabbit failure), does the \"transient\" allocation expire at some point? Who does the expiry, the resource tracker?\n* How do we decide if a claim should fail. Is just the server doesn\u0027t fit? Or do we fail if the generation id is different to what we got when we first listed the candidate hosts from placement? Maybe that needs to be configurable. I think we need functional tests to decide the way forward here.\n\nThere are some tricky parts in the above story that we don\u0027t appear to discuss here.","commit_id":"6b3c00b0673fd4af670fab7e5edff48af4d3111d"},{"author":{"_account_id":1063,"name":"Ed Leafe","email":"ed@leafe.com","username":"ed-leafe"},"change_message_id":"6fb4750207720da4dfa86bef4259935b4ffd639c","unresolved":false,"context_lines":[{"line_number":54,"context_line":"be passed over the wire eventually down to the compute so that the latter would"},{"line_number":55,"context_line":"PUT or DELETE (pick your preference) /claims so it would trigger updating the"},{"line_number":56,"context_line":"corresponding allocation entries with the right consumer_id values."},{"line_number":57,"context_line":""},{"line_number":58,"context_line":"Alternatives"},{"line_number":59,"context_line":"------------"},{"line_number":60,"context_line":""}],"source_content_type":"text/x-rst","patch_set":1,"id":"ffe62b97_d5e8c57c","line":57,"in_reply_to":"ffe62b97_95f720d4","updated":"2017-03-29 17:31:05.000000000","message":"I envision a slightly different set of steps. After the scheduler filtering/weighing, it has an ordered list of acceptable hosts. It then tries to claim the resources of the first host. If that fails, instead of fetching a new list of candidate hosts, it simply tries to claim the next host on its list. Only when the list is exhausted will it re-fetch candidate hosts.","commit_id":"6b3c00b0673fd4af670fab7e5edff48af4d3111d"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"df806fbf97113af207ca3ce1b50d6fc19ec4fd21","unresolved":false,"context_lines":[{"line_number":54,"context_line":"be passed over the wire eventually down to the compute so that the latter would"},{"line_number":55,"context_line":"PUT or DELETE (pick your preference) /claims so it would trigger updating the"},{"line_number":56,"context_line":"corresponding allocation entries with the right consumer_id values."},{"line_number":57,"context_line":""},{"line_number":58,"context_line":"Alternatives"},{"line_number":59,"context_line":"------------"},{"line_number":60,"context_line":""}],"source_content_type":"text/x-rst","patch_set":1,"id":"ffe62b97_2d41daa9","line":57,"in_reply_to":"ffe62b97_d5e8c57c","updated":"2017-03-29 18:07:09.000000000","message":"I agree with Ed on the subitem for iterating hosts for claims.\n\nAlso agree with some of the questions John has, there are missing details here on the actual operational flow here, including what errors can occur throughout the process and what actions are taken when each error occurs.","commit_id":"6b3c00b0673fd4af670fab7e5edff48af4d3111d"},{"author":{"_account_id":11564,"name":"Chris Dent","email":"cdent@anticdent.org","username":"chdent"},"change_message_id":"5a732ebe0d1fc81578cfce5ac51848b0d145de47","unresolved":false,"context_lines":[{"line_number":59,"context_line":"------------"},{"line_number":60,"context_line":""},{"line_number":61,"context_line":"We could consider that Placement doesn\u0027t need to know the on-going resource"},{"line_number":62,"context_line":"creations and that it is only a capacity issue."},{"line_number":63,"context_line":""},{"line_number":64,"context_line":"Data model impact"},{"line_number":65,"context_line":"-----------------"}],"source_content_type":"text/x-rst","patch_set":1,"id":"ba2be162_58d914f2","line":62,"updated":"2017-03-06 14:11:19.000000000","message":"In the past when we\u0027ve talked about making \"claims\" from the scheduler it has been via making POSTs to /allocations, not to a new URL, what\u0027s the value/point of a different URL here?\n\nIt would be okay for a POST to /allocations to return a Location to /claims/{uuid}, as the resource against which to assert the claim is fully claimed or dropped.","commit_id":"6b3c00b0673fd4af670fab7e5edff48af4d3111d"},{"author":{"_account_id":1063,"name":"Ed Leafe","email":"ed@leafe.com","username":"ed-leafe"},"change_message_id":"6fb4750207720da4dfa86bef4259935b4ffd639c","unresolved":false,"context_lines":[{"line_number":59,"context_line":"------------"},{"line_number":60,"context_line":""},{"line_number":61,"context_line":"We could consider that Placement doesn\u0027t need to know the on-going resource"},{"line_number":62,"context_line":"creations and that it is only a capacity issue."},{"line_number":63,"context_line":""},{"line_number":64,"context_line":"Data model impact"},{"line_number":65,"context_line":"-----------------"}],"source_content_type":"text/x-rst","patch_set":1,"id":"ffe62b97_7bf43ff1","line":62,"in_reply_to":"ba2be162_58d914f2","updated":"2017-03-29 17:31:05.000000000","message":"I believe that the idea is indeed to be able to rollback all the allocations if the build fails. But this approach will require placement to keep a record of all Claim IDs and their associated resources, and it would be nice to also have a way to delete that information when the build succeeds. That will be tougher to do, since the claim would come from the scheduler, but the rollback/commit would have to come from conductor.","commit_id":"6b3c00b0673fd4af670fab7e5edff48af4d3111d"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"df806fbf97113af207ca3ce1b50d6fc19ec4fd21","unresolved":false,"context_lines":[{"line_number":59,"context_line":"------------"},{"line_number":60,"context_line":""},{"line_number":61,"context_line":"We could consider that Placement doesn\u0027t need to know the on-going resource"},{"line_number":62,"context_line":"creations and that it is only a capacity issue."},{"line_number":63,"context_line":""},{"line_number":64,"context_line":"Data model impact"},{"line_number":65,"context_line":"-----------------"}],"source_content_type":"text/x-rst","patch_set":1,"id":"ffe62b97_cdddcedd","line":62,"in_reply_to":"ffe62b97_7bf43ff1","updated":"2017-03-29 18:07:09.000000000","message":"Do we really want a new claims resource at all? I would think we\u0027d create the allocation record(s) as Chris pointed out, and then update those later with the actual instance allocation in the RT on a successful build/move operation. On failure, we\u0027d delete the allocation. Having a separate claims resource makes me think of quota reservations too much which helped contribute to the over-complication of the quota system. What is in this spec all sounds like our quota system, which we shouldn\u0027t redo in placement for claims.","commit_id":"6b3c00b0673fd4af670fab7e5edff48af4d3111d"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"3bd248ffd46db4c86b452aa40516154d214243d3","unresolved":false,"context_lines":[{"line_number":59,"context_line":"------------"},{"line_number":60,"context_line":""},{"line_number":61,"context_line":"We could consider that Placement doesn\u0027t need to know the on-going resource"},{"line_number":62,"context_line":"creations and that it is only a capacity issue."},{"line_number":63,"context_line":""},{"line_number":64,"context_line":"Data model impact"},{"line_number":65,"context_line":"-----------------"}],"source_content_type":"text/x-rst","patch_set":1,"id":"ffe62b97_a3d5f91d","line":62,"in_reply_to":"ffe62b97_cdddcedd","updated":"2017-03-29 18:36:47.000000000","message":"To summarize some of what I talked about in IRC for this, I would think we could do something like this:\n\n1. Conductor asks scheduler for a host.\n2. Scheduler asks placement for available resource providers, gets the list and filters them down to a subset, and then picks one, host1.\n3. We make a \u0027claim\u0027 on that host by having the scheduler create the allocation records for the resource provider for host1. The BuildRequest.uuid is the same as the eventual instance.uuid in the cell, so we can pass that uuid from conductor to scheduler in step 1. That is the consumer_id used for the allocations.\n4. Scheduler returns host1 to conductor.\n5. Conductor looks up the cell mapping for host1 and creates the instance in that cell, and deletes the BuildRequest.\n6. Conductor casts the build to the compute host.\n7.1. If everything is good, we have nothing else to do, the allocations are already done.\n7.2. If something fails, we retry (compute casts to conductor to rebuild on another host). During the retry operation, we delete the allocations from the RP for host1 and then make new allocations on the RP for host2, and repeat steps 4-7 until we either succeed or can\u0027t retry anymore and fail with NoValidHost.","commit_id":"6b3c00b0673fd4af670fab7e5edff48af4d3111d"},{"author":{"_account_id":5754,"name":"Alex Xu","email":"hejie.xu@intel.com","username":"xuhj"},"change_message_id":"2a5523d362aed40f360fb3b4be6feb32426056a9","unresolved":false,"context_lines":[{"line_number":101,"context_line":""},{"line_number":102,"context_line":"    {"},{"line_number":103,"context_line":"        \"resource_provider\": {"},{"line_number":104,"context_line":"          \"uuid\": \"7dd1cd8a-4058-4f58-a24a-e38a5f4d563e\""},{"line_number":105,"context_line":"        },"},{"line_number":106,"context_line":"        \"resources\": {"},{"line_number":107,"context_line":"          \"VCPU\": 2,"}],"source_content_type":"text/x-rst","patch_set":1,"id":"1a1ced50_b1072e80","line":104,"range":{"start_line":104,"start_character":19,"end_line":104,"end_character":55},"updated":"2017-03-20 10:02:48.000000000","message":"So this means that the scheduler still needs to call \u0027GET /resource_providers?resources\u003dxxx\u0027 to get the resource provider which matches the request first?\n\nIf yes, there will be a race between \u0027GET /resource_providers?resources\u003dxxx\u0027 and \u0027POST /claims\u0027, to get rid of the race, we need the \u0027generation\u0027 at here.","commit_id":"6b3c00b0673fd4af670fab7e5edff48af4d3111d"},{"author":{"_account_id":1063,"name":"Ed Leafe","email":"ed@leafe.com","username":"ed-leafe"},"change_message_id":"6fb4750207720da4dfa86bef4259935b4ffd639c","unresolved":false,"context_lines":[{"line_number":101,"context_line":""},{"line_number":102,"context_line":"    {"},{"line_number":103,"context_line":"        \"resource_provider\": {"},{"line_number":104,"context_line":"          \"uuid\": \"7dd1cd8a-4058-4f58-a24a-e38a5f4d563e\""},{"line_number":105,"context_line":"        },"},{"line_number":106,"context_line":"        \"resources\": {"},{"line_number":107,"context_line":"          \"VCPU\": 2,"}],"source_content_type":"text/x-rst","patch_set":1,"id":"ffe62b97_dbbeabd3","line":104,"range":{"start_line":104,"start_character":19,"end_line":104,"end_character":55},"in_reply_to":"1a1ced50_b1072e80","updated":"2017-03-29 17:31:05.000000000","message":"The scheduler already has the RP ID (from the selected host), the generation (from the selected host), and the resource amounts (from the request). So I don\u0027t see any additional calls required. You\u0027re correct that the claim would have to include the generation, though, to avoid races.","commit_id":"6b3c00b0673fd4af670fab7e5edff48af4d3111d"},{"author":{"_account_id":11564,"name":"Chris Dent","email":"cdent@anticdent.org","username":"chdent"},"change_message_id":"5a732ebe0d1fc81578cfce5ac51848b0d145de47","unresolved":false,"context_lines":[{"line_number":146,"context_line":"pointing to the created claim::"},{"line_number":147,"context_line":""},{"line_number":148,"context_line":"    201 Created"},{"line_number":149,"context_line":"    Location: /claims/eaaf1c04-ced2-40e4-89a2-87edded06d64"},{"line_number":150,"context_line":""},{"line_number":151,"context_line":""},{"line_number":152,"context_line":"`PUT /claims`"}],"source_content_type":"text/x-rst","patch_set":1,"id":"ba2be162_98c2ac51","line":149,"updated":"2017-03-06 14:11:19.000000000","message":"Since you create a URL here","commit_id":"6b3c00b0673fd4af670fab7e5edff48af4d3111d"},{"author":{"_account_id":11564,"name":"Chris Dent","email":"cdent@anticdent.org","username":"chdent"},"change_message_id":"5a732ebe0d1fc81578cfce5ac51848b0d145de47","unresolved":false,"context_lines":[{"line_number":149,"context_line":"    Location: /claims/eaaf1c04-ced2-40e4-89a2-87edded06d64"},{"line_number":150,"context_line":""},{"line_number":151,"context_line":""},{"line_number":152,"context_line":"`PUT /claims`"},{"line_number":153,"context_line":"**************************"},{"line_number":154,"context_line":""},{"line_number":155,"context_line":"Updates one claim."}],"source_content_type":"text/x-rst","patch_set":1,"id":"ba2be162_d8a6e4bf","line":152,"updated":"2017-03-06 14:11:19.000000000","message":"may as well use it here, making the PUT to a singular resource","commit_id":"6b3c00b0673fd4af670fab7e5edff48af4d3111d"},{"author":{"_account_id":11564,"name":"Chris Dent","email":"cdent@anticdent.org","username":"chdent"},"change_message_id":"5a732ebe0d1fc81578cfce5ac51848b0d145de47","unresolved":false,"context_lines":[{"line_number":160,"context_line":""},{"line_number":161,"context_line":"    {"},{"line_number":162,"context_line":"        \"claim\": {"},{"line_number":163,"context_line":"          \"uuid\": \"7dd1cd8a-4058-4f58-a24a-e38a5f4d563e\""},{"line_number":164,"context_line":"          \"consumer_id\": \"eaaf1c04-ced2-40e4-89a2-87edded06d64\""},{"line_number":165,"context_line":"        }"},{"line_number":166,"context_line":"    }"}],"source_content_type":"text/x-rst","patch_set":1,"id":"ba2be162_1884fc08","line":163,"updated":"2017-03-06 14:11:19.000000000","message":"which would allow this field to not be here","commit_id":"6b3c00b0673fd4af670fab7e5edff48af4d3111d"},{"author":{"_account_id":5754,"name":"Alex Xu","email":"hejie.xu@intel.com","username":"xuhj"},"change_message_id":"2a5523d362aed40f360fb3b4be6feb32426056a9","unresolved":false,"context_lines":[{"line_number":161,"context_line":"    {"},{"line_number":162,"context_line":"        \"claim\": {"},{"line_number":163,"context_line":"          \"uuid\": \"7dd1cd8a-4058-4f58-a24a-e38a5f4d563e\""},{"line_number":164,"context_line":"          \"consumer_id\": \"eaaf1c04-ced2-40e4-89a2-87edded06d64\""},{"line_number":165,"context_line":"        }"},{"line_number":166,"context_line":"    }"},{"line_number":167,"context_line":""}],"source_content_type":"text/x-rst","patch_set":1,"id":"1a1ced50_3d5a3551","line":164,"updated":"2017-03-20 10:02:48.000000000","message":"you already have \u0027uuid\u0027, why we need another consumer_id?","commit_id":"6b3c00b0673fd4af670fab7e5edff48af4d3111d"},{"author":{"_account_id":11564,"name":"Chris Dent","email":"cdent@anticdent.org","username":"chdent"},"change_message_id":"5a732ebe0d1fc81578cfce5ac51848b0d145de47","unresolved":false,"context_lines":[{"line_number":193,"context_line":"The returned HTTP response code will be one of the following:"},{"line_number":194,"context_line":""},{"line_number":195,"context_line":"    * `200 OK` if the claim record is successfully updated"},{"line_number":196,"context_line":"    * `400 Bad Request` for bad or invalid syntax"},{"line_number":197,"context_line":""},{"line_number":198,"context_line":""},{"line_number":199,"context_line":"Security impact"}],"source_content_type":"text/x-rst","patch_set":1,"id":"ba2be162_b8b850c9","line":196,"updated":"2017-03-06 14:11:19.000000000","message":"409 in the case where the claim is no longer available to be claimed (which wouldn\u0027t normally happen but should be protected against)","commit_id":"6b3c00b0673fd4af670fab7e5edff48af4d3111d"},{"author":{"_account_id":782,"name":"John Garbutt","email":"john@johngarbutt.com","username":"johngarbutt"},"change_message_id":"3bb449d7c1aa88972567d92e98d8d477d55cd3dc","unresolved":false,"context_lines":[{"line_number":255,"context_line":"Testing"},{"line_number":256,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d"},{"line_number":257,"context_line":""},{"line_number":258,"context_line":"Nothing really fancy new, classic coverage of unit and functional tests."},{"line_number":259,"context_line":""},{"line_number":260,"context_line":"Documentation Impact"},{"line_number":261,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"}],"source_content_type":"text/x-rst","patch_set":1,"id":"ffe62b97_d5ada89b","line":258,"updated":"2017-03-28 09:12:07.000000000","message":"I think we do need something new here. Can we have a performance related functional test to prove that the changes we make somehow improve the through put or accuracy of the placement decisions?\n\nI.e. if you do 1000s requests all at once, how many fail because we retry too many times and fail to find a slot. Maybe lets say there is room for 1100 instances, spread over 100 hosts, a few VMs (50?) already in the system, or something like that?\n\nIdeally we find something that is failing to place everything today, and we see that slowly improve as we try out some new ideas (be that in success rate, speed and accuracy).\n\nFWIW, I did start to take a look at a small slice of that in here:\nhttps://github.com/openstack/nova/blob/master/nova/tests/unit/scheduler/test_caching_scheduler.py#L209\nThe above was purely looking at the filter and weights system, and where we were spending lots of time. FWIW, turns out it was the logging that took most of the time.","commit_id":"6b3c00b0673fd4af670fab7e5edff48af4d3111d"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"df806fbf97113af207ca3ce1b50d6fc19ec4fd21","unresolved":false,"context_lines":[{"line_number":255,"context_line":"Testing"},{"line_number":256,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d"},{"line_number":257,"context_line":""},{"line_number":258,"context_line":"Nothing really fancy new, classic coverage of unit and functional tests."},{"line_number":259,"context_line":""},{"line_number":260,"context_line":"Documentation Impact"},{"line_number":261,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"}],"source_content_type":"text/x-rst","patch_set":1,"id":"ffe62b97_8d1fc66a","line":258,"in_reply_to":"ffe62b97_d5ada89b","updated":"2017-03-29 18:07:09.000000000","message":"Yeah we definitely need a test plan here I think, especially for making sure we test error conditions and how we recover from them.\n\nAlso, we\u0027ll need to test both packing and spreading. host_subset_size determines pack vs spread and defaults to 1 for packing. So we\u0027d need tests that set this to something greater than 1, for example.","commit_id":"6b3c00b0673fd4af670fab7e5edff48af4d3111d"},{"author":{"_account_id":7,"name":"Jay Pipes","email":"jaypipes@gmail.com","username":"jaypipes"},"change_message_id":"252b398ac0e381a0d5241e5fa1269c17f38f86b4","unresolved":false,"context_lines":[{"line_number":11,"context_line":"https://blueprints.launchpad.net/nova/+spec/placement-claims"},{"line_number":12,"context_line":""},{"line_number":13,"context_line":"Have intermediate locking of resources so the Placement API could return more"},{"line_number":14,"context_line":"accurate ResourceProviders on a query for a specific set of resource amounts."},{"line_number":15,"context_line":""},{"line_number":16,"context_line":"Problem description"},{"line_number":17,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"}],"source_content_type":"text/x-rst","patch_set":2,"id":"dfeb2761_3d54ca20","line":14,"updated":"2017-04-04 15:32:48.000000000","message":"\"Have intermediate locking of resources\" isn\u0027t really what this spec proposes. What this spec proposes is simply: \"Have the Nova scheduler create allocation records in the placement API after selecting a compute host.\"","commit_id":"db981b08ea83dbd87fe21d8633d6f75e857a83b9"},{"author":{"_account_id":11564,"name":"Chris Dent","email":"cdent@anticdent.org","username":"chdent"},"change_message_id":"2b7b01b8efa540edd7218403750c3d517fce1c19","unresolved":false,"context_lines":[{"line_number":11,"context_line":"https://blueprints.launchpad.net/nova/+spec/placement-claims"},{"line_number":12,"context_line":""},{"line_number":13,"context_line":"Have intermediate locking of resources so the Placement API could return more"},{"line_number":14,"context_line":"accurate ResourceProviders on a query for a specific set of resource amounts."},{"line_number":15,"context_line":""},{"line_number":16,"context_line":"Problem description"},{"line_number":17,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"}],"source_content_type":"text/x-rst","patch_set":2,"id":"dfeb2761_8b8a71bc","line":14,"updated":"2017-04-04 14:40:49.000000000","message":"not that it really matters, but  would prefere we use the phrase \"resource providers\" rather than \"ResourceProviders\" because the latter looks like a class of object and that\u0027s not what we return. Using that term makes it easy to think that the interface to placement are the nova.objects that are on the back end.\n\nBut to repeat: not that it really matters.","commit_id":"db981b08ea83dbd87fe21d8633d6f75e857a83b9"},{"author":{"_account_id":7,"name":"Jay Pipes","email":"jaypipes@gmail.com","username":"jaypipes"},"change_message_id":"252b398ac0e381a0d5241e5fa1269c17f38f86b4","unresolved":false,"context_lines":[{"line_number":21,"context_line":"classes. Currently, the ResourceTracker (RT) in the nova-compute service is"},{"line_number":22,"context_line":"responsible for updating inventory and allocation values for resource"},{"line_number":23,"context_line":"providers, e.g. compute nodes, in the Placement service. The RT is also"},{"line_number":24,"context_line":"responsible for making claims during server build and move operations."},{"line_number":25,"context_line":""},{"line_number":26,"context_line":"While this model fits very well for a cloud having lots of capacity left, we"},{"line_number":27,"context_line":"know that this can be suboptimal because of requests concurrency. Somehow, we"}],"source_content_type":"text/x-rst","patch_set":2,"id":"dfeb2761_1d334616","line":24,"updated":"2017-04-04 15:32:48.000000000","message":"Recommend replacing the last sentence with \"The process of updating allocation values in the RT is called a \u0027claim\u0027.\"","commit_id":"db981b08ea83dbd87fe21d8633d6f75e857a83b9"},{"author":{"_account_id":7,"name":"Jay Pipes","email":"jaypipes@gmail.com","username":"jaypipes"},"change_message_id":"252b398ac0e381a0d5241e5fa1269c17f38f86b4","unresolved":false,"context_lines":[{"line_number":26,"context_line":"While this model fits very well for a cloud having lots of capacity left, we"},{"line_number":27,"context_line":"know that this can be suboptimal because of requests concurrency. Somehow, we"},{"line_number":28,"context_line":"would like to make sure that what the Placement API returns is based on not"},{"line_number":29,"context_line":"only what is existing, but also what is currently *subject to exist*."},{"line_number":30,"context_line":""},{"line_number":31,"context_line":"To his merit, Mr. Jay Pipes carries the idea to have the scheduler claiming"},{"line_number":32,"context_line":"a destination by posting a list of allocations to the Placement API and passing"}],"source_content_type":"text/x-rst","patch_set":2,"id":"dfeb2761_fd3782fe","line":29,"updated":"2017-04-04 15:32:48.000000000","message":"Recommend removing the above paragraph. It really doesn\u0027t make much sense to me and is just confusing.","commit_id":"db981b08ea83dbd87fe21d8633d6f75e857a83b9"},{"author":{"_account_id":7,"name":"Jay Pipes","email":"jaypipes@gmail.com","username":"jaypipes"},"change_message_id":"252b398ac0e381a0d5241e5fa1269c17f38f86b4","unresolved":false,"context_lines":[{"line_number":28,"context_line":"would like to make sure that what the Placement API returns is based on not"},{"line_number":29,"context_line":"only what is existing, but also what is currently *subject to exist*."},{"line_number":30,"context_line":""},{"line_number":31,"context_line":"To his merit, Mr. Jay Pipes carries the idea to have the scheduler claiming"},{"line_number":32,"context_line":"a destination by posting a list of allocations to the Placement API and passing"},{"line_number":33,"context_line":"the UUIDs back to the conductor so later in the boot process, a compute service"},{"line_number":34,"context_line":"could delete those allocations if the spawn sequence had problems."}],"source_content_type":"text/x-rst","patch_set":2,"id":"dfeb2761_dd383eef","line":31,"updated":"2017-04-04 15:32:48.000000000","message":"Replace \"To his merit, Mr. Jay Pipes carries the idea to have the scheduler claiming a destination by posting a list of allocations ...\" with \"In this spec, we propose to have the scheduler allocate resources instead of the compute node.\"","commit_id":"db981b08ea83dbd87fe21d8633d6f75e857a83b9"},{"author":{"_account_id":7,"name":"Jay Pipes","email":"jaypipes@gmail.com","username":"jaypipes"},"change_message_id":"252b398ac0e381a0d5241e5fa1269c17f38f86b4","unresolved":false,"context_lines":[{"line_number":31,"context_line":"To his merit, Mr. Jay Pipes carries the idea to have the scheduler claiming"},{"line_number":32,"context_line":"a destination by posting a list of allocations to the Placement API and passing"},{"line_number":33,"context_line":"the UUIDs back to the conductor so later in the boot process, a compute service"},{"line_number":34,"context_line":"could delete those allocations if the spawn sequence had problems."},{"line_number":35,"context_line":"Given the Placement API is not feature-parity with all the scheduler filters"},{"line_number":36,"context_line":"and weighters, the trade-off is that the scheduler would first get a list of"},{"line_number":37,"context_line":"resource providers, iterate over those with the help of filters and weighters"}],"source_content_type":"text/x-rst","patch_set":2,"id":"dfeb2761_bd3d7ade","line":34,"updated":"2017-04-04 15:32:48.000000000","message":"The latter part of the above sentence can be boiled down to: \"Compute hosts would only communicate with the scheduler and placement API if a fatal error occurred launching the instance.\"","commit_id":"db981b08ea83dbd87fe21d8633d6f75e857a83b9"},{"author":{"_account_id":7,"name":"Jay Pipes","email":"jaypipes@gmail.com","username":"jaypipes"},"change_message_id":"252b398ac0e381a0d5241e5fa1269c17f38f86b4","unresolved":false,"context_lines":[{"line_number":32,"context_line":"a destination by posting a list of allocations to the Placement API and passing"},{"line_number":33,"context_line":"the UUIDs back to the conductor so later in the boot process, a compute service"},{"line_number":34,"context_line":"could delete those allocations if the spawn sequence had problems."},{"line_number":35,"context_line":"Given the Placement API is not feature-parity with all the scheduler filters"},{"line_number":36,"context_line":"and weighters, the trade-off is that the scheduler would first get a list of"},{"line_number":37,"context_line":"resource providers, iterate over those with the help of filters and weighters"},{"line_number":38,"context_line":"and post the allocations against the found target."}],"source_content_type":"text/x-rst","patch_set":2,"id":"dfeb2761_9d2636ce","line":35,"updated":"2017-04-04 15:32:48.000000000","message":"s/is not feature-parity with all/does not yet implement all/","commit_id":"db981b08ea83dbd87fe21d8633d6f75e857a83b9"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"e94275f36495c29dc922629e6b2bc38e485bac62","unresolved":false,"context_lines":[{"line_number":33,"context_line":"the UUIDs back to the conductor so later in the boot process, a compute service"},{"line_number":34,"context_line":"could delete those allocations if the spawn sequence had problems."},{"line_number":35,"context_line":"Given the Placement API is not feature-parity with all the scheduler filters"},{"line_number":36,"context_line":"and weighters, the trade-off is that the scheduler would first get a list of"},{"line_number":37,"context_line":"resource providers, iterate over those with the help of filters and weighters"},{"line_number":38,"context_line":"and post the allocations against the found target."},{"line_number":39,"context_line":"How Placement could be (or even *should* be) feature-parity with all the"}],"source_content_type":"text/x-rst","patch_set":2,"id":"bff0334d_6e5e0613","line":36,"range":{"start_line":36,"start_character":4,"end_line":36,"end_character":13},"updated":"2017-04-10 15:52:53.000000000","message":"weighers","commit_id":"db981b08ea83dbd87fe21d8633d6f75e857a83b9"},{"author":{"_account_id":7,"name":"Jay Pipes","email":"jaypipes@gmail.com","username":"jaypipes"},"change_message_id":"252b398ac0e381a0d5241e5fa1269c17f38f86b4","unresolved":false,"context_lines":[{"line_number":35,"context_line":"Given the Placement API is not feature-parity with all the scheduler filters"},{"line_number":36,"context_line":"and weighters, the trade-off is that the scheduler would first get a list of"},{"line_number":37,"context_line":"resource providers, iterate over those with the help of filters and weighters"},{"line_number":38,"context_line":"and post the allocations against the found target."},{"line_number":39,"context_line":"How Placement could be (or even *should* be) feature-parity with all the"},{"line_number":40,"context_line":"filters/weighters is not intended to be targeted in that spec and we all"},{"line_number":41,"context_line":"acknowledge it\u0027s a very different discussion."}],"source_content_type":"text/x-rst","patch_set":2,"id":"dfeb2761_7d2372be","line":38,"updated":"2017-04-04 15:32:48.000000000","message":"That\u0027s not a trade-off. I would just replace the above sentence with \"In this way, we both reduce the occurrence of retry operations as well as the time period that race conditions can occur.\"","commit_id":"db981b08ea83dbd87fe21d8633d6f75e857a83b9"},{"author":{"_account_id":7,"name":"Jay Pipes","email":"jaypipes@gmail.com","username":"jaypipes"},"change_message_id":"252b398ac0e381a0d5241e5fa1269c17f38f86b4","unresolved":false,"context_lines":[{"line_number":38,"context_line":"and post the allocations against the found target."},{"line_number":39,"context_line":"How Placement could be (or even *should* be) feature-parity with all the"},{"line_number":40,"context_line":"filters/weighters is not intended to be targeted in that spec and we all"},{"line_number":41,"context_line":"acknowledge it\u0027s a very different discussion."},{"line_number":42,"context_line":""},{"line_number":43,"context_line":"Use Cases"},{"line_number":44,"context_line":"---------"}],"source_content_type":"text/x-rst","patch_set":2,"id":"dfeb2761_5d2c2eab","line":41,"updated":"2017-04-04 15:32:48.000000000","message":"Recommend just removing the last sentence.","commit_id":"db981b08ea83dbd87fe21d8633d6f75e857a83b9"},{"author":{"_account_id":11564,"name":"Chris Dent","email":"cdent@anticdent.org","username":"chdent"},"change_message_id":"2b7b01b8efa540edd7218403750c3d517fce1c19","unresolved":false,"context_lines":[{"line_number":38,"context_line":"and post the allocations against the found target."},{"line_number":39,"context_line":"How Placement could be (or even *should* be) feature-parity with all the"},{"line_number":40,"context_line":"filters/weighters is not intended to be targeted in that spec and we all"},{"line_number":41,"context_line":"acknowledge it\u0027s a very different discussion."},{"line_number":42,"context_line":""},{"line_number":43,"context_line":"Use Cases"},{"line_number":44,"context_line":"---------"}],"source_content_type":"text/x-rst","patch_set":2,"id":"dfeb2761_6b5e0d45","line":41,"updated":"2017-04-04 14:40:49.000000000","message":"This paragraph could be refactored a bit to make a bit more sense. I think you\u0027re trying to say that the implementation described here is the way it is because we want to maintain feature parity with existing filters and weighers and that functionality is not present on the placement service side and _may never_ be present, so this is the path we have for now.\n\nI think that can be said more succinctly than you\u0027ve got it above.\n\nIf your\u0027e saying something different from that, then it\u0027s not clear.","commit_id":"db981b08ea83dbd87fe21d8633d6f75e857a83b9"},{"author":{"_account_id":7166,"name":"Sylvain Bauza","email":"sbauza@redhat.com","username":"sbauza"},"change_message_id":"99a74d5201430a64bf5ccc0a852db0b8d8b3b87f","unresolved":false,"context_lines":[{"line_number":38,"context_line":"and post the allocations against the found target."},{"line_number":39,"context_line":"How Placement could be (or even *should* be) feature-parity with all the"},{"line_number":40,"context_line":"filters/weighters is not intended to be targeted in that spec and we all"},{"line_number":41,"context_line":"acknowledge it\u0027s a very different discussion."},{"line_number":42,"context_line":""},{"line_number":43,"context_line":"Use Cases"},{"line_number":44,"context_line":"---------"}],"source_content_type":"text/x-rst","patch_set":2,"id":"dfeb2761_1a4eac96","line":41,"in_reply_to":"dfeb2761_6b5e0d45","updated":"2017-04-04 15:11:06.000000000","message":"Sure, will rewrite it, thanks for clarifying it.","commit_id":"db981b08ea83dbd87fe21d8633d6f75e857a83b9"},{"author":{"_account_id":7,"name":"Jay Pipes","email":"jaypipes@gmail.com","username":"jaypipes"},"change_message_id":"252b398ac0e381a0d5241e5fa1269c17f38f86b4","unresolved":false,"context_lines":[{"line_number":52,"context_line":""},{"line_number":53,"context_line":"First of all, let\u0027s clarify a very important point. The current REST API"},{"line_number":54,"context_line":"provided by the Placement service is rich enough to help our usecase, and we"},{"line_number":55,"context_line":"don\u0027t target to modify its API to fit our needs."},{"line_number":56,"context_line":"Rather, the spec targets to modify when/where allocations are created and"},{"line_number":57,"context_line":"insteaf of letting the RT (see above for the acronym) posting the allocations,"},{"line_number":58,"context_line":"we propose the scheduler to premptively write those allocations and pass them"}],"source_content_type":"text/x-rst","patch_set":2,"id":"dfeb2761_3d296a9a","line":55,"updated":"2017-04-04 15:32:48.000000000","message":"++","commit_id":"db981b08ea83dbd87fe21d8633d6f75e857a83b9"},{"author":{"_account_id":11564,"name":"Chris Dent","email":"cdent@anticdent.org","username":"chdent"},"change_message_id":"2b7b01b8efa540edd7218403750c3d517fce1c19","unresolved":false,"context_lines":[{"line_number":54,"context_line":"provided by the Placement service is rich enough to help our usecase, and we"},{"line_number":55,"context_line":"don\u0027t target to modify its API to fit our needs."},{"line_number":56,"context_line":"Rather, the spec targets to modify when/where allocations are created and"},{"line_number":57,"context_line":"insteaf of letting the RT (see above for the acronym) posting the allocations,"},{"line_number":58,"context_line":"we propose the scheduler to premptively write those allocations and pass them"},{"line_number":59,"context_line":"down to the compute node thanks to the conductor which would get them"},{"line_number":60,"context_line":"when querying the scheduler. That behaviour would only happen if all the"}],"source_content_type":"text/x-rst","patch_set":2,"id":"dfeb2761_e605b412","line":57,"updated":"2017-04-04 14:40:49.000000000","message":"instead\n\n(If you need to add parens to the acronym, and the explanation is longer than the term either don\u0027t use the acronym, or just use the term in the acronym, please.)","commit_id":"db981b08ea83dbd87fe21d8633d6f75e857a83b9"},{"author":{"_account_id":11564,"name":"Chris Dent","email":"cdent@anticdent.org","username":"chdent"},"change_message_id":"2b7b01b8efa540edd7218403750c3d517fce1c19","unresolved":false,"context_lines":[{"line_number":55,"context_line":"don\u0027t target to modify its API to fit our needs."},{"line_number":56,"context_line":"Rather, the spec targets to modify when/where allocations are created and"},{"line_number":57,"context_line":"insteaf of letting the RT (see above for the acronym) posting the allocations,"},{"line_number":58,"context_line":"we propose the scheduler to premptively write those allocations and pass them"},{"line_number":59,"context_line":"down to the compute node thanks to the conductor which would get them"},{"line_number":60,"context_line":"when querying the scheduler. That behaviour would only happen if all the"},{"line_number":61,"context_line":"compute services have a new service version (meaning they are all Pike),"}],"source_content_type":"text/x-rst","patch_set":2,"id":"dfeb2761_863dc8c5","line":58,"updated":"2017-04-04 14:40:49.000000000","message":"preemptively","commit_id":"db981b08ea83dbd87fe21d8633d6f75e857a83b9"},{"author":{"_account_id":11564,"name":"Chris Dent","email":"cdent@anticdent.org","username":"chdent"},"change_message_id":"2b7b01b8efa540edd7218403750c3d517fce1c19","unresolved":false,"context_lines":[{"line_number":60,"context_line":"when querying the scheduler. That behaviour would only happen if all the"},{"line_number":61,"context_line":"compute services have a new service version (meaning they are all Pike),"},{"line_number":62,"context_line":"implying that having both Pike and Ocata compute nodes would mean that we would"},{"line_number":63,"context_line":"keep the legacy behaviour with allocation still set by computes."},{"line_number":64,"context_line":""},{"line_number":65,"context_line":"The conductor would then update the related RequestSpec record by setting"},{"line_number":66,"context_line":"an \"allocations\" field and push the RequestSpec attribute in the RPC methods"}],"source_content_type":"text/x-rst","patch_set":2,"id":"dfeb2761_06865867","line":63,"updated":"2017-04-04 14:40:49.000000000","message":"I don\u0027t understand what \"pass them down to the compute node\" means with any confidence, nor why it is necessary.\n\nIf the scheduler sets the allocations, then isn\u0027t it possible for the resource tracker to simply set them again, effectively confirming them? And if it can\u0027t place it deletes them (as you describe below).\n\nIt\u0027s not clear why we need to carry state via the RequestSpec from the scheduler, to the conductor to the compute node about the allocations? Can you clarify please?","commit_id":"db981b08ea83dbd87fe21d8633d6f75e857a83b9"},{"author":{"_account_id":7,"name":"Jay Pipes","email":"jaypipes@gmail.com","username":"jaypipes"},"change_message_id":"252b398ac0e381a0d5241e5fa1269c17f38f86b4","unresolved":false,"context_lines":[{"line_number":60,"context_line":"when querying the scheduler. That behaviour would only happen if all the"},{"line_number":61,"context_line":"compute services have a new service version (meaning they are all Pike),"},{"line_number":62,"context_line":"implying that having both Pike and Ocata compute nodes would mean that we would"},{"line_number":63,"context_line":"keep the legacy behaviour with allocation still set by computes."},{"line_number":64,"context_line":""},{"line_number":65,"context_line":"The conductor would then update the related RequestSpec record by setting"},{"line_number":66,"context_line":"an \"allocations\" field and push the RequestSpec attribute in the RPC methods"}],"source_content_type":"text/x-rst","patch_set":2,"id":"dfeb2761_1d1c6676","line":63,"updated":"2017-04-04 15:32:48.000000000","message":"The above paragraph starting from \"Rather...\" can be simplified to: \"We will use a service version check in the scheduler to ensure all compute hosts in the system can handle a request that has had allocations already written by the scheduler.\"","commit_id":"db981b08ea83dbd87fe21d8633d6f75e857a83b9"},{"author":{"_account_id":7166,"name":"Sylvain Bauza","email":"sbauza@redhat.com","username":"sbauza"},"change_message_id":"99a74d5201430a64bf5ccc0a852db0b8d8b3b87f","unresolved":false,"context_lines":[{"line_number":60,"context_line":"when querying the scheduler. That behaviour would only happen if all the"},{"line_number":61,"context_line":"compute services have a new service version (meaning they are all Pike),"},{"line_number":62,"context_line":"implying that having both Pike and Ocata compute nodes would mean that we would"},{"line_number":63,"context_line":"keep the legacy behaviour with allocation still set by computes."},{"line_number":64,"context_line":""},{"line_number":65,"context_line":"The conductor would then update the related RequestSpec record by setting"},{"line_number":66,"context_line":"an \"allocations\" field and push the RequestSpec attribute in the RPC methods"}],"source_content_type":"text/x-rst","patch_set":2,"id":"dfeb2761_7a02583f","line":63,"in_reply_to":"dfeb2761_06865867","updated":"2017-04-04 15:11:06.000000000","message":"That\u0027s mostly because we would like to avoid yet another API call, and rather signal to the compute node that we already have set the allocations.\n\nIt\u0027s also important because if we agree (and possibly Matt and Dan could disagree) that computes could potentially still call Placement for deleting the allocations (in case of a problem), how would the compute know which allocations to delete ?","commit_id":"db981b08ea83dbd87fe21d8633d6f75e857a83b9"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"e94275f36495c29dc922629e6b2bc38e485bac62","unresolved":false,"context_lines":[{"line_number":62,"context_line":"implying that having both Pike and Ocata compute nodes would mean that we would"},{"line_number":63,"context_line":"keep the legacy behaviour with allocation still set by computes."},{"line_number":64,"context_line":""},{"line_number":65,"context_line":"The conductor would then update the related RequestSpec record by setting"},{"line_number":66,"context_line":"an \"allocations\" field and push the RequestSpec attribute in the RPC methods"},{"line_number":67,"context_line":"calling the target compute service."},{"line_number":68,"context_line":""},{"line_number":69,"context_line":"Then, the compute service would verify the RequestSpec, and if it\u0027s seeing the"},{"line_number":70,"context_line":"\"allocations\" field set, then the RT would not call the Placement API for"}],"source_content_type":"text/x-rst","patch_set":2,"id":"bff0334d_ce3d12c5","line":67,"range":{"start_line":65,"start_character":0,"end_line":67,"end_character":35},"updated":"2017-04-10 15:52:53.000000000","message":"I think stuffing allocations into the RequestSpec, which is supposed to model the things that the user requested when building their instance, is a hack and cheap convenience to avoid passing a new parameter to the build_and_run_instance RPC method.","commit_id":"db981b08ea83dbd87fe21d8633d6f75e857a83b9"},{"author":{"_account_id":7,"name":"Jay Pipes","email":"jaypipes@gmail.com","username":"jaypipes"},"change_message_id":"252b398ac0e381a0d5241e5fa1269c17f38f86b4","unresolved":false,"context_lines":[{"line_number":68,"context_line":""},{"line_number":69,"context_line":"Then, the compute service would verify the RequestSpec, and if it\u0027s seeing the"},{"line_number":70,"context_line":"\"allocations\" field set, then the RT would not call the Placement API for"},{"line_number":71,"context_line":"adding allocations. If it\u0027s not seeing allocations, then it would do that like"},{"line_number":72,"context_line":"the legacy behaviour."},{"line_number":73,"context_line":""},{"line_number":74,"context_line":"If there is a problem when spawning the instance by the driver, then the"}],"source_content_type":"text/x-rst","patch_set":2,"id":"dfeb2761_fd20a2ae","line":71,"updated":"2017-04-04 15:32:48.000000000","message":"It\u0027s not that the RT would not call the placement API. The report client already checks allocations via a GET /allocations request before writing allocations:\n\nhttps://github.com/openstack/nova/blob/master/nova/scheduler/client/report.py#L795-L798\n\nReally the only thing that would change would be there would be no need to create a Claim object that tracked anything other than those resources that are not (yet) tracked in the placement API -- namely, NUMA topologies and PCI devices.\n\nSo, basically, we\u0027d want to create a new transitional LightClaim object that *only* did the NUMA topology and PCI device \"claiming\" but didn\u0027t bother with the VCPU, memory or disk stuff. And eventually we\u0027d just kill the whole nova.compute.claim module entirely.","commit_id":"db981b08ea83dbd87fe21d8633d6f75e857a83b9"},{"author":{"_account_id":7,"name":"Jay Pipes","email":"jaypipes@gmail.com","username":"jaypipes"},"change_message_id":"252b398ac0e381a0d5241e5fa1269c17f38f86b4","unresolved":false,"context_lines":[{"line_number":73,"context_line":""},{"line_number":74,"context_line":"If there is a problem when spawning the instance by the driver, then the"},{"line_number":75,"context_line":"compute node would call the Placement API by deleting the allocations and"},{"line_number":76,"context_line":"rescheduling the instance to the conductor. It could be preferrable to not"},{"line_number":77,"context_line":"have compute calling Placement API and rather have the scheduler deleting the"},{"line_number":78,"context_line":"allocations at the same place where it created those, but I honestly think it"},{"line_number":79,"context_line":"could be a better performance behaviour if we delete allocations the most"}],"source_content_type":"text/x-rst","patch_set":2,"id":"dfeb2761_dd115e5f","line":76,"updated":"2017-04-04 15:32:48.000000000","message":"Correct.","commit_id":"db981b08ea83dbd87fe21d8633d6f75e857a83b9"},{"author":{"_account_id":7,"name":"Jay Pipes","email":"jaypipes@gmail.com","username":"jaypipes"},"change_message_id":"252b398ac0e381a0d5241e5fa1269c17f38f86b4","unresolved":false,"context_lines":[{"line_number":77,"context_line":"have compute calling Placement API and rather have the scheduler deleting the"},{"line_number":78,"context_line":"allocations at the same place where it created those, but I honestly think it"},{"line_number":79,"context_line":"could be a better performance behaviour if we delete allocations the most"},{"line_number":80,"context_line":"quickly when we know the instance can\u0027t be created."},{"line_number":81,"context_line":""},{"line_number":82,"context_line":"That said, there is a current problem with reschedules for Cells V2 where"},{"line_number":83,"context_line":"the compute calls back conductor. Unfortunately, given it would call a cell"}],"source_content_type":"text/x-rst","patch_set":2,"id":"dfeb2761_bd169a54","line":80,"updated":"2017-04-04 15:32:48.000000000","message":"Recommend removing the last sentence of the above paragraph.","commit_id":"db981b08ea83dbd87fe21d8633d6f75e857a83b9"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"e94275f36495c29dc922629e6b2bc38e485bac62","unresolved":false,"context_lines":[{"line_number":77,"context_line":"have compute calling Placement API and rather have the scheduler deleting the"},{"line_number":78,"context_line":"allocations at the same place where it created those, but I honestly think it"},{"line_number":79,"context_line":"could be a better performance behaviour if we delete allocations the most"},{"line_number":80,"context_line":"quickly when we know the instance can\u0027t be created."},{"line_number":81,"context_line":""},{"line_number":82,"context_line":"That said, there is a current problem with reschedules for Cells V2 where"},{"line_number":83,"context_line":"the compute calls back conductor. Unfortunately, given it would call a cell"}],"source_content_type":"text/x-rst","patch_set":2,"id":"bff0334d_2e7d9e7e","line":80,"in_reply_to":"dfeb2761_bd169a54","updated":"2017-04-10 15:52:53.000000000","message":"I tend to think we need to discuss that alternative, i.e. who deletes the allocation in the case of a failure and a retry (we can\u0027t even do retries with super-conductor). So I\u0027m good with leaving it but eventually documenting the decision once we have a decision and explaining why we went for one over the other.","commit_id":"db981b08ea83dbd87fe21d8633d6f75e857a83b9"},{"author":{"_account_id":7,"name":"Jay Pipes","email":"jaypipes@gmail.com","username":"jaypipes"},"change_message_id":"252b398ac0e381a0d5241e5fa1269c17f38f86b4","unresolved":false,"context_lines":[{"line_number":83,"context_line":"the compute calls back conductor. Unfortunately, given it would call a cell"},{"line_number":84,"context_line":"conductor, the conductor wouldn\u0027t be able to call again the scheduler given"},{"line_number":85,"context_line":"the latter is not in the same message queue. While it\u0027s a huge problem for"},{"line_number":86,"context_line":"cells v2, we agree it\u0027s a problem not related to that spec."},{"line_number":87,"context_line":""},{"line_number":88,"context_line":""},{"line_number":89,"context_line":"Alternatives"}],"source_content_type":"text/x-rst","patch_set":2,"id":"dfeb2761_9d0f5636","line":86,"updated":"2017-04-04 15:32:48.000000000","message":"s/that spec/this spec/","commit_id":"db981b08ea83dbd87fe21d8633d6f75e857a83b9"},{"author":{"_account_id":7,"name":"Jay Pipes","email":"jaypipes@gmail.com","username":"jaypipes"},"change_message_id":"252b398ac0e381a0d5241e5fa1269c17f38f86b4","unresolved":false,"context_lines":[{"line_number":96,"context_line":"-----------------"},{"line_number":97,"context_line":""},{"line_number":98,"context_line":"The RequestSpec object would be amended by a new field namely \"allocations\""},{"line_number":99,"context_line":"which would be an AllocationList field type."},{"line_number":100,"context_line":""},{"line_number":101,"context_line":"REST API impact"},{"line_number":102,"context_line":"---------------"}],"source_content_type":"text/x-rst","patch_set":2,"id":"dfeb2761_7d0c9240","line":99,"updated":"2017-04-04 15:32:48.000000000","message":"++ well, technically it would a field.ObjectField(\u0027AllocationList\u0027) but I get your point. :)","commit_id":"db981b08ea83dbd87fe21d8633d6f75e857a83b9"},{"author":{"_account_id":7,"name":"Jay Pipes","email":"jaypipes@gmail.com","username":"jaypipes"},"change_message_id":"c87cb7e31e45b28c47f2b60b9077593767e4abea","unresolved":false,"context_lines":[{"line_number":96,"context_line":"-----------------"},{"line_number":97,"context_line":""},{"line_number":98,"context_line":"The RequestSpec object would be amended by a new field namely \"allocations\""},{"line_number":99,"context_line":"which would be an AllocationList field type."},{"line_number":100,"context_line":""},{"line_number":101,"context_line":"REST API impact"},{"line_number":102,"context_line":"---------------"}],"source_content_type":"text/x-rst","patch_set":2,"id":"dfeb2761_ee0b467a","line":99,"updated":"2017-04-04 16:34:20.000000000","message":"It\u0027s not really for deleting the allocations if something goes wrong. As you rightly point out, that\u0027s done just by DELETE /allocations/{instance_uuid}. The reason we want to pass this information in the RequestSpec is to prevent needing to GET /allocations/{instance_uuid} from the compute host in the case of nested resource providers and shared resource providers. In addition to the scheduler selecting a destination compute host, the scheduler will be selecting particular NUMA cells (child providers of the compute node parent provider), SR-IOV PFs and VFs (child providers of the compute node parent or a NUMA node parent) and shared storage pool providers. We want to communicate these selections to the compute host without requiring the compute host to ask the placement API for it.","commit_id":"db981b08ea83dbd87fe21d8633d6f75e857a83b9"},{"author":{"_account_id":1063,"name":"Ed Leafe","email":"ed@leafe.com","username":"ed-leafe"},"change_message_id":"8f357c2132bf838034d0df17c588ccdd8e07d8c7","unresolved":false,"context_lines":[{"line_number":96,"context_line":"-----------------"},{"line_number":97,"context_line":""},{"line_number":98,"context_line":"The RequestSpec object would be amended by a new field namely \"allocations\""},{"line_number":99,"context_line":"which would be an AllocationList field type."},{"line_number":100,"context_line":""},{"line_number":101,"context_line":"REST API impact"},{"line_number":102,"context_line":"---------------"}],"source_content_type":"text/x-rst","patch_set":2,"id":"dfeb2761_8e115a55","line":99,"in_reply_to":"dfeb2761_7d0c9240","updated":"2017-04-04 16:21:13.000000000","message":"Why is it necessary to pass the full list of allocations? I thought we were going to add a transitional field like \u0027allocated\u0027 that would serve as a signal that the allocations have already been made. If the build fails, we don\u0027t need all of the allocations to delete them; just delete them by consumer_id (i.e., the instance UUID).","commit_id":"db981b08ea83dbd87fe21d8633d6f75e857a83b9"},{"author":{"_account_id":1063,"name":"Ed Leafe","email":"ed@leafe.com","username":"ed-leafe"},"change_message_id":"8f357c2132bf838034d0df17c588ccdd8e07d8c7","unresolved":false,"context_lines":[{"line_number":122,"context_line":"------------------"},{"line_number":123,"context_line":""},{"line_number":124,"context_line":"Placement HTTP calls would be made within the scheduler service instead of"},{"line_number":125,"context_line":"compute services. Nothing really changes from a performance perspective."},{"line_number":126,"context_line":""},{"line_number":127,"context_line":"Other deployer impact"},{"line_number":128,"context_line":"---------------------"}],"source_content_type":"text/x-rst","patch_set":2,"id":"dfeb2761_ee546600","line":125,"updated":"2017-04-04 16:21:13.000000000","message":"Well, one could argue that reducing raciness would improve the overall performance of the server creation process, but that\u0027s difficult to measure.","commit_id":"db981b08ea83dbd87fe21d8633d6f75e857a83b9"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"e94275f36495c29dc922629e6b2bc38e485bac62","unresolved":false,"context_lines":[{"line_number":151,"context_line":"----------"},{"line_number":152,"context_line":""},{"line_number":153,"context_line":"* Add the nested AllocationList object to the RequestSpec"},{"line_number":154,"context_line":"* Augment the RPC compute interfaces to accept a RequestSpec and bump the"},{"line_number":155,"context_line":"  service version for computes."},{"line_number":156,"context_line":"* Amend compute nodes to verify if the passed RequestSpec has an allocations"},{"line_number":157,"context_line":"  field and if so, don\u0027t POST allocations to Placement service."}],"source_content_type":"text/x-rst","patch_set":2,"id":"bff0334d_8ebdca03","line":154,"updated":"2017-04-10 15:52:53.000000000","message":"It already does take a request spec:\n\nhttps://github.com/openstack/nova/blob/master/nova/compute/rpcapi.py#L1068\n\nThe compute doesn\u0027t use it at all though.","commit_id":"db981b08ea83dbd87fe21d8633d6f75e857a83b9"},{"author":{"_account_id":1063,"name":"Ed Leafe","email":"ed@leafe.com","username":"ed-leafe"},"change_message_id":"a2ddfde6dbcb2e0fd88595771d04ea42c91edc55","unresolved":false,"context_lines":[{"line_number":48,"context_line":""},{"line_number":49,"context_line":"First of all, let\u0027s clarify a very important point. The current REST API"},{"line_number":50,"context_line":"provided by the Placement service is rich enough to help our usecase, and we"},{"line_number":51,"context_line":"don\u0027t target to modify its API to fit our needs."},{"line_number":52,"context_line":"Rather, the spec targets to modify when/where allocations are created and"},{"line_number":53,"context_line":"instead of letting the RT (see above for the acronym) posting the allocations,"},{"line_number":54,"context_line":"we propose the conductor to preemptively write those allocations and pass them"}],"source_content_type":"text/x-rst","patch_set":3,"id":"bff0334d_1aa663ac","line":51,"range":{"start_line":51,"start_character":6,"end_line":51,"end_character":12},"updated":"2017-04-11 20:46:22.000000000","message":"s/target/need","commit_id":"de89abab2b7bace7cc2791f56ae458996c254b56"},{"author":{"_account_id":11564,"name":"Chris Dent","email":"cdent@anticdent.org","username":"chdent"},"change_message_id":"3cf9e4353b45e62fa0682d026e77e1df438483c1","unresolved":false,"context_lines":[{"line_number":50,"context_line":"provided by the Placement service is rich enough to help our usecase, and we"},{"line_number":51,"context_line":"don\u0027t target to modify its API to fit our needs."},{"line_number":52,"context_line":"Rather, the spec targets to modify when/where allocations are created and"},{"line_number":53,"context_line":"instead of letting the RT (see above for the acronym) posting the allocations,"},{"line_number":54,"context_line":"we propose the conductor to preemptively write those allocations and pass them"},{"line_number":55,"context_line":"down to the compute node once the scheduler returns a destination."},{"line_number":56,"context_line":"We will use a service version check in the conductor to ensure all compute"}],"source_content_type":"text/x-rst","patch_set":3,"id":"bff0334d_a8d98a57","line":53,"updated":"2017-04-11 09:39:13.000000000","message":"instead of letting the resource tracker post the allocations, we propose...","commit_id":"de89abab2b7bace7cc2791f56ae458996c254b56"},{"author":{"_account_id":11564,"name":"Chris Dent","email":"cdent@anticdent.org","username":"chdent"},"change_message_id":"3cf9e4353b45e62fa0682d026e77e1df438483c1","unresolved":false,"context_lines":[{"line_number":51,"context_line":"don\u0027t target to modify its API to fit our needs."},{"line_number":52,"context_line":"Rather, the spec targets to modify when/where allocations are created and"},{"line_number":53,"context_line":"instead of letting the RT (see above for the acronym) posting the allocations,"},{"line_number":54,"context_line":"we propose the conductor to preemptively write those allocations and pass them"},{"line_number":55,"context_line":"down to the compute node once the scheduler returns a destination."},{"line_number":56,"context_line":"We will use a service version check in the conductor to ensure all compute"},{"line_number":57,"context_line":"hosts in the system can handle a request that has had allocations already"}],"source_content_type":"text/x-rst","patch_set":3,"id":"bff0334d_48e8bee0","line":54,"updated":"2017-04-11 09:39:13.000000000","message":"s/to//","commit_id":"de89abab2b7bace7cc2791f56ae458996c254b56"},{"author":{"_account_id":11564,"name":"Chris Dent","email":"cdent@anticdent.org","username":"chdent"},"change_message_id":"3cf9e4353b45e62fa0682d026e77e1df438483c1","unresolved":false,"context_lines":[{"line_number":58,"context_line":"written by the scheduler."},{"line_number":59,"context_line":""},{"line_number":60,"context_line":"The conductor would then pass that AllocationList object as a RPC attribute to"},{"line_number":61,"context_line":"the compute RPC API responsible for calling the target compute service."},{"line_number":62,"context_line":""},{"line_number":63,"context_line":"Then, the compute service would lookup that parameter and if set, then the RT"},{"line_number":64,"context_line":"would not call the Placement API for adding allocations during the claim"}],"source_content_type":"text/x-rst","patch_set":3,"id":"bff0334d_fb519ada","line":61,"updated":"2017-04-11 09:39:13.000000000","message":"So, I don\u0027t want to block on this because we reached some form of consensus in the hangout and I\u0027m willing to live with it if people really want, but in case anyone else is also experiencing doubts about this model I still question why it is better to:\n\n* create a long term dependence on objects we decided were only going to live on the placement side\n* use those objects over RPC, and thus need to attend to versioning less casually than we do now, after we decided to stop that for placement-related objects\n* modify build_and_run_instances (and whatever else) to accept a 12th argument, the AllocationList\n* increase the amount of traffic over a medium people complain is already too encumbered and can be hard to effectively manage for scale\n* increase the code and behavioral complexity in both the conductor and nova-compute for gathering, marshalling, sending, receiving, demarshalling the additional objects\n* need to be conditionally aware of service versions and adjust behavior accordingly\n* plenty of other things I\u0027m sure I\u0027m not aware of because: goodness there\u0027s a lot going on\n\nWhen all the information that is provided by those multiple steps above can be accomplished by doing a `GET /allocations/{instance_uuid}` on the compute node. Something it already does. If the allocations don\u0027t exist, be prepared to write them. If they do, but the placement fails, delete them.\n\nThe placement service has already persisted the allocations, so we don\u0027t need to make them exist over RPC. We\u0027ve created a dependence on the placement HTTP service being up and correct already, so arguments about HTTP failure seem specious. Especially since the placement can horizontally scale and make itself redundantly reliable rather easily because it\u0027s just a web server with a database connection.\n\nI recall from the discussion that having \"more control at the center\" was a good reason for sending the AllocationList over RPC, but I\u0027m not seeing how this actually does that, or what the benefit is. I suspect there is either something I don\u0027t understand about the conductor, or I\u0027m missing some step about what needs to happen with the allocation data on the compute-node (however it gets there). I\u0027d like to figure out what those things are, if they exist, so we can at least describe them here and make sure all our assumptions about how stuff work are explicit.","commit_id":"de89abab2b7bace7cc2791f56ae458996c254b56"},{"author":{"_account_id":1063,"name":"Ed Leafe","email":"ed@leafe.com","username":"ed-leafe"},"change_message_id":"cf3b1045f79c67b1a80123bb538fc83687d898d9","unresolved":false,"context_lines":[{"line_number":58,"context_line":"written by the scheduler."},{"line_number":59,"context_line":""},{"line_number":60,"context_line":"The conductor would then pass that AllocationList object as a RPC attribute to"},{"line_number":61,"context_line":"the compute RPC API responsible for calling the target compute service."},{"line_number":62,"context_line":""},{"line_number":63,"context_line":"Then, the compute service would lookup that parameter and if set, then the RT"},{"line_number":64,"context_line":"would not call the Placement API for adding allocations during the claim"}],"source_content_type":"text/x-rst","patch_set":3,"id":"bff0334d_4a7cf970","line":61,"in_reply_to":"bff0334d_0de3bda5","updated":"2017-04-12 16:14:17.000000000","message":"That may be *a* question, but it wasn\u0027t the one I was concerned with. My concern was the design of the code to implement this. By sending the AllocationList through RPC, we have to change the code in several places to handle this new thing, and we have to abandon the simple objects we are using and switch to a versioned object. None of those intermediaries have any use for this data; they\u0027re just passing it along until it ends up at compute. IMO, a much cleaner, and more maintainable, design would be to have the computes ask for the information when they need it. I doubt that the AllocationList is going to be so weighty that it impacts the performance of either RPC or HTTP, so that\u0027s not a concern.\n\nWe designed a robust placement engine with a clean, direct HTTP interface. The database was designed for maximum performance. We can stand up as many placement API handlers as we need to handle whatever load of requests are encountered, and the database load should never be very high, as the data involved is small and well-indexed.\n\nAnd as far as your security concerns, I don\u0027t think that having compute call placement opens up new attack surfaces. If a rogue VM somehow gets out into the hypervisor layer to control the node as an admin, the game is already lost. They can just as easily attack RPC channels as HTTP channels, so I don\u0027t see how that is a significant argument.\n\nSo I understand that during the hangout the consensus was to carry this information through RPC, and I guess that\u0027s what we\u0027ll end up doing. But for the spec, I wanted to be sure that the alternative approach is clearly spelled out, so that if (when? :) we get mired in a bunch of PITA changes to implement this, we at least have the simpler approach documented.","commit_id":"de89abab2b7bace7cc2791f56ae458996c254b56"},{"author":{"_account_id":7166,"name":"Sylvain Bauza","email":"sbauza@redhat.com","username":"sbauza"},"change_message_id":"d74b7e1c2872f4e270a66e29b03c3277bf81c274","unresolved":false,"context_lines":[{"line_number":58,"context_line":"written by the scheduler."},{"line_number":59,"context_line":""},{"line_number":60,"context_line":"The conductor would then pass that AllocationList object as a RPC attribute to"},{"line_number":61,"context_line":"the compute RPC API responsible for calling the target compute service."},{"line_number":62,"context_line":""},{"line_number":63,"context_line":"Then, the compute service would lookup that parameter and if set, then the RT"},{"line_number":64,"context_line":"would not call the Placement API for adding allocations during the claim"}],"source_content_type":"text/x-rst","patch_set":3,"id":"bff0334d_0de3bda5","line":61,"in_reply_to":"bff0334d_7afc4f6f","updated":"2017-04-12 09:32:27.000000000","message":"So, the question should be : is it better to increase the RPC payload for calling the compute node or is it better to have HTTP calls within the compute node ?\n\nAFAICS, I think operators would prefer to have more payload (and maybe using more conductors for that) rather than having compute nodes calling the control plane for 2 reasons :\n* one is about security : we know some operators put their compute nodes into separated networks because they don\u0027t trust them. I know we already have control calls from the compute nodes, but I think they\u0027d prefer to have less than more.\n* the other is about scaling : at the moment, scaling out my cloud is by adding more computes and more conductors, not adding more controllers. If we keep the situation as it is, then the scalability could be bottlenecked by Placement service performance.","commit_id":"de89abab2b7bace7cc2791f56ae458996c254b56"},{"author":{"_account_id":1063,"name":"Ed Leafe","email":"ed@leafe.com","username":"ed-leafe"},"change_message_id":"a2ddfde6dbcb2e0fd88595771d04ea42c91edc55","unresolved":false,"context_lines":[{"line_number":58,"context_line":"written by the scheduler."},{"line_number":59,"context_line":""},{"line_number":60,"context_line":"The conductor would then pass that AllocationList object as a RPC attribute to"},{"line_number":61,"context_line":"the compute RPC API responsible for calling the target compute service."},{"line_number":62,"context_line":""},{"line_number":63,"context_line":"Then, the compute service would lookup that parameter and if set, then the RT"},{"line_number":64,"context_line":"would not call the Placement API for adding allocations during the claim"}],"source_content_type":"text/x-rst","patch_set":3,"id":"bff0334d_7afc4f6f","line":61,"in_reply_to":"bff0334d_fb519ada","updated":"2017-04-11 20:46:22.000000000","message":"At the risk of sounding like Chris\u0027s parrot, I also prefer to make a single HTTP call from the compute node instead of passing state with a bunch of RPC objects. It\u0027s a much cleaner design, and will require fewer changes to the existing code to implement.","commit_id":"de89abab2b7bace7cc2791f56ae458996c254b56"},{"author":{"_account_id":1063,"name":"Ed Leafe","email":"ed@leafe.com","username":"ed-leafe"},"change_message_id":"a2ddfde6dbcb2e0fd88595771d04ea42c91edc55","unresolved":false,"context_lines":[{"line_number":67,"context_line":"Placement API."},{"line_number":68,"context_line":""},{"line_number":69,"context_line":"Move operations and reschedules (if unexpected failures happen when spawning"},{"line_number":70,"context_line":"the instance) will delete original allocations by having the conductor querying"},{"line_number":71,"context_line":"the Placement API for knowing whether an instance already has allocations and"},{"line_number":72,"context_line":"delete those."},{"line_number":73,"context_line":"Destroying an instance will also see the conductor querying the existing"},{"line_number":74,"context_line":"allocations in the Placement API and asking to delete those."},{"line_number":75,"context_line":""}],"source_content_type":"text/x-rst","patch_set":3,"id":"bff0334d_fa161fa8","line":72,"range":{"start_line":70,"start_character":71,"end_line":72,"end_character":13},"updated":"2017-04-11 20:46:22.000000000","message":"Not really necessary. Just DELETE /allocations/{instance_id} and be done with it.","commit_id":"de89abab2b7bace7cc2791f56ae458996c254b56"},{"author":{"_account_id":1063,"name":"Ed Leafe","email":"ed@leafe.com","username":"ed-leafe"},"change_message_id":"a2ddfde6dbcb2e0fd88595771d04ea42c91edc55","unresolved":false,"context_lines":[{"line_number":70,"context_line":"the instance) will delete original allocations by having the conductor querying"},{"line_number":71,"context_line":"the Placement API for knowing whether an instance already has allocations and"},{"line_number":72,"context_line":"delete those."},{"line_number":73,"context_line":"Destroying an instance will also see the conductor querying the existing"},{"line_number":74,"context_line":"allocations in the Placement API and asking to delete those."},{"line_number":75,"context_line":""},{"line_number":76,"context_line":"That all said, there is a current problem with reschedules for Cells V2 where"},{"line_number":77,"context_line":"the compute calls back conductor. Unfortunately, given it would call a cell"}],"source_content_type":"text/x-rst","patch_set":3,"id":"bff0334d_7a2a0fdb","line":74,"range":{"start_line":73,"start_character":51,"end_line":74,"end_character":32},"updated":"2017-04-11 20:46:22.000000000","message":"ditto","commit_id":"de89abab2b7bace7cc2791f56ae458996c254b56"},{"author":{"_account_id":11564,"name":"Chris Dent","email":"cdent@anticdent.org","username":"chdent"},"change_message_id":"3cf9e4353b45e62fa0682d026e77e1df438483c1","unresolved":false,"context_lines":[{"line_number":84,"context_line":"------------"},{"line_number":85,"context_line":""},{"line_number":86,"context_line":"We could consider that Placement doesn\u0027t need to know the on-going resource"},{"line_number":87,"context_line":"creations and that it is only a capacity issue."},{"line_number":88,"context_line":""},{"line_number":89,"context_line":"Data model impact"},{"line_number":90,"context_line":"-----------------"}],"source_content_type":"text/x-rst","patch_set":3,"id":"bff0334d_9b34ce94","line":87,"updated":"2017-04-11 09:39:13.000000000","message":"For the sake of clarity and understanding it might be useful to put the \"get the allocations over http instead of sending over rpc\" alternative in here.","commit_id":"de89abab2b7bace7cc2791f56ae458996c254b56"},{"author":{"_account_id":11564,"name":"Chris Dent","email":"cdent@anticdent.org","username":"chdent"},"change_message_id":"3cf9e4353b45e62fa0682d026e77e1df438483c1","unresolved":false,"context_lines":[{"line_number":89,"context_line":"Data model impact"},{"line_number":90,"context_line":"-----------------"},{"line_number":91,"context_line":""},{"line_number":92,"context_line":"None."},{"line_number":93,"context_line":""},{"line_number":94,"context_line":"REST API impact"},{"line_number":95,"context_line":"---------------"}],"source_content_type":"text/x-rst","patch_set":3,"id":"bff0334d_5b2ac62b","line":92,"updated":"2017-04-11 09:39:13.000000000","message":"Do we consider the objects as part of the data model, or just the database schema?","commit_id":"de89abab2b7bace7cc2791f56ae458996c254b56"},{"author":{"_account_id":11564,"name":"Chris Dent","email":"cdent@anticdent.org","username":"chdent"},"change_message_id":"3cf9e4353b45e62fa0682d026e77e1df438483c1","unresolved":false,"context_lines":[{"line_number":115,"context_line":"------------------"},{"line_number":116,"context_line":""},{"line_number":117,"context_line":"Placement HTTP calls would be made within the conductor service instead of"},{"line_number":118,"context_line":"compute services. Nothing really changes from a performance perspective."},{"line_number":119,"context_line":""},{"line_number":120,"context_line":"Other deployer impact"},{"line_number":121,"context_line":"---------------------"}],"source_content_type":"text/x-rst","patch_set":3,"id":"bff0334d_bbd43221","line":118,"updated":"2017-04-11 09:39:13.000000000","message":"This makes it sounds like all HTTP from the compute node to the placement service is going to stop, which is not accurate.","commit_id":"de89abab2b7bace7cc2791f56ae458996c254b56"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"5d2e24ff9678f02f80c511c19a846ff0aa533c44","unresolved":false,"context_lines":[{"line_number":5,"context_line":" http://creativecommons.org/licenses/by/3.0/legalcode"},{"line_number":6,"context_line":""},{"line_number":7,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"},{"line_number":8,"context_line":"Placement API claiming resources"},{"line_number":9,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"},{"line_number":10,"context_line":""},{"line_number":11,"context_line":"https://blueprints.launchpad.net/nova/+spec/placement-claims"}],"source_content_type":"text/x-rst","patch_set":4,"id":"7ffa3b31_3941c33a","line":8,"range":{"start_line":8,"start_character":0,"end_line":8,"end_character":32},"updated":"2017-04-17 20:55:37.000000000","message":"Rename this, just \"Claims in scheduler\" or something.","commit_id":"d59286ab36ccc0cebcef62371e421ddf714a694b"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"5d2e24ff9678f02f80c511c19a846ff0aa533c44","unresolved":false,"context_lines":[{"line_number":10,"context_line":""},{"line_number":11,"context_line":"https://blueprints.launchpad.net/nova/+spec/placement-claims"},{"line_number":12,"context_line":""},{"line_number":13,"context_line":"Have the Nova conductor create allocation records in the placement API after"},{"line_number":14,"context_line":"selecting a compute host."},{"line_number":15,"context_line":""},{"line_number":16,"context_line":"Problem description"}],"source_content_type":"text/x-rst","patch_set":4,"id":"7ffa3b31_9987b762","line":13,"range":{"start_line":13,"start_character":14,"end_line":13,"end_character":23},"updated":"2017-04-17 20:55:37.000000000","message":"As Ed noted, we talked about this in the scheduler subteam meeting today and we aren\u0027t clear on if the conductor should post the allocations and call scheduler to fully rebuild the host list on retry (if the \"claim\" fails in conductor), or if scheduler should make the claim since it has the filtered/weighed host list, and could be more efficient to retry there using the next host in the list.\n\nWithout scale testing of both approaches it\u0027s hard to know which would be better. I personally lean toward doing the claim retry in the scheduler where we have the hosts already rather than conductor. I think we could document both approaches though with pros/cons for each and maybe punt the decision a bit until we get some testing for each.","commit_id":"d59286ab36ccc0cebcef62371e421ddf714a694b"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"5d2e24ff9678f02f80c511c19a846ff0aa533c44","unresolved":false,"context_lines":[{"line_number":26,"context_line":"While this model fits very well for a cloud having lots of capacity left, we"},{"line_number":27,"context_line":"know that this can be suboptimal because of requests concurrency."},{"line_number":28,"context_line":""},{"line_number":29,"context_line":"In this spec, we propose to have the conductor allocate resources instead of"},{"line_number":30,"context_line":"the compute node."},{"line_number":31,"context_line":"Given the Placement API does not implement all the scheduler filters and"},{"line_number":32,"context_line":"weighers, the proposal is that the scheduler would first get a list of"}],"source_content_type":"text/x-rst","patch_set":4,"id":"7ffa3b31_b9e9531b","line":29,"range":{"start_line":29,"start_character":37,"end_line":29,"end_character":46},"updated":"2017-04-17 20:55:37.000000000","message":"scheduler I think based on today\u0027s meeting.","commit_id":"d59286ab36ccc0cebcef62371e421ddf714a694b"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"5d2e24ff9678f02f80c511c19a846ff0aa533c44","unresolved":false,"context_lines":[{"line_number":31,"context_line":"Given the Placement API does not implement all the scheduler filters and"},{"line_number":32,"context_line":"weighers, the proposal is that the scheduler would first get a list of"},{"line_number":33,"context_line":"resource providers, iterate over those with the help of filters and weighters,"},{"line_number":34,"context_line":"pass the result back to the conductor so the conductor would post the"},{"line_number":35,"context_line":"allocations against the found target."},{"line_number":36,"context_line":"In this way, we both reduce the occurrence of retry operations as well as the"},{"line_number":37,"context_line":"time period that race conditions can occur."},{"line_number":38,"context_line":""}],"source_content_type":"text/x-rst","patch_set":4,"id":"7ffa3b31_d9e1bf2c","line":35,"range":{"start_line":34,"start_character":0,"end_line":35,"end_character":37},"updated":"2017-04-17 20:55:37.000000000","message":"This would need to be re-worded if scheduler posts the allocations.","commit_id":"d59286ab36ccc0cebcef62371e421ddf714a694b"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"5d2e24ff9678f02f80c511c19a846ff0aa533c44","unresolved":false,"context_lines":[{"line_number":46,"context_line":"Proposed change"},{"line_number":47,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"},{"line_number":48,"context_line":""},{"line_number":49,"context_line":"First of all, let\u0027s clarify a very important point. The current REST API"},{"line_number":50,"context_line":"provided by the Placement service is rich enough to help our usecase, and we"},{"line_number":51,"context_line":"don\u0027t target to modify its API to fit our needs."},{"line_number":52,"context_line":"Rather, the spec targets to modify when/where allocations are created and"},{"line_number":53,"context_line":"instead of letting the RT (see above for the acronym) post the allocations, we"},{"line_number":54,"context_line":"propose the conductor preemptively write those allocations and pass them down"},{"line_number":55,"context_line":"to the compute node once the scheduler returns a destination. We will use a"}],"source_content_type":"text/x-rst","patch_set":4,"id":"7ffa3b31_d9a77fa7","line":52,"range":{"start_line":49,"start_character":0,"end_line":52,"end_character":7},"updated":"2017-04-17 20:55:37.000000000","message":"I\u0027d remove all of this. The REST API impact section says None, and there is nothing in the spec that would lead me to believe we\u0027re changing the placement REST API so I\u0027m not concerned or need clarification.","commit_id":"d59286ab36ccc0cebcef62371e421ddf714a694b"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"5d2e24ff9678f02f80c511c19a846ff0aa533c44","unresolved":false,"context_lines":[{"line_number":50,"context_line":"provided by the Placement service is rich enough to help our usecase, and we"},{"line_number":51,"context_line":"don\u0027t target to modify its API to fit our needs."},{"line_number":52,"context_line":"Rather, the spec targets to modify when/where allocations are created and"},{"line_number":53,"context_line":"instead of letting the RT (see above for the acronym) post the allocations, we"},{"line_number":54,"context_line":"propose the conductor preemptively write those allocations and pass them down"},{"line_number":55,"context_line":"to the compute node once the scheduler returns a destination. We will use a"},{"line_number":56,"context_line":"service version check in the conductor to ensure all compute hosts in the"}],"source_content_type":"text/x-rst","patch_set":4,"id":"7ffa3b31_19c16781","line":53,"range":{"start_line":53,"start_character":26,"end_line":53,"end_character":53},"updated":"2017-04-17 20:55:37.000000000","message":"nit: this is unnecessary, that\u0027s what (RT) means.","commit_id":"d59286ab36ccc0cebcef62371e421ddf714a694b"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"5d2e24ff9678f02f80c511c19a846ff0aa533c44","unresolved":false,"context_lines":[{"line_number":51,"context_line":"don\u0027t target to modify its API to fit our needs."},{"line_number":52,"context_line":"Rather, the spec targets to modify when/where allocations are created and"},{"line_number":53,"context_line":"instead of letting the RT (see above for the acronym) post the allocations, we"},{"line_number":54,"context_line":"propose the conductor preemptively write those allocations and pass them down"},{"line_number":55,"context_line":"to the compute node once the scheduler returns a destination. We will use a"},{"line_number":56,"context_line":"service version check in the conductor to ensure all compute hosts in the"},{"line_number":57,"context_line":"system can handle a request that has had allocations already written by the"}],"source_content_type":"text/x-rst","patch_set":4,"id":"7ffa3b31_f9bfdbf8","line":54,"range":{"start_line":54,"start_character":12,"end_line":54,"end_character":21},"updated":"2017-04-17 20:55:37.000000000","message":"scheduler","commit_id":"d59286ab36ccc0cebcef62371e421ddf714a694b"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"5d2e24ff9678f02f80c511c19a846ff0aa533c44","unresolved":false,"context_lines":[{"line_number":51,"context_line":"don\u0027t target to modify its API to fit our needs."},{"line_number":52,"context_line":"Rather, the spec targets to modify when/where allocations are created and"},{"line_number":53,"context_line":"instead of letting the RT (see above for the acronym) post the allocations, we"},{"line_number":54,"context_line":"propose the conductor preemptively write those allocations and pass them down"},{"line_number":55,"context_line":"to the compute node once the scheduler returns a destination. We will use a"},{"line_number":56,"context_line":"service version check in the conductor to ensure all compute hosts in the"},{"line_number":57,"context_line":"system can handle a request that has had allocations already written by the"},{"line_number":58,"context_line":"scheduler."}],"source_content_type":"text/x-rst","patch_set":4,"id":"7ffa3b31_b96033a7","line":55,"range":{"start_line":54,"start_character":63,"end_line":55,"end_character":20},"updated":"2017-04-17 20:55:37.000000000","message":"As Ed pointed out, I know we said we\u0027d pass the AllocationList versioned object down to the compute through RPC, but based on today\u0027s scheduler subteam meeting I think we\u0027re changing course on that and will just have the compute get the allocations. If they exist already, great, the compute doesn\u0027t need to do that, otherwise the compute falls back to the existing behavior and makes the claim (posts allocations) itself.","commit_id":"d59286ab36ccc0cebcef62371e421ddf714a694b"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"5d2e24ff9678f02f80c511c19a846ff0aa533c44","unresolved":false,"context_lines":[{"line_number":53,"context_line":"instead of letting the RT (see above for the acronym) post the allocations, we"},{"line_number":54,"context_line":"propose the conductor preemptively write those allocations and pass them down"},{"line_number":55,"context_line":"to the compute node once the scheduler returns a destination. We will use a"},{"line_number":56,"context_line":"service version check in the conductor to ensure all compute hosts in the"},{"line_number":57,"context_line":"system can handle a request that has had allocations already written by the"},{"line_number":58,"context_line":"scheduler."},{"line_number":59,"context_line":""}],"source_content_type":"text/x-rst","patch_set":4,"id":"7ffa3b31_996337b2","line":56,"range":{"start_line":56,"start_character":29,"end_line":56,"end_character":38},"updated":"2017-04-17 20:55:37.000000000","message":"scheduler","commit_id":"d59286ab36ccc0cebcef62371e421ddf714a694b"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"5d2e24ff9678f02f80c511c19a846ff0aa533c44","unresolved":false,"context_lines":[{"line_number":57,"context_line":"system can handle a request that has had allocations already written by the"},{"line_number":58,"context_line":"scheduler."},{"line_number":59,"context_line":""},{"line_number":60,"context_line":"The conductor would then pass that AllocationList object as a RPC attribute to"},{"line_number":61,"context_line":"the compute RPC API responsible for calling the target compute service."},{"line_number":62,"context_line":""},{"line_number":63,"context_line":"Then, the compute service would lookup that parameter and if set, then the RT"},{"line_number":64,"context_line":"would not call the Placement API for adding allocations during the claim"}],"source_content_type":"text/x-rst","patch_set":4,"id":"7ffa3b31_d9689f88","line":61,"range":{"start_line":60,"start_character":0,"end_line":61,"end_character":71},"updated":"2017-04-17 20:55:37.000000000","message":"As noted above, we\u0027re not doing this anymore.","commit_id":"d59286ab36ccc0cebcef62371e421ddf714a694b"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"5d2e24ff9678f02f80c511c19a846ff0aa533c44","unresolved":false,"context_lines":[{"line_number":60,"context_line":"The conductor would then pass that AllocationList object as a RPC attribute to"},{"line_number":61,"context_line":"the compute RPC API responsible for calling the target compute service."},{"line_number":62,"context_line":""},{"line_number":63,"context_line":"Then, the compute service would lookup that parameter and if set, then the RT"},{"line_number":64,"context_line":"would not call the Placement API for adding allocations during the claim"},{"line_number":65,"context_line":"operation. That claim contextmanager will itself become lightweight because it"},{"line_number":66,"context_line":"will only verify PCI and NUMA resources that aren\u0027t yet provided to the"},{"line_number":67,"context_line":"Placement API."},{"line_number":68,"context_line":""}],"source_content_type":"text/x-rst","patch_set":4,"id":"7ffa3b31_19f4c7ce","line":65,"range":{"start_line":63,"start_character":0,"end_line":65,"end_character":10},"updated":"2017-04-17 20:55:37.000000000","message":"This changes - the compute will GET /allocations/{instance_id} and if an empty list comes back then it does the claim itself because the controller didn\u0027t do it yet. This is fallback for Pike and a failure in Queens.","commit_id":"d59286ab36ccc0cebcef62371e421ddf714a694b"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"5d2e24ff9678f02f80c511c19a846ff0aa533c44","unresolved":false,"context_lines":[{"line_number":69,"context_line":"Move operations and reschedules (if unexpected failures happen when spawning"},{"line_number":70,"context_line":"the instance) will delete original allocations by having the conductor calling"},{"line_number":71,"context_line":"DELETE /allocations/{instance_id} and be done with it."},{"line_number":72,"context_line":"Destroying an instance will also see the conductor doing the same call to the"},{"line_number":73,"context_line":"Placement API."},{"line_number":74,"context_line":""},{"line_number":75,"context_line":"That all said, there is a current problem with reschedules for Cells V2 where"}],"source_content_type":"text/x-rst","patch_set":4,"id":"7ffa3b31_19cba772","line":72,"range":{"start_line":72,"start_character":41,"end_line":72,"end_character":50},"updated":"2017-04-17 20:55:37.000000000","message":"To be clear, I don\u0027t know that conductor would be doing this when we delete an instance. I had assumed it would be handled like cleaning up volumes or networks, i.e. if the compute host is down, we cleanup in the API, otherwise we cleanup in the compute.","commit_id":"d59286ab36ccc0cebcef62371e421ddf714a694b"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"a66b66d2c205ce22c1c021ce6498a85f2ff34e3a","unresolved":false,"context_lines":[{"line_number":69,"context_line":"Move operations and reschedules (if unexpected failures happen when spawning"},{"line_number":70,"context_line":"the instance) will delete original allocations by having the conductor calling"},{"line_number":71,"context_line":"DELETE /allocations/{instance_id} and be done with it."},{"line_number":72,"context_line":"Destroying an instance will also see the conductor doing the same call to the"},{"line_number":73,"context_line":"Placement API."},{"line_number":74,"context_line":""},{"line_number":75,"context_line":"That all said, there is a current problem with reschedules for Cells V2 where"}],"source_content_type":"text/x-rst","patch_set":4,"id":"7ffa3b31_f9211bfd","line":72,"range":{"start_line":72,"start_character":41,"end_line":72,"end_character":50},"in_reply_to":"7ffa3b31_19cba772","updated":"2017-04-17 20:57:38.000000000","message":"Actually yeah, we really don\u0027t do this in conductor. Consider the soft delete case. When we soft delete an instance, it\u0027s put into a special state in the API and then on a timer the compute does the hard delete. So in that case, the compute needs to remove the allocations when it does the hard delete. This is also because before the timer is up the user can restore the instance and we don\u0027t want to lose the allocations before that happens.","commit_id":"d59286ab36ccc0cebcef62371e421ddf714a694b"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"5d2e24ff9678f02f80c511c19a846ff0aa533c44","unresolved":false,"context_lines":[{"line_number":75,"context_line":"That all said, there is a current problem with reschedules for Cells V2 where"},{"line_number":76,"context_line":"the compute calls back conductor. Unfortunately, given it would call a cell"},{"line_number":77,"context_line":"conductor, the conductor wouldn\u0027t be able to call again the scheduler given"},{"line_number":78,"context_line":"the latter is not in the same message queue. While it\u0027s a huge problem for"},{"line_number":79,"context_line":"cells v2, we agree it\u0027s a problem not related to this spec."},{"line_number":80,"context_line":""},{"line_number":81,"context_line":""}],"source_content_type":"text/x-rst","patch_set":4,"id":"7ffa3b31_99d69727","line":78,"range":{"start_line":78,"start_character":58,"end_line":78,"end_character":62},"updated":"2017-04-17 20:55:37.000000000","message":"I wouldn\u0027t classify this as a \"huge\" problem for cells v2. The point of claims in the scheduler is that we shouldn\u0027t need to rely on the retry loop we have today. It\u0027s also not a problem in a single-cell cells v2 deployment where we could have operators opt into or out of the retry behavior. But as noted, that\u0027s outside the scope of this spec.","commit_id":"d59286ab36ccc0cebcef62371e421ddf714a694b"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"5d2e24ff9678f02f80c511c19a846ff0aa533c44","unresolved":false,"context_lines":[{"line_number":84,"context_line":""},{"line_number":85,"context_line":"We could consider that Placement doesn\u0027t need to know the on-going resource"},{"line_number":86,"context_line":"creations and that it is only a capacity issue."},{"line_number":87,"context_line":"We could also leave the compute call the Placement API for knowing the related"},{"line_number":88,"context_line":"allocations for the specific instance instead of passing them thru RPC as part"},{"line_number":89,"context_line":"of the payload."},{"line_number":90,"context_line":""},{"line_number":91,"context_line":"Data model impact"},{"line_number":92,"context_line":"-----------------"}],"source_content_type":"text/x-rst","patch_set":4,"id":"7ffa3b31_d999bf79","line":89,"range":{"start_line":87,"start_character":0,"end_line":89,"end_character":15},"updated":"2017-04-17 20:55:37.000000000","message":"Right, I think we do this now, and document passing allocations over rpc as the alternative, and document why we aren\u0027t doing that (in part because it ties the AllocationList object to nova, which is really for placement).","commit_id":"d59286ab36ccc0cebcef62371e421ddf714a694b"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"5d2e24ff9678f02f80c511c19a846ff0aa533c44","unresolved":false,"context_lines":[{"line_number":91,"context_line":"Data model impact"},{"line_number":92,"context_line":"-----------------"},{"line_number":93,"context_line":""},{"line_number":94,"context_line":"None, we will use the same oslo-versioned objects, ie. AllocationList for"},{"line_number":95,"context_line":"the conductor passing them to the compute."},{"line_number":96,"context_line":"We will also not persist them in the DB for the related instance."},{"line_number":97,"context_line":""},{"line_number":98,"context_line":"REST API impact"},{"line_number":99,"context_line":"---------------"}],"source_content_type":"text/x-rst","patch_set":4,"id":"7ffa3b31_998d7733","line":96,"range":{"start_line":94,"start_character":0,"end_line":96,"end_character":65},"updated":"2017-04-17 20:55:37.000000000","message":"Just \"None\".","commit_id":"d59286ab36ccc0cebcef62371e421ddf714a694b"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"5d2e24ff9678f02f80c511c19a846ff0aa533c44","unresolved":false,"context_lines":[{"line_number":113,"context_line":"Other end user impact"},{"line_number":114,"context_line":"---------------------"},{"line_number":115,"context_line":""},{"line_number":116,"context_line":"Nothing visible from end-user."},{"line_number":117,"context_line":""},{"line_number":118,"context_line":"Performance Impact"},{"line_number":119,"context_line":"------------------"}],"source_content_type":"text/x-rst","patch_set":4,"id":"7ffa3b31_397e6319","line":116,"range":{"start_line":116,"start_character":0,"end_line":116,"end_character":30},"updated":"2017-04-17 20:55:37.000000000","message":"None","commit_id":"d59286ab36ccc0cebcef62371e421ddf714a694b"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"5d2e24ff9678f02f80c511c19a846ff0aa533c44","unresolved":false,"context_lines":[{"line_number":118,"context_line":"Performance Impact"},{"line_number":119,"context_line":"------------------"},{"line_number":120,"context_line":""},{"line_number":121,"context_line":"Placement HTTP calls would be made within the conductor service instead of"},{"line_number":122,"context_line":"compute services for allocating instances."},{"line_number":123,"context_line":"Nothing really changes from a performance perspective, just a bit more RPC"},{"line_number":124,"context_line":"payload."}],"source_content_type":"text/x-rst","patch_set":4,"id":"7ffa3b31_f9c59b3d","line":121,"range":{"start_line":121,"start_character":46,"end_line":121,"end_character":55},"updated":"2017-04-17 20:55:37.000000000","message":"scheduler","commit_id":"d59286ab36ccc0cebcef62371e421ddf714a694b"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"5d2e24ff9678f02f80c511c19a846ff0aa533c44","unresolved":false,"context_lines":[{"line_number":120,"context_line":""},{"line_number":121,"context_line":"Placement HTTP calls would be made within the conductor service instead of"},{"line_number":122,"context_line":"compute services for allocating instances."},{"line_number":123,"context_line":"Nothing really changes from a performance perspective, just a bit more RPC"},{"line_number":124,"context_line":"payload."},{"line_number":125,"context_line":""},{"line_number":126,"context_line":"Other deployer impact"},{"line_number":127,"context_line":"---------------------"}],"source_content_type":"text/x-rst","patch_set":4,"id":"7ffa3b31_99e0d7ce","line":124,"range":{"start_line":123,"start_character":0,"end_line":124,"end_character":8},"updated":"2017-04-17 20:55:37.000000000","message":"This could probably be removed.","commit_id":"d59286ab36ccc0cebcef62371e421ddf714a694b"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"5d2e24ff9678f02f80c511c19a846ff0aa533c44","unresolved":false,"context_lines":[{"line_number":144,"context_line":"  sylvain-bauza"},{"line_number":145,"context_line":""},{"line_number":146,"context_line":"Other contributors:"},{"line_number":147,"context_line":"  \u003claunchpad-id or None\u003e"},{"line_number":148,"context_line":""},{"line_number":149,"context_line":"Work Items"},{"line_number":150,"context_line":"----------"}],"source_content_type":"text/x-rst","patch_set":4,"id":"7ffa3b31_79e74be6","line":147,"range":{"start_line":147,"start_character":2,"end_line":147,"end_character":24},"updated":"2017-04-17 20:55:37.000000000","message":"None?","commit_id":"d59286ab36ccc0cebcef62371e421ddf714a694b"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"5d2e24ff9678f02f80c511c19a846ff0aa533c44","unresolved":false,"context_lines":[{"line_number":151,"context_line":""},{"line_number":152,"context_line":"* Amend RT to filter out allocations for instances that don\u0027t have a host set"},{"line_number":153,"context_line":"  yet when it does self-heal check."},{"line_number":154,"context_line":"* Amend compute nodes to verify the RPC method parameter called \u0027allocations\u0027"},{"line_number":155,"context_line":"  and if so, don\u0027t POST allocations to Placement service."},{"line_number":156,"context_line":"* Modify conductor to verify whether an allocation exists for an instance and"},{"line_number":157,"context_line":"  delete it if this is a reschedule or a move operation."},{"line_number":158,"context_line":"* Modify conductor\u0027 instance destroy method to also delete allocations."}],"source_content_type":"text/x-rst","patch_set":4,"id":"7ffa3b31_19ccc75f","line":155,"range":{"start_line":154,"start_character":2,"end_line":155,"end_character":57},"updated":"2017-04-17 20:55:37.000000000","message":"This changes.","commit_id":"d59286ab36ccc0cebcef62371e421ddf714a694b"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"5d2e24ff9678f02f80c511c19a846ff0aa533c44","unresolved":false,"context_lines":[{"line_number":153,"context_line":"  yet when it does self-heal check."},{"line_number":154,"context_line":"* Amend compute nodes to verify the RPC method parameter called \u0027allocations\u0027"},{"line_number":155,"context_line":"  and if so, don\u0027t POST allocations to Placement service."},{"line_number":156,"context_line":"* Modify conductor to verify whether an allocation exists for an instance and"},{"line_number":157,"context_line":"  delete it if this is a reschedule or a move operation."},{"line_number":158,"context_line":"* Modify conductor\u0027 instance destroy method to also delete allocations."},{"line_number":159,"context_line":"* Modify conductor to POST allocations to Placement if all computes are new"}],"source_content_type":"text/x-rst","patch_set":4,"id":"7ffa3b31_79006b68","line":156,"range":{"start_line":156,"start_character":22,"end_line":156,"end_character":57},"updated":"2017-04-17 20:55:37.000000000","message":"I think we can just DELETE /allocations/{instance_id} and handle a 404, we don\u0027t need to GET and then DELETE.","commit_id":"d59286ab36ccc0cebcef62371e421ddf714a694b"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"5d2e24ff9678f02f80c511c19a846ff0aa533c44","unresolved":false,"context_lines":[{"line_number":155,"context_line":"  and if so, don\u0027t POST allocations to Placement service."},{"line_number":156,"context_line":"* Modify conductor to verify whether an allocation exists for an instance and"},{"line_number":157,"context_line":"  delete it if this is a reschedule or a move operation."},{"line_number":158,"context_line":"* Modify conductor\u0027 instance destroy method to also delete allocations."},{"line_number":159,"context_line":"* Modify conductor to POST allocations to Placement if all computes are new"},{"line_number":160,"context_line":"  and pass them over RPC to compute."},{"line_number":161,"context_line":""}],"source_content_type":"text/x-rst","patch_set":4,"id":"7ffa3b31_59036f74","line":158,"range":{"start_line":158,"start_character":9,"end_line":158,"end_character":18},"updated":"2017-04-17 20:55:37.000000000","message":"There is no conductor instance destroy method. You either destroy the instance in the API (local delete), or the compute.","commit_id":"d59286ab36ccc0cebcef62371e421ddf714a694b"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"5d2e24ff9678f02f80c511c19a846ff0aa533c44","unresolved":false,"context_lines":[{"line_number":156,"context_line":"* Modify conductor to verify whether an allocation exists for an instance and"},{"line_number":157,"context_line":"  delete it if this is a reschedule or a move operation."},{"line_number":158,"context_line":"* Modify conductor\u0027 instance destroy method to also delete allocations."},{"line_number":159,"context_line":"* Modify conductor to POST allocations to Placement if all computes are new"},{"line_number":160,"context_line":"  and pass them over RPC to compute."},{"line_number":161,"context_line":""},{"line_number":162,"context_line":"Dependencies"}],"source_content_type":"text/x-rst","patch_set":4,"id":"7ffa3b31_39fae374","line":159,"range":{"start_line":159,"start_character":9,"end_line":159,"end_character":18},"updated":"2017-04-17 20:55:37.000000000","message":"scheduler","commit_id":"d59286ab36ccc0cebcef62371e421ddf714a694b"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"5d2e24ff9678f02f80c511c19a846ff0aa533c44","unresolved":false,"context_lines":[{"line_number":157,"context_line":"  delete it if this is a reschedule or a move operation."},{"line_number":158,"context_line":"* Modify conductor\u0027 instance destroy method to also delete allocations."},{"line_number":159,"context_line":"* Modify conductor to POST allocations to Placement if all computes are new"},{"line_number":160,"context_line":"  and pass them over RPC to compute."},{"line_number":161,"context_line":""},{"line_number":162,"context_line":"Dependencies"},{"line_number":163,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"}],"source_content_type":"text/x-rst","patch_set":4,"id":"7ffa3b31_19f5e783","line":160,"range":{"start_line":160,"start_character":2,"end_line":160,"end_character":36},"updated":"2017-04-17 20:55:37.000000000","message":"remove this","commit_id":"d59286ab36ccc0cebcef62371e421ddf714a694b"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"fa50a80871a10b6fef87eb9bc1bc3a679b29d213","unresolved":false,"context_lines":[{"line_number":55,"context_line":""},{"line_number":56,"context_line":"Then, the compute service would query the Placement API for knowing if there"},{"line_number":57,"context_line":"are existing allocations for that specific consumer (the instance) and if so,"},{"line_number":58,"context_line":"then the RT would not call the Placement API for adding allocations during the"},{"line_number":59,"context_line":"claim operation. That claim contextmanager will itself become lightweight"},{"line_number":60,"context_line":"because it will only verify PCI and NUMA resources that aren\u0027t yet provided to"},{"line_number":61,"context_line":"the Placement API."},{"line_number":62,"context_line":""}],"source_content_type":"text/x-rst","patch_set":5,"id":"7ffa3b31_d75f1f70","line":59,"range":{"start_line":58,"start_character":9,"end_line":59,"end_character":15},"updated":"2017-04-19 00:41:40.000000000","message":"I\u0027m a bit worried about what we\u0027re going to do with the overhead stuff in the compute which gets it\u0027s information from the virt driver (at least for libvirt, hyperv and xen):\n\nhttps://bugs.launchpad.net/nova/+bug/1683858\n\nWe don\u0027t account for that overhead in the allocations posted to placement today, but we also can\u0027t get that information in the scheduler.\n\nAnd if the compute sees allocations were already posted for the instance, then it won\u0027t calculate and test the claim, which uses the overhead values, but that\u0027s how we get around the bug for allocations today is if we pick the wrong compute node because placement doesn\u0027t have the overhead information, the claim can fail and trigger a retry. Once we move to no more claims in the compute, then we don\u0027t have that check anymore.\n\nJay talked about setting reserved higher on the resource provider inventory record, but that\u0027s per provider (compute node), the overhead stuff in the virt driver is per instance. He also thought we could put some kind of multiplier into an overheads table in the placement API which would be populated via the get_inventory() method on the virt driver, but that starts to get complicated math depending on what the virt drivers are doing today (some of it is multipliers, some of it is hard-coded reservations).\n\nCould we change the claim on the compute to just be a very lightweight check with the overhead calculation and if we go over then we fail and trigger a retry (knowing that retry capabilities go away with multi-cell + super-conductor...)?\n\nCould the scheduler create the allocations and return the host to conductor, and conductor make a new rpc call to the compute to get the overhead calculation and conductor could use that to tell us if we\u0027re over, at which point conductor would kick out that host, delete the allocations, and do the full retry through the scheduler again? That gets pretty gross, but it\u0027s an idea.\n\nAnother idea is we don\u0027t do a claim on the compute, but we get the overhead and add that to the existing allocation with PUT /allocations/{instance_id} if something changes. If that fails with a 409 because we\u0027re over the available inventory, then we fail and trigger a retry.","commit_id":"390543dfee30cf9127ec8fe2bd5472e7fb1efb37"},{"author":{"_account_id":1063,"name":"Ed Leafe","email":"ed@leafe.com","username":"ed-leafe"},"change_message_id":"2f4360397a52a7df0ea9acd7e17effa744d411ff","unresolved":false,"context_lines":[{"line_number":56,"context_line":"Then, the compute service would query the Placement API for knowing if there"},{"line_number":57,"context_line":"are existing allocations for that specific consumer (the instance) and if so,"},{"line_number":58,"context_line":"then the RT would not call the Placement API for adding allocations during the"},{"line_number":59,"context_line":"claim operation. That claim contextmanager will itself become lightweight"},{"line_number":60,"context_line":"because it will only verify PCI and NUMA resources that aren\u0027t yet provided to"},{"line_number":61,"context_line":"the Placement API."},{"line_number":62,"context_line":""}],"source_content_type":"text/x-rst","patch_set":5,"id":"7ffa3b31_f9fad472","line":59,"in_reply_to":"7ffa3b31_0c753fcb","updated":"2017-04-20 15:04:38.000000000","message":"Re: overhead - I think we\u0027re over-thinking and over-engineering\n\nOverhead is a cost of virtualization, just like the resources used by the hypervisor. Of course, it\u0027s difficult to calculate the *exact* amount, which is leading to all of these involved scenarios for doing exactly that. However, it isn\u0027t that hard to determine an upper bound for the total overhead for a fully-virtualized host - it would probably be consistent for each hypervisor type. If we just add that to the reserved amount when we define the resource provider, we don\u0027t have to do anything else.\n\nAdding it to reserved also fits the whole placement model much more cleanly. \"Reserved\" represents that part of the resource which isn\u0027t available to the consumers of the resource, and overhead clearly falls into that category.","commit_id":"390543dfee30cf9127ec8fe2bd5472e7fb1efb37"},{"author":{"_account_id":5754,"name":"Alex Xu","email":"hejie.xu@intel.com","username":"xuhj"},"change_message_id":"f33e276163f166797d98db6ff76b787c8c96105e","unresolved":false,"context_lines":[{"line_number":56,"context_line":"Then, the compute service would query the Placement API for knowing if there"},{"line_number":57,"context_line":"are existing allocations for that specific consumer (the instance) and if so,"},{"line_number":58,"context_line":"then the RT would not call the Placement API for adding allocations during the"},{"line_number":59,"context_line":"claim operation. That claim contextmanager will itself become lightweight"},{"line_number":60,"context_line":"because it will only verify PCI and NUMA resources that aren\u0027t yet provided to"},{"line_number":61,"context_line":"the Placement API."},{"line_number":62,"context_line":""}],"source_content_type":"text/x-rst","patch_set":5,"id":"7ffa3b31_b9ca1a4b","line":59,"in_reply_to":"7ffa3b31_22eb86dc","updated":"2017-04-20 08:51:36.000000000","message":"another idea:\n\nIt based on:\n#1 scheduler can know a ResourceProvider is a KVM compute node or Xen compute node. This can be done by Traits. We tag a KVM trait to KVM compute node. We tag a XEN trait to a XEN compute node.\n\n#2 conductor can know how much overhead for a flavor on all the virt drivers. This can be done by a library which invoked by nova-conductor. That library calculates the overhead for all the virt driver based on flavor. Basically, it moves all the implementation of \u0027estimate_instance_overhead\u0027 interface in all the virt driver to a library. (I didn\u0027t mean to a real library. it is called by nova-conductor locally. It is just new abstract interface). \n\n\nSo the whole process with real example is as below:\n\n#1  The user requests to boot up a server with flavor which requires 1 VCPU, 1024MB memory. And it requires \u0027fields.CPUEmulatorThreadsPolicy.ISOLATE\u0027.\n\n\n#2 The nova-conductor calculates the overhead for each virt-driver (by the new abstract interface). Then nova-conductor will know that:\n\n     a. If the host is KVM, the overhead is 1 VCPU. The request should be 2 VCPU and 1024MB Memory. (based on https://github.com/openstack/nova/blob/master/nova/virt/libvirt/driver.py#L713)\n\n     b.  If the host is Xen, the overhead is 13MB memory (based on https://github.com/openstack/nova/blob/master/nova/virt/xenapi/driver.py#L137). The request should be 1VCPU and 1037MB.\n\n\n#3 The nova-conduct sends the request to the placement. The request is as below:\n\n\n      *Please give me a ResourceProvider, the ResourceProvider should have KVM trait and 2VCPU and 1024MB memory, or the ResourceProvider should have XEN trait and 1 VCPU and 1037MB.\n\n\n#4 The placement return one ResourceProvider which matches the one of two requirements.\n\n\nThen we only need one request to finish all of those.\n\nThen next problem is that the current \u0027POST /allocations\u0027 API is not enough to express that complex request. At least it didn\u0027t consider the trait for now.\n\n\nSo we need a new interface it can request like this:\n{\n    \u0027or\u0027: [\n         { \n              \u0027VCPU\u0027: 2\n              \u0027MEMORY\u0027: 1024MB\n              \u0027required_traits\u0027: [\u0027KVM\u0027]\n         },\n         {\n              \u0027VCPU\u0027: 1\n              \u0027MEMORY\u0027: 1037MB\n              \u0027required_traits\u0027: [\u0027XEN\u0027]\n         }\n    ]\n}\n\n(This isn\u0027t the real API which I proposed, I\u0027m still thinking about the API. This is just for people can understand what I\u0027m saying)","commit_id":"390543dfee30cf9127ec8fe2bd5472e7fb1efb37"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"f808224e0ab1cc811599e98d51c2fd5237409c58","unresolved":false,"context_lines":[{"line_number":56,"context_line":"Then, the compute service would query the Placement API for knowing if there"},{"line_number":57,"context_line":"are existing allocations for that specific consumer (the instance) and if so,"},{"line_number":58,"context_line":"then the RT would not call the Placement API for adding allocations during the"},{"line_number":59,"context_line":"claim operation. That claim contextmanager will itself become lightweight"},{"line_number":60,"context_line":"because it will only verify PCI and NUMA resources that aren\u0027t yet provided to"},{"line_number":61,"context_line":"the Placement API."},{"line_number":62,"context_line":""}],"source_content_type":"text/x-rst","patch_set":5,"id":"7ffa3b31_22eb86dc","line":59,"in_reply_to":"7ffa3b31_5708a42a","updated":"2017-04-19 15:59:47.000000000","message":"In IRC this morning we discussed some of the options and Dan didn\u0027t like the idea of posting allocations based on the flavor in the control plane and then updating the allocations with the overheads on the compute, since if we go over and fail we fail the build. To me that\u0027s not too bad as long as we had retries, but those go away with super-conductor, so we can\u0027t rely on that model long-term.\n\nWe also talked about the option of doing something like:\n\n- conductor calls scheduler.select_destinations to pick a host\n- conductor makes an rpc call to the compute for the chosen\n  host to get the overheads for the instance\n- conductor posts the allocations to placement for the flavor\n  plus the overheads (we could also cache the overhead at\n  some point based on a hash key of\n  flavorid/image_meta/hypervisor_type/hypervisor_version)\n- if the posted allocations are successful, we build the\n  instance on that host, else we retry through the scheduler\n  to pick a new host.\n\nI tend to like this option since it fixes bug 1683858 in that the allocations in placement have the actual usage, like what we claim today in the RT (flavor + overhead). Having the actual real usage in the allocations records means that later scheduling requests get the benefit of placement having that information, which could result in fewer retries when we find out that we can\u0027t post allocations to a given provider because the overhead from the compute tips it over what\u0027s available inventory on that provider. This is the most complicated solution but does not require any operator involvement.\n\nYet another option we discussed, that Chris mentions above, is not including the overhead in the allocations and make up for the overhead with extra buffer in the resource providers reserved inventory. We can use the get_inventory() method in the virt drivers to specify reserved inventory for each resource class (VCPU, MEMORY_MB and DISK_GB) and that could be configurable (as it is today using reserved_host_memory_mb and reserved_host_disk_mb config options - we\u0027d also have to add one for cpu for pinning I think). This is the simplest solution code-wise, but it does put a burden on the operator to set the proper configuration values and they could have a hard time estimating this with expected capacity on a given host.\n\nChris, to answer your question about quotas, no we don\u0027t account for the overhead in the quota usage. The quota usage is reserved in the API here:\n\nhttps://github.com/openstack/nova/blob/master/nova/compute/api.py#L997\n\nAnd if the build requests are created successfully in the API, before casting to conductor to pick a host, we commit the reservations:\n\nhttps://github.com/openstack/nova/blob/master/nova/compute/api.py#L1105","commit_id":"390543dfee30cf9127ec8fe2bd5472e7fb1efb37"},{"author":{"_account_id":5754,"name":"Alex Xu","email":"hejie.xu@intel.com","username":"xuhj"},"change_message_id":"663738f56c62aeb620b41770d2598ca2e301f061","unresolved":false,"context_lines":[{"line_number":56,"context_line":"Then, the compute service would query the Placement API for knowing if there"},{"line_number":57,"context_line":"are existing allocations for that specific consumer (the instance) and if so,"},{"line_number":58,"context_line":"then the RT would not call the Placement API for adding allocations during the"},{"line_number":59,"context_line":"claim operation. That claim contextmanager will itself become lightweight"},{"line_number":60,"context_line":"because it will only verify PCI and NUMA resources that aren\u0027t yet provided to"},{"line_number":61,"context_line":"the Placement API."},{"line_number":62,"context_line":""}],"source_content_type":"text/x-rst","patch_set":5,"id":"7ffa3b31_c6d38f03","line":59,"in_reply_to":"7ffa3b31_b9ca1a4b","updated":"2017-04-21 14:33:23.000000000","message":"For my idea, one more append. I missed that currently we still use \u0027GET /resource_providers?resources\u003d...\u0027. That API is more easy to implement \"Please give me a ResourceProvider, the ResourceProvider should have KVM trait and 2VCPU and 1024MB memory, or the ResourceProvider should have XEN trait and 1 VCPU and 1037MB\".\n\nBut we need return to the ResourceProvider to the conductor instead of  just a hostname. The conductor needs to know whether the ResourceProvider have KVM trait or XEN trait.","commit_id":"390543dfee30cf9127ec8fe2bd5472e7fb1efb37"},{"author":{"_account_id":782,"name":"John Garbutt","email":"john@johngarbutt.com","username":"johngarbutt"},"change_message_id":"5730478fb65efe7287da175a4a2e4b3f7ec61cb0","unresolved":false,"context_lines":[{"line_number":56,"context_line":"Then, the compute service would query the Placement API for knowing if there"},{"line_number":57,"context_line":"are existing allocations for that specific consumer (the instance) and if so,"},{"line_number":58,"context_line":"then the RT would not call the Placement API for adding allocations during the"},{"line_number":59,"context_line":"claim operation. That claim contextmanager will itself become lightweight"},{"line_number":60,"context_line":"because it will only verify PCI and NUMA resources that aren\u0027t yet provided to"},{"line_number":61,"context_line":"the Placement API."},{"line_number":62,"context_line":""}],"source_content_type":"text/x-rst","patch_set":5,"id":"7ffa3b31_0c753fcb","line":59,"in_reply_to":"7ffa3b31_b9ca1a4b","updated":"2017-04-20 14:19:33.000000000","message":"I am +1 matt\u0027s extra RPC call after picking the host to get an updated more accurate overhead. Keeps any retry in the conductor before we write out the DB entires in the cell DB, etc.","commit_id":"390543dfee30cf9127ec8fe2bd5472e7fb1efb37"},{"author":{"_account_id":11564,"name":"Chris Dent","email":"cdent@anticdent.org","username":"chdent"},"change_message_id":"9f1d36941103eb2f2be2e45ddde6a285884d9b0b","unresolved":false,"context_lines":[{"line_number":56,"context_line":"Then, the compute service would query the Placement API for knowing if there"},{"line_number":57,"context_line":"are existing allocations for that specific consumer (the instance) and if so,"},{"line_number":58,"context_line":"then the RT would not call the Placement API for adding allocations during the"},{"line_number":59,"context_line":"claim operation. That claim contextmanager will itself become lightweight"},{"line_number":60,"context_line":"because it will only verify PCI and NUMA resources that aren\u0027t yet provided to"},{"line_number":61,"context_line":"the Placement API."},{"line_number":62,"context_line":""}],"source_content_type":"text/x-rst","patch_set":5,"id":"7ffa3b31_5708a42a","line":59,"in_reply_to":"7ffa3b31_d75f1f70","updated":"2017-04-19 09:29:50.000000000","message":"Are overheads currently counted when we do accounting for quota? Since the intent is to use allocations for quota, if fixing overhead changes those numbers that may present a problem because it will change how humans need to think about how much an m1.tiny (or whatever) costs in terms of quota.\n\nIf it is accounted for then we will have to \"fix\" the allocations and to me the simplest thing would seem to be what matt suggests: write a first allocation from the scheduler and then fix it from nova-compute.\n\nIf it is not, then I still hold out hope that we can figure out some way to incorporate a global estimate of \"expected overhead on this compute node (given this virt driver)\" into the \u0027reserved\u0027 part of the inventory that compute node writes for itself. This would presumably need to fudge on the side of \"somewhat too much\" for safety (which may be good anyway as people habitually oversubscribe their hosts anyway and that sucks).\n\nI\u0027m not keen on the idea of an overheads table as it\u0027s yet another moving part.","commit_id":"390543dfee30cf9127ec8fe2bd5472e7fb1efb37"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"fa50a80871a10b6fef87eb9bc1bc3a679b29d213","unresolved":false,"context_lines":[{"line_number":64,"context_line":"the instance) will delete original allocations by having the conductor calling"},{"line_number":65,"context_line":"DELETE /allocations/{instance_id} and be done with it."},{"line_number":66,"context_line":"Destroying an instance is a special case : either we directly delete the"},{"line_number":67,"context_line":"instance in the Compute API, or the compute service deletes the instance after"},{"line_number":68,"context_line":"a timeout. Given that, the Placement API should be called for deleting the"},{"line_number":69,"context_line":"related instances in the two services for each of the possibilities."},{"line_number":70,"context_line":""},{"line_number":71,"context_line":"That all said, there is a current problem with reschedules for Cells V2 where"}],"source_content_type":"text/x-rst","patch_set":5,"id":"7ffa3b31_77504b5e","line":68,"range":{"start_line":67,"start_character":29,"end_line":68,"end_character":9},"updated":"2017-04-19 00:41:40.000000000","message":"or the compute service deletes a soft-deleted instance after the reclaim interval has expired.","commit_id":"390543dfee30cf9127ec8fe2bd5472e7fb1efb37"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"fa50a80871a10b6fef87eb9bc1bc3a679b29d213","unresolved":false,"context_lines":[{"line_number":80,"context_line":""},{"line_number":81,"context_line":"We could consider that Placement doesn\u0027t need to know the on-going resource"},{"line_number":82,"context_line":"creations and that it is only a capacity issue."},{"line_number":83,"context_line":"We could also pass the allocations thru RPC when calling the compute service"},{"line_number":84,"context_line":"by adding them as a method argument, but given the Allocation objects are a"},{"line_number":85,"context_line":"Placement specific model, we prefer not use them directly in Nova."},{"line_number":86,"context_line":"One last argument could be about where to call Placement API for both creating"}],"source_content_type":"text/x-rst","patch_set":5,"id":"7ffa3b31_37c88320","line":83,"updated":"2017-04-19 00:41:40.000000000","message":"nit: put a blank line between alternatives","commit_id":"390543dfee30cf9127ec8fe2bd5472e7fb1efb37"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"fa50a80871a10b6fef87eb9bc1bc3a679b29d213","unresolved":false,"context_lines":[{"line_number":83,"context_line":"We could also pass the allocations thru RPC when calling the compute service"},{"line_number":84,"context_line":"by adding them as a method argument, but given the Allocation objects are a"},{"line_number":85,"context_line":"Placement specific model, we prefer not use them directly in Nova."},{"line_number":86,"context_line":"One last argument could be about where to call Placement API for both creating"},{"line_number":87,"context_line":"allocations and deleting them, whether in the conductor or in the scheduler"},{"line_number":88,"context_line":"services, but we leave that for a review discussion during the implementation."},{"line_number":89,"context_line":"We possibly reserve ourselves to verify the performance between those two"}],"source_content_type":"text/x-rst","patch_set":5,"id":"7ffa3b31_17cb471e","line":86,"updated":"2017-04-19 00:41:40.000000000","message":"same","commit_id":"390543dfee30cf9127ec8fe2bd5472e7fb1efb37"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"fa50a80871a10b6fef87eb9bc1bc3a679b29d213","unresolved":false,"context_lines":[{"line_number":117,"context_line":"Performance Impact"},{"line_number":118,"context_line":"------------------"},{"line_number":119,"context_line":""},{"line_number":120,"context_line":"Placement HTTP calls would be made within the conductor service instead of"},{"line_number":121,"context_line":"compute services for allocating instances."},{"line_number":122,"context_line":"Nothing really changes from a performance perspective, just a bit more RPC"},{"line_number":123,"context_line":"payload."}],"source_content_type":"text/x-rst","patch_set":5,"id":"7ffa3b31_f7d1fbce","line":120,"range":{"start_line":120,"start_character":46,"end_line":120,"end_character":55},"updated":"2017-04-19 00:41:40.000000000","message":"scheduler","commit_id":"390543dfee30cf9127ec8fe2bd5472e7fb1efb37"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"fa50a80871a10b6fef87eb9bc1bc3a679b29d213","unresolved":false,"context_lines":[{"line_number":119,"context_line":""},{"line_number":120,"context_line":"Placement HTTP calls would be made within the conductor service instead of"},{"line_number":121,"context_line":"compute services for allocating instances."},{"line_number":122,"context_line":"Nothing really changes from a performance perspective, just a bit more RPC"},{"line_number":123,"context_line":"payload."},{"line_number":124,"context_line":""},{"line_number":125,"context_line":"Other deployer impact"},{"line_number":126,"context_line":"---------------------"}],"source_content_type":"text/x-rst","patch_set":5,"id":"7ffa3b31_b7db73eb","line":123,"range":{"start_line":122,"start_character":55,"end_line":123,"end_character":8},"updated":"2017-04-19 00:41:40.000000000","message":"remove this part","commit_id":"390543dfee30cf9127ec8fe2bd5472e7fb1efb37"},{"author":{"_account_id":7,"name":"Jay Pipes","email":"jaypipes@gmail.com","username":"jaypipes"},"change_message_id":"6260d79e8d81b05ad72de70e129050644e14da9a","unresolved":false,"context_lines":[{"line_number":25,"context_line":"updating allocation values in the RT is called a \u0027claim\u0027."},{"line_number":26,"context_line":""},{"line_number":27,"context_line":"While this model fits very well for a cloud having lots of capacity left, we"},{"line_number":28,"context_line":"know that this can be suboptimal because of requests concurrency."},{"line_number":29,"context_line":""},{"line_number":30,"context_line":"In this spec, we propose to have the scheduler allocate resources instead of"},{"line_number":31,"context_line":"the compute node."}],"source_content_type":"text/x-rst","patch_set":6,"id":"5ff73747_c1afa692","line":28,"updated":"2017-04-24 14:28:21.000000000","message":"This last sentence part is awkward. Perhaps change to \", but in more resource-constrained environments can lead to contention and an increase in retry operations.\"","commit_id":"7f9a9131d501f9e1d51f1d1b14a1d3d696062e6f"},{"author":{"_account_id":7,"name":"Jay Pipes","email":"jaypipes@gmail.com","username":"jaypipes"},"change_message_id":"6260d79e8d81b05ad72de70e129050644e14da9a","unresolved":false,"context_lines":[{"line_number":35,"context_line":"and once it decrements the resource usage in the consume_from_request()"},{"line_number":36,"context_line":"synchronous operation, it would post the allocations against the found target."},{"line_number":37,"context_line":"In this way, we both reduce the occurrence of retry operations as well as the"},{"line_number":38,"context_line":"time period that race conditions can occur."},{"line_number":39,"context_line":""},{"line_number":40,"context_line":"Use Cases"},{"line_number":41,"context_line":"---------"}],"source_content_type":"text/x-rst","patch_set":6,"id":"5ff73747_a1949a45","line":38,"updated":"2017-04-24 14:28:21.000000000","message":"++","commit_id":"7f9a9131d501f9e1d51f1d1b14a1d3d696062e6f"},{"author":{"_account_id":7,"name":"Jay Pipes","email":"jaypipes@gmail.com","username":"jaypipes"},"change_message_id":"6260d79e8d81b05ad72de70e129050644e14da9a","unresolved":false,"context_lines":[{"line_number":51,"context_line":"letting the ResourceTracker (RT) post the allocations, we propose the scheduler"},{"line_number":52,"context_line":"preemptively write those allocations. We will use a service version check in"},{"line_number":53,"context_line":"the scheduler to ensure all compute hosts in the system can handle a request"},{"line_number":54,"context_line":"that has had allocations already written by the compute service."},{"line_number":55,"context_line":""},{"line_number":56,"context_line":"Then, the compute service would query the Placement API for knowing if there"},{"line_number":57,"context_line":"are existing allocations for that specific consumer (the instance) and if so,"}],"source_content_type":"text/x-rst","patch_set":6,"id":"5ff73747_81919e55","line":54,"updated":"2017-04-24 14:28:21.000000000","message":"s/by the compute service/by the scheduler service/\n\nThat\u0027s a pretty important thing and the reason for the -1 :)","commit_id":"7f9a9131d501f9e1d51f1d1b14a1d3d696062e6f"},{"author":{"_account_id":7,"name":"Jay Pipes","email":"jaypipes@gmail.com","username":"jaypipes"},"change_message_id":"6260d79e8d81b05ad72de70e129050644e14da9a","unresolved":false,"context_lines":[{"line_number":53,"context_line":"the scheduler to ensure all compute hosts in the system can handle a request"},{"line_number":54,"context_line":"that has had allocations already written by the compute service."},{"line_number":55,"context_line":""},{"line_number":56,"context_line":"Then, the compute service would query the Placement API for knowing if there"},{"line_number":57,"context_line":"are existing allocations for that specific consumer (the instance) and if so,"},{"line_number":58,"context_line":"then the RT would not call the Placement API for adding allocations during the"},{"line_number":59,"context_line":"claim operation. That claim contextmanager will itself become lightweight"}],"source_content_type":"text/x-rst","patch_set":6,"id":"5ff73747_619e9264","line":56,"updated":"2017-04-24 14:28:21.000000000","message":"s/knowing if there are existing//","commit_id":"7f9a9131d501f9e1d51f1d1b14a1d3d696062e6f"},{"author":{"_account_id":7,"name":"Jay Pipes","email":"jaypipes@gmail.com","username":"jaypipes"},"change_message_id":"6260d79e8d81b05ad72de70e129050644e14da9a","unresolved":false,"context_lines":[{"line_number":55,"context_line":""},{"line_number":56,"context_line":"Then, the compute service would query the Placement API for knowing if there"},{"line_number":57,"context_line":"are existing allocations for that specific consumer (the instance) and if so,"},{"line_number":58,"context_line":"then the RT would not call the Placement API for adding allocations during the"},{"line_number":59,"context_line":"claim operation. That claim contextmanager will itself become lightweight"},{"line_number":60,"context_line":"because it will only verify PCI and NUMA resources that aren\u0027t yet provided to"},{"line_number":61,"context_line":"the Placement API."}],"source_content_type":"text/x-rst","patch_set":6,"id":"5ff73747_419b9671","line":58,"updated":"2017-04-24 14:28:21.000000000","message":"s/for adding/to add/","commit_id":"7f9a9131d501f9e1d51f1d1b14a1d3d696062e6f"},{"author":{"_account_id":7,"name":"Jay Pipes","email":"jaypipes@gmail.com","username":"jaypipes"},"change_message_id":"6260d79e8d81b05ad72de70e129050644e14da9a","unresolved":false,"context_lines":[{"line_number":57,"context_line":"are existing allocations for that specific consumer (the instance) and if so,"},{"line_number":58,"context_line":"then the RT would not call the Placement API for adding allocations during the"},{"line_number":59,"context_line":"claim operation. That claim contextmanager will itself become lightweight"},{"line_number":60,"context_line":"because it will only verify PCI and NUMA resources that aren\u0027t yet provided to"},{"line_number":61,"context_line":"the Placement API."},{"line_number":62,"context_line":"Given virt drivers can return the overhead of a specific instance that is then"},{"line_number":63,"context_line":"taken into account for claiming the resource in the RT, and given overheads"}],"source_content_type":"text/x-rst","patch_set":6,"id":"5ff73747_218aca1a","line":60,"updated":"2017-04-24 14:28:21.000000000","message":"s/provided to/handled by/","commit_id":"7f9a9131d501f9e1d51f1d1b14a1d3d696062e6f"},{"author":{"_account_id":7,"name":"Jay Pipes","email":"jaypipes@gmail.com","username":"jaypipes"},"change_message_id":"6260d79e8d81b05ad72de70e129050644e14da9a","unresolved":false,"context_lines":[{"line_number":69,"context_line":"compute node."},{"line_number":70,"context_line":""},{"line_number":71,"context_line":"Move operations and reschedules (if unexpected failures happen when spawning"},{"line_number":72,"context_line":"the instance) will delete original allocations by having the conductor calling"},{"line_number":73,"context_line":"DELETE /allocations/{instance_id} and be done with it. Destroying an instance"},{"line_number":74,"context_line":"is a special case : either we directly delete the instance in the Compute API,"},{"line_number":75,"context_line":"or the compute service deletes a soft-deleted instance after the reclaim"}],"source_content_type":"text/x-rst","patch_set":6,"id":"5ff73747_018fce2a","line":72,"updated":"2017-04-24 14:28:21.000000000","message":"s/conductor calling/conductor call/","commit_id":"7f9a9131d501f9e1d51f1d1b14a1d3d696062e6f"},{"author":{"_account_id":7,"name":"Jay Pipes","email":"jaypipes@gmail.com","username":"jaypipes"},"change_message_id":"6260d79e8d81b05ad72de70e129050644e14da9a","unresolved":false,"context_lines":[{"line_number":75,"context_line":"or the compute service deletes a soft-deleted instance after the reclaim"},{"line_number":76,"context_line":"interval has expired. Given that, the Placement API should be called for"},{"line_number":77,"context_line":"deleting the related instances in the two services for each of the"},{"line_number":78,"context_line":"possibilities."},{"line_number":79,"context_line":""},{"line_number":80,"context_line":"That all said, there is a current problem with reschedules for Cells V2 where"},{"line_number":81,"context_line":"the compute calls back conductor. Unfortunately, given it would call a cell"}],"source_content_type":"text/x-rst","patch_set":6,"id":"5ff73747_e183c203","line":78,"updated":"2017-04-24 14:28:21.000000000","message":"The last two sentences above are very difficult to understand. Perhaps replace with:\n\n\"When terminating an instance, the associated allocations will be deleted from the Placement API when the instance is physically removed on the compute host. In the case of \"soft-deleted\" instances, this would occur when during the reclaim interval operation.\"","commit_id":"7f9a9131d501f9e1d51f1d1b14a1d3d696062e6f"},{"author":{"_account_id":7,"name":"Jay Pipes","email":"jaypipes@gmail.com","username":"jaypipes"},"change_message_id":"6260d79e8d81b05ad72de70e129050644e14da9a","unresolved":false,"context_lines":[{"line_number":88,"context_line":"------------"},{"line_number":89,"context_line":""},{"line_number":90,"context_line":"We could consider that Placement doesn\u0027t need to know the on-going resource"},{"line_number":91,"context_line":"creations and that it is only a capacity issue."},{"line_number":92,"context_line":""},{"line_number":93,"context_line":"We could also pass the allocations thru RPC when calling the compute service"},{"line_number":94,"context_line":"by adding them as a method argument, but given the Allocation objects are a"}],"source_content_type":"text/x-rst","patch_set":6,"id":"5ff73747_c188c620","line":91,"updated":"2017-04-24 14:28:21.000000000","message":"Recommend removing the above statement. It\u0027s actually not something we ever really considered and doesn\u0027t make all that much sense, IMHO.","commit_id":"7f9a9131d501f9e1d51f1d1b14a1d3d696062e6f"},{"author":{"_account_id":7166,"name":"Sylvain Bauza","email":"sbauza@redhat.com","username":"sbauza"},"change_message_id":"715668c13db57ab0cd35738075dbe509094c235d","unresolved":false,"context_lines":[{"line_number":170,"context_line":"  created, and if so, don\u0027t POST allocations to Placement service."},{"line_number":171,"context_line":"* Modify conductor to DELETE /allocations/\u003cinstance_id\u003e if this is a reschedule"},{"line_number":172,"context_line":"  or a move operation."},{"line_number":173,"context_line":"* Modify API and compute services instance destroy methods to also delete"},{"line_number":174,"context_line":"  related allocations."},{"line_number":175,"context_line":"* Modify scheduler to POST allocations to Placement if all computes are new."},{"line_number":176,"context_line":""},{"line_number":177,"context_line":"Dependencies"}],"source_content_type":"text/x-rst","patch_set":7,"id":"5ff73747_349e6154","line":174,"range":{"start_line":173,"start_character":0,"end_line":174,"end_character":22},"updated":"2017-04-25 15:18:59.000000000","message":"sh*t, forgot to remove that line...","commit_id":"546eb2fb6228a9de4d41cfe9edd1ef13cb26ebeb"},{"author":{"_account_id":4393,"name":"Dan Smith","email":"dms@danplanet.com","username":"danms"},"change_message_id":"df838277cda34bf1a036697d0667ea891bccea42","unresolved":false,"context_lines":[{"line_number":24,"context_line":"providers, e.g. compute nodes, in the Placement service. The process of"},{"line_number":25,"context_line":"updating allocation values in the RT is called a \u0027claim\u0027."},{"line_number":26,"context_line":""},{"line_number":27,"context_line":"While this model fits very well for a cloud having lots of capacity left, but"},{"line_number":28,"context_line":"in more resource-constrained environments can lead to contention and an"},{"line_number":29,"context_line":"increase in retry operations."},{"line_number":30,"context_line":""}],"source_content_type":"text/x-rst","patch_set":8,"id":"5ff73747_124495b6","line":27,"range":{"start_line":27,"start_character":0,"end_line":27,"end_character":72},"updated":"2017-04-25 16:18:29.000000000","message":"I don\u0027t think I agree with this, but that\u0027s fine :D","commit_id":"32a4da557b4686983054f3b3497475f7ea0f46cd"},{"author":{"_account_id":11564,"name":"Chris Dent","email":"cdent@anticdent.org","username":"chdent"},"change_message_id":"f08b51dd3a53eae6abfb402d28a0a83d2b997bad","unresolved":false,"context_lines":[{"line_number":24,"context_line":"providers, e.g. compute nodes, in the Placement service. The process of"},{"line_number":25,"context_line":"updating allocation values in the RT is called a \u0027claim\u0027."},{"line_number":26,"context_line":""},{"line_number":27,"context_line":"While this model fits very well for a cloud having lots of capacity left, but"},{"line_number":28,"context_line":"in more resource-constrained environments can lead to contention and an"},{"line_number":29,"context_line":"increase in retry operations."},{"line_number":30,"context_line":""}],"source_content_type":"text/x-rst","patch_set":8,"id":"5ff73747_23eaa0ab","line":27,"in_reply_to":"5ff73747_124495b6","updated":"2017-04-25 17:31:27.000000000","message":"s/but//","commit_id":"32a4da557b4686983054f3b3497475f7ea0f46cd"},{"author":{"_account_id":11564,"name":"Chris Dent","email":"cdent@anticdent.org","username":"chdent"},"change_message_id":"f08b51dd3a53eae6abfb402d28a0a83d2b997bad","unresolved":false,"context_lines":[{"line_number":25,"context_line":"updating allocation values in the RT is called a \u0027claim\u0027."},{"line_number":26,"context_line":""},{"line_number":27,"context_line":"While this model fits very well for a cloud having lots of capacity left, but"},{"line_number":28,"context_line":"in more resource-constrained environments can lead to contention and an"},{"line_number":29,"context_line":"increase in retry operations."},{"line_number":30,"context_line":""},{"line_number":31,"context_line":"In this spec, we propose to have the scheduler allocate resources instead of"}],"source_content_type":"text/x-rst","patch_set":8,"id":"5ff73747_c3dadc15","line":28,"updated":"2017-04-25 17:31:27.000000000","message":"s/can/it can/","commit_id":"32a4da557b4686983054f3b3497475f7ea0f46cd"},{"author":{"_account_id":1063,"name":"Ed Leafe","email":"ed@leafe.com","username":"ed-leafe"},"change_message_id":"1cb636e35c61085e8015f20114d70a213a54ee74","unresolved":false,"context_lines":[{"line_number":26,"context_line":""},{"line_number":27,"context_line":"While this model fits very well for a cloud having lots of capacity left, but"},{"line_number":28,"context_line":"in more resource-constrained environments can lead to contention and an"},{"line_number":29,"context_line":"increase in retry operations."},{"line_number":30,"context_line":""},{"line_number":31,"context_line":"In this spec, we propose to have the scheduler allocate resources instead of"},{"line_number":32,"context_line":"the compute node."}],"source_content_type":"text/x-rst","patch_set":8,"id":"5ff73747_087c62b3","line":29,"range":{"start_line":29,"start_character":28,"end_line":29,"end_character":29},"updated":"2017-04-25 17:00:56.000000000","message":"It also effectively prohibits running multiple schedulers to scale out, as that would increase races between them.","commit_id":"32a4da557b4686983054f3b3497475f7ea0f46cd"},{"author":{"_account_id":1063,"name":"Ed Leafe","email":"ed@leafe.com","username":"ed-leafe"},"change_message_id":"1cb636e35c61085e8015f20114d70a213a54ee74","unresolved":false,"context_lines":[{"line_number":31,"context_line":"In this spec, we propose to have the scheduler allocate resources instead of"},{"line_number":32,"context_line":"the compute node."},{"line_number":33,"context_line":"Given the Placement API does not implement all the scheduler filters and"},{"line_number":34,"context_line":"weighers, the proposal is that the scheduler would first get a list of"},{"line_number":35,"context_line":"resource providers, iterate over those with the help of filters and weighters,"},{"line_number":36,"context_line":"and once it decrements the resource usage in the consume_from_request()"},{"line_number":37,"context_line":"synchronous operation, it would post the allocations against the found target."},{"line_number":38,"context_line":"In this way, we both reduce the occurrence of retry operations as well as the"}],"source_content_type":"text/x-rst","patch_set":8,"id":"5ff73747_830cd53b","line":35,"range":{"start_line":34,"start_character":57,"end_line":35,"end_character":18},"updated":"2017-04-25 17:00:56.000000000","message":"...that can satisfy the requested amounts for the basic resource classes...","commit_id":"32a4da557b4686983054f3b3497475f7ea0f46cd"},{"author":{"_account_id":11564,"name":"Chris Dent","email":"cdent@anticdent.org","username":"chdent"},"change_message_id":"f08b51dd3a53eae6abfb402d28a0a83d2b997bad","unresolved":false,"context_lines":[{"line_number":32,"context_line":"the compute node."},{"line_number":33,"context_line":"Given the Placement API does not implement all the scheduler filters and"},{"line_number":34,"context_line":"weighers, the proposal is that the scheduler would first get a list of"},{"line_number":35,"context_line":"resource providers, iterate over those with the help of filters and weighters,"},{"line_number":36,"context_line":"and once it decrements the resource usage in the consume_from_request()"},{"line_number":37,"context_line":"synchronous operation, it would post the allocations against the found target."},{"line_number":38,"context_line":"In this way, we both reduce the occurrence of retry operations as well as the"}],"source_content_type":"text/x-rst","patch_set":8,"id":"5ff73747_e3f4b889","line":35,"in_reply_to":"5ff73747_830cd53b","updated":"2017-04-25 17:31:27.000000000","message":"+1","commit_id":"32a4da557b4686983054f3b3497475f7ea0f46cd"},{"author":{"_account_id":1063,"name":"Ed Leafe","email":"ed@leafe.com","username":"ed-leafe"},"change_message_id":"1cb636e35c61085e8015f20114d70a213a54ee74","unresolved":false,"context_lines":[{"line_number":33,"context_line":"Given the Placement API does not implement all the scheduler filters and"},{"line_number":34,"context_line":"weighers, the proposal is that the scheduler would first get a list of"},{"line_number":35,"context_line":"resource providers, iterate over those with the help of filters and weighters,"},{"line_number":36,"context_line":"and once it decrements the resource usage in the consume_from_request()"},{"line_number":37,"context_line":"synchronous operation, it would post the allocations against the found target."},{"line_number":38,"context_line":"In this way, we both reduce the occurrence of retry operations as well as the"},{"line_number":39,"context_line":"time period that race conditions can occur."}],"source_content_type":"text/x-rst","patch_set":8,"id":"5ff73747_83f615f9","line":36,"range":{"start_line":36,"start_character":4,"end_line":36,"end_character":71},"updated":"2017-04-25 17:00:56.000000000","message":"I would think that it should go just before, so that if there is a race, we just move on to the next host, and not have to undo the consume call.","commit_id":"32a4da557b4686983054f3b3497475f7ea0f46cd"},{"author":{"_account_id":11564,"name":"Chris Dent","email":"cdent@anticdent.org","username":"chdent"},"change_message_id":"f08b51dd3a53eae6abfb402d28a0a83d2b997bad","unresolved":false,"context_lines":[{"line_number":33,"context_line":"Given the Placement API does not implement all the scheduler filters and"},{"line_number":34,"context_line":"weighers, the proposal is that the scheduler would first get a list of"},{"line_number":35,"context_line":"resource providers, iterate over those with the help of filters and weighters,"},{"line_number":36,"context_line":"and once it decrements the resource usage in the consume_from_request()"},{"line_number":37,"context_line":"synchronous operation, it would post the allocations against the found target."},{"line_number":38,"context_line":"In this way, we both reduce the occurrence of retry operations as well as the"},{"line_number":39,"context_line":"time period that race conditions can occur."}],"source_content_type":"text/x-rst","patch_set":8,"id":"5ff73747_e339583f","line":36,"in_reply_to":"5ff73747_83f615f9","updated":"2017-04-25 17:31:27.000000000","message":"+1","commit_id":"32a4da557b4686983054f3b3497475f7ea0f46cd"},{"author":{"_account_id":1063,"name":"Ed Leafe","email":"ed@leafe.com","username":"ed-leafe"},"change_message_id":"1cb636e35c61085e8015f20114d70a213a54ee74","unresolved":false,"context_lines":[{"line_number":43,"context_line":""},{"line_number":44,"context_line":"As an operator, I\u0027d like to minimize the amount of time between requesting a"},{"line_number":45,"context_line":"resource and actually consuming that resource in order to avoid cascading"},{"line_number":46,"context_line":"scheduling failures."},{"line_number":47,"context_line":""},{"line_number":48,"context_line":"Proposed change"},{"line_number":49,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"}],"source_content_type":"text/x-rst","patch_set":8,"id":"5ff73747_235501c7","line":46,"updated":"2017-04-25 17:00:56.000000000","message":"I\u0027m not sure that\u0027s what an operator is thinking about. Perhaps \"I\u0027d like to be better able to scale out my schedulers to better handle high request loads\" or something along those lines.","commit_id":"32a4da557b4686983054f3b3497475f7ea0f46cd"},{"author":{"_account_id":1063,"name":"Ed Leafe","email":"ed@leafe.com","username":"ed-leafe"},"change_message_id":"1cb636e35c61085e8015f20114d70a213a54ee74","unresolved":false,"context_lines":[{"line_number":50,"context_line":""},{"line_number":51,"context_line":"The spec targets to modify when/where allocations are created and instead of"},{"line_number":52,"context_line":"letting the ResourceTracker (RT) post the allocations, we propose the scheduler"},{"line_number":53,"context_line":"preemptively write those allocations. We will use a service version check in"},{"line_number":54,"context_line":"the scheduler to ensure all compute hosts in the system can handle a request"},{"line_number":55,"context_line":"that has had allocations already written by the scheduler service."},{"line_number":56,"context_line":""}],"source_content_type":"text/x-rst","patch_set":8,"id":"5ff73747_a35a31f0","line":53,"range":{"start_line":53,"start_character":0,"end_line":53,"end_character":12},"updated":"2017-04-25 17:00:56.000000000","message":"not needed.","commit_id":"32a4da557b4686983054f3b3497475f7ea0f46cd"},{"author":{"_account_id":4393,"name":"Dan Smith","email":"dms@danplanet.com","username":"danms"},"change_message_id":"df838277cda34bf1a036697d0667ea891bccea42","unresolved":false,"context_lines":[{"line_number":63,"context_line":"taken into account for claiming the resource in the RT, and given overheads"},{"line_number":64,"context_line":"are very specific per hypervisor type, version and instance flavors, a proposed"},{"line_number":65,"context_line":"trade-off for having placement be able to correctly verify those overheads is"},{"line_number":66,"context_line":"to ask operators to update the current \u0027reserved\u0027 configuration options that"},{"line_number":67,"context_line":"relate to the hypervisor offset amount needed for running by capping it to the"},{"line_number":68,"context_line":"maximum amount per resource class they think it would require to run on the"},{"line_number":69,"context_line":"compute node."},{"line_number":70,"context_line":""},{"line_number":71,"context_line":"Move operations and reschedules (if unexpected failures happen when spawning"},{"line_number":72,"context_line":"the instance) will delete original allocations by having the conductor call"}],"source_content_type":"text/x-rst","patch_set":8,"id":"5ff73747_4d46fc23","line":69,"range":{"start_line":66,"start_character":3,"end_line":69,"end_character":13},"updated":"2017-04-25 16:18:29.000000000","message":"I thought we agreed that we can just have the compute node update the reserved amount itself? Like, config says reserve 2G of memory, so after we get the instance, we calculate the overhead of all our instances and update our reserved amount to 2G+$overhead?\n\nMaybe we said that can be a second phase? Either way it should be something we can do eventually I think.","commit_id":"32a4da557b4686983054f3b3497475f7ea0f46cd"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"19be12c9f531e65b650a7210442d1d852d0076f0","unresolved":false,"context_lines":[{"line_number":63,"context_line":"taken into account for claiming the resource in the RT, and given overheads"},{"line_number":64,"context_line":"are very specific per hypervisor type, version and instance flavors, a proposed"},{"line_number":65,"context_line":"trade-off for having placement be able to correctly verify those overheads is"},{"line_number":66,"context_line":"to ask operators to update the current \u0027reserved\u0027 configuration options that"},{"line_number":67,"context_line":"relate to the hypervisor offset amount needed for running by capping it to the"},{"line_number":68,"context_line":"maximum amount per resource class they think it would require to run on the"},{"line_number":69,"context_line":"compute node."},{"line_number":70,"context_line":""},{"line_number":71,"context_line":"Move operations and reschedules (if unexpected failures happen when spawning"},{"line_number":72,"context_line":"the instance) will delete original allocations by having the conductor call"}],"source_content_type":"text/x-rst","patch_set":8,"id":"5ff73747_78a16fa9","line":69,"range":{"start_line":66,"start_character":3,"end_line":69,"end_character":13},"in_reply_to":"5ff73747_08d6a2f2","updated":"2017-04-25 21:25:23.000000000","message":"Yeah initially we weren\u0027t going to automate anything, just config options and leave it up to operators to do capacity planning with overhead in mind. They can err on the side of being conservative or aggressive depending on their priorities. Jay also mentioned being able to write some docs for guidance here.","commit_id":"32a4da557b4686983054f3b3497475f7ea0f46cd"},{"author":{"_account_id":1063,"name":"Ed Leafe","email":"ed@leafe.com","username":"ed-leafe"},"change_message_id":"1cb636e35c61085e8015f20114d70a213a54ee74","unresolved":false,"context_lines":[{"line_number":63,"context_line":"taken into account for claiming the resource in the RT, and given overheads"},{"line_number":64,"context_line":"are very specific per hypervisor type, version and instance flavors, a proposed"},{"line_number":65,"context_line":"trade-off for having placement be able to correctly verify those overheads is"},{"line_number":66,"context_line":"to ask operators to update the current \u0027reserved\u0027 configuration options that"},{"line_number":67,"context_line":"relate to the hypervisor offset amount needed for running by capping it to the"},{"line_number":68,"context_line":"maximum amount per resource class they think it would require to run on the"},{"line_number":69,"context_line":"compute node."},{"line_number":70,"context_line":""},{"line_number":71,"context_line":"Move operations and reschedules (if unexpected failures happen when spawning"},{"line_number":72,"context_line":"the instance) will delete original allocations by having the conductor call"}],"source_content_type":"text/x-rst","patch_set":8,"id":"5ff73747_08d6a2f2","line":69,"range":{"start_line":66,"start_character":3,"end_line":69,"end_character":13},"in_reply_to":"5ff73747_4d46fc23","updated":"2017-04-25 17:00:56.000000000","message":"My understanding is that reserved should be a one-time setting, not something continuously updated for a RP. So for a compute node, figure the upper end for # of instances, multiply that by the overhead per instance for that HV, and add that to the HV overhead to get your reserved amount. If it later turns out that we need to get things more super-fine-grained than that, we can add more complexity, but initially that would not be necessary.","commit_id":"32a4da557b4686983054f3b3497475f7ea0f46cd"},{"author":{"_account_id":4393,"name":"Dan Smith","email":"dms@danplanet.com","username":"danms"},"change_message_id":"b39adc64efcead671d56daa9236a8fa1147dcc1d","unresolved":false,"context_lines":[{"line_number":94,"context_line":""},{"line_number":95,"context_line":"One last argument could be about where to call Placement API for both creating"},{"line_number":96,"context_line":"allocations and deleting them, whether in the conductor or in the scheduler"},{"line_number":97,"context_line":"services, but we leave that for a review discussion during the implementation."},{"line_number":98,"context_line":"We possibly reserve ourselves to verify the performance between those two"},{"line_number":99,"context_line":"alternatives using some benchmark tooling."},{"line_number":100,"context_line":""}],"source_content_type":"text/x-rst","patch_set":8,"id":"5ff73747_6db2d896","line":97,"range":{"start_line":97,"start_character":10,"end_line":97,"end_character":78},"updated":"2017-04-25 16:25:24.000000000","message":"Cool, I still think conductor is the place to have it, but I\u0027m good with landing this and hashing that out there.","commit_id":"32a4da557b4686983054f3b3497475f7ea0f46cd"},{"author":{"_account_id":1063,"name":"Ed Leafe","email":"ed@leafe.com","username":"ed-leafe"},"change_message_id":"1cb636e35c61085e8015f20114d70a213a54ee74","unresolved":false,"context_lines":[{"line_number":94,"context_line":""},{"line_number":95,"context_line":"One last argument could be about where to call Placement API for both creating"},{"line_number":96,"context_line":"allocations and deleting them, whether in the conductor or in the scheduler"},{"line_number":97,"context_line":"services, but we leave that for a review discussion during the implementation."},{"line_number":98,"context_line":"We possibly reserve ourselves to verify the performance between those two"},{"line_number":99,"context_line":"alternatives using some benchmark tooling."},{"line_number":100,"context_line":""}],"source_content_type":"text/x-rst","patch_set":8,"id":"5ff73747_83b975ea","line":97,"range":{"start_line":97,"start_character":10,"end_line":97,"end_character":78},"in_reply_to":"5ff73747_6db2d896","updated":"2017-04-25 17:00:56.000000000","message":"It would be helpful to document the pros and cons of each here, so that we can look back and remember why we wanted/didn\u0027t want to follow a particular approach.","commit_id":"32a4da557b4686983054f3b3497475f7ea0f46cd"},{"author":{"_account_id":11564,"name":"Chris Dent","email":"cdent@anticdent.org","username":"chdent"},"change_message_id":"f08b51dd3a53eae6abfb402d28a0a83d2b997bad","unresolved":false,"context_lines":[{"line_number":94,"context_line":""},{"line_number":95,"context_line":"One last argument could be about where to call Placement API for both creating"},{"line_number":96,"context_line":"allocations and deleting them, whether in the conductor or in the scheduler"},{"line_number":97,"context_line":"services, but we leave that for a review discussion during the implementation."},{"line_number":98,"context_line":"We possibly reserve ourselves to verify the performance between those two"},{"line_number":99,"context_line":"alternatives using some benchmark tooling."},{"line_number":100,"context_line":""}],"source_content_type":"text/x-rst","patch_set":8,"id":"5ff73747_e3671851","line":97,"in_reply_to":"5ff73747_83b975ea","updated":"2017-04-25 17:31:27.000000000","message":"I also would prefer that we record some of that history, but on the hand it would be great to get on with things as most of the specs for this stuff have needed tuneups from the things we\u0027ve learned while making it happen.","commit_id":"32a4da557b4686983054f3b3497475f7ea0f46cd"},{"author":{"_account_id":11564,"name":"Chris Dent","email":"cdent@anticdent.org","username":"chdent"},"change_message_id":"f08b51dd3a53eae6abfb402d28a0a83d2b997bad","unresolved":false,"context_lines":[{"line_number":96,"context_line":"allocations and deleting them, whether in the conductor or in the scheduler"},{"line_number":97,"context_line":"services, but we leave that for a review discussion during the implementation."},{"line_number":98,"context_line":"We possibly reserve ourselves to verify the performance between those two"},{"line_number":99,"context_line":"alternatives using some benchmark tooling."},{"line_number":100,"context_line":""},{"line_number":101,"context_line":"Data model impact"},{"line_number":102,"context_line":"-----------------"}],"source_content_type":"text/x-rst","patch_set":8,"id":"5ff73747_435c8c7d","line":99,"updated":"2017-04-25 17:31:27.000000000","message":"I don\u0027t understand what this sentence means.","commit_id":"32a4da557b4686983054f3b3497475f7ea0f46cd"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"19be12c9f531e65b650a7210442d1d852d0076f0","unresolved":false,"context_lines":[{"line_number":96,"context_line":"allocations and deleting them, whether in the conductor or in the scheduler"},{"line_number":97,"context_line":"services, but we leave that for a review discussion during the implementation."},{"line_number":98,"context_line":"We possibly reserve ourselves to verify the performance between those two"},{"line_number":99,"context_line":"alternatives using some benchmark tooling."},{"line_number":100,"context_line":""},{"line_number":101,"context_line":"Data model impact"},{"line_number":102,"context_line":"-----------------"}],"source_content_type":"text/x-rst","patch_set":8,"id":"5ff73747_b8fdb73b","line":99,"in_reply_to":"5ff73747_435c8c7d","updated":"2017-04-25 21:25:23.000000000","message":"He\u0027s just staying that we mean to POC both claims (plus retries) in scheduler vs claims in conductor + retries, and see if there is a big difference in performance (time to schedule), like with say 1000 compute nodes and 50 concurrent build requests, running multiple times, with both approaches and see which performs better. So while the spec talks about doing the claims and retries in the scheduler, we want to POC doing it in conductor too and see which is better. Doing it in conductor could be better for other non-performance reasons too.","commit_id":"32a4da557b4686983054f3b3497475f7ea0f46cd"},{"author":{"_account_id":11564,"name":"Chris Dent","email":"cdent@anticdent.org","username":"chdent"},"change_message_id":"f08b51dd3a53eae6abfb402d28a0a83d2b997bad","unresolved":false,"context_lines":[{"line_number":170,"context_line":"  created, and if so, don\u0027t POST allocations to Placement service."},{"line_number":171,"context_line":"* Modify conductor to DELETE /allocations/\u003cinstance_id\u003e if this is a reschedule"},{"line_number":172,"context_line":"  or a move operation."},{"line_number":173,"context_line":"* Modify scheduler to POST allocations to Placement if all computes are new."},{"line_number":174,"context_line":""},{"line_number":175,"context_line":"Dependencies"},{"line_number":176,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"}],"source_content_type":"text/x-rst","patch_set":8,"id":"5ff73747_23d4e0bc","line":173,"updated":"2017-04-25 17:31:27.000000000","message":"I don\u0027t understand what this sentence means.","commit_id":"32a4da557b4686983054f3b3497475f7ea0f46cd"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"19be12c9f531e65b650a7210442d1d852d0076f0","unresolved":false,"context_lines":[{"line_number":170,"context_line":"  created, and if so, don\u0027t POST allocations to Placement service."},{"line_number":171,"context_line":"* Modify conductor to DELETE /allocations/\u003cinstance_id\u003e if this is a reschedule"},{"line_number":172,"context_line":"  or a move operation."},{"line_number":173,"context_line":"* Modify scheduler to POST allocations to Placement if all computes are new."},{"line_number":174,"context_line":""},{"line_number":175,"context_line":"Dependencies"},{"line_number":176,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"}],"source_content_type":"text/x-rst","patch_set":8,"id":"5ff73747_18702bdc","line":173,"in_reply_to":"5ff73747_23d4e0bc","updated":"2017-04-25 21:25:23.000000000","message":"This is the part where scheduler service does the \u0027claim\u0027 which is creating the allocations in placement once we\u0027ve picked a host. The \u0027all computes are new\u0027 part means we don\u0027t start posting allocations in the control layer (scheduler or conductor, whichever we go with) until all compute services are upgraded to Pike so they can handle this (this being the 4th bullet in this list).","commit_id":"32a4da557b4686983054f3b3497475f7ea0f46cd"}]}
