)]}'
{"specs/pike/approved/placement-project-user.rst":[{"author":{"_account_id":11564,"name":"Chris Dent","email":"cdent@anticdent.org","username":"chdent"},"change_message_id":"f8ac73554cf85c4fd6801f595bb6f4bb6736f716","unresolved":false,"context_lines":[{"line_number":105,"context_line":""},{"line_number":106,"context_line":"The GET method for /allocations will now accept new query strings in the URI"},{"line_number":107,"context_line":"called \u0027resources\u0027, \u0027project_id\u0027, and \u0027user_id\u0027 that will return a list of"},{"line_number":108,"context_line":"allocations matching the requested resources, project_id, and user_id."},{"line_number":109,"context_line":""},{"line_number":110,"context_line":"Example::"},{"line_number":111,"context_line":""}],"source_content_type":"text/x-rst","patch_set":1,"id":"dfeb2761_3ab61540","line":108,"updated":"2017-04-04 13:15:02.000000000","message":"Can you expand on why it is important/useful to filter the output based on the resources and what the impact of the filter will be? Will it cause allocations to not include a class of resource (even if set) or will it not include an allocation that includes an unspecified class, or something else?\n\nThe reason I ask is because at PUT time an allocation is a single unit which contains multiple classes of resources for multiple resource providers. To maintain the existing semantics of the GET representation it might be better to get all allocations for a project and/or user id and do resource class filtering on the client side. This is because we want an allocation to always be an allocation as it might be found anywhere.","commit_id":"f00e7859699520cd86bfc0b3452fb79673c2d052"},{"author":{"_account_id":4690,"name":"melanie witt","display_name":"melwitt","email":"melwittt@gmail.com","username":"melwitt"},"change_message_id":"374d725d39ab2b0446654308cf88cbc85edf2d32","unresolved":false,"context_lines":[{"line_number":105,"context_line":""},{"line_number":106,"context_line":"The GET method for /allocations will now accept new query strings in the URI"},{"line_number":107,"context_line":"called \u0027resources\u0027, \u0027project_id\u0027, and \u0027user_id\u0027 that will return a list of"},{"line_number":108,"context_line":"allocations matching the requested resources, project_id, and user_id."},{"line_number":109,"context_line":""},{"line_number":110,"context_line":"Example::"},{"line_number":111,"context_line":""}],"source_content_type":"text/x-rst","patch_set":1,"id":"dfeb2761_699cd069","line":108,"in_reply_to":"dfeb2761_3ab61540","updated":"2017-04-04 16:56:53.000000000","message":"Now that you mention it, I don\u0027t think it\u0027s important to filter the output based on resources. I had just been thinking if I\u0027m only interested in VCPU and MEMORY_MB, I would ask for only those. But I think it would be fine to get all of them and pick out the classes of interest from the result.","commit_id":"f00e7859699520cd86bfc0b3452fb79673c2d052"},{"author":{"_account_id":11564,"name":"Chris Dent","email":"cdent@anticdent.org","username":"chdent"},"change_message_id":"f8ac73554cf85c4fd6801f595bb6f4bb6736f716","unresolved":false,"context_lines":[{"line_number":163,"context_line":"API over querying all cells for instances owned by a project and iterating"},{"line_number":164,"context_line":"over them, summing the cores and ram values. Instead of N database queries"},{"line_number":165,"context_line":"for N cells, there will be two database queries by placement, one for the"},{"line_number":166,"context_line":"consumers associated with a project/user and one for the matching allocations."},{"line_number":167,"context_line":"Performance will also be improved in that cells being temporarily down will no"},{"line_number":168,"context_line":"longer have the potential for end users to exceed allowed quota limits."},{"line_number":169,"context_line":""}],"source_content_type":"text/x-rst","patch_set":1,"id":"dfeb2761_da0ba9d3","line":166,"updated":"2017-04-04 13:15:02.000000000","message":"Presumably just one with a join of some kind?","commit_id":"f00e7859699520cd86bfc0b3452fb79673c2d052"},{"author":{"_account_id":4690,"name":"melanie witt","display_name":"melwitt","email":"melwittt@gmail.com","username":"melwitt"},"change_message_id":"374d725d39ab2b0446654308cf88cbc85edf2d32","unresolved":false,"context_lines":[{"line_number":163,"context_line":"API over querying all cells for instances owned by a project and iterating"},{"line_number":164,"context_line":"over them, summing the cores and ram values. Instead of N database queries"},{"line_number":165,"context_line":"for N cells, there will be two database queries by placement, one for the"},{"line_number":166,"context_line":"consumers associated with a project/user and one for the matching allocations."},{"line_number":167,"context_line":"Performance will also be improved in that cells being temporarily down will no"},{"line_number":168,"context_line":"longer have the potential for end users to exceed allowed quota limits."},{"line_number":169,"context_line":""}],"source_content_type":"text/x-rst","patch_set":1,"id":"dfeb2761_49722c4b","line":166,"in_reply_to":"dfeb2761_da0ba9d3","updated":"2017-04-04 16:56:53.000000000","message":"Yes, I think you\u0027re right. It could be one joining on the consumer_id column.","commit_id":"f00e7859699520cd86bfc0b3452fb79673c2d052"},{"author":{"_account_id":11564,"name":"Chris Dent","email":"cdent@anticdent.org","username":"chdent"},"change_message_id":"f8ac73554cf85c4fd6801f595bb6f4bb6736f716","unresolved":false,"context_lines":[{"line_number":193,"context_line":"----------"},{"line_number":194,"context_line":""},{"line_number":195,"context_line":"* Create database migration that creates the `consumers` table"},{"line_number":196,"context_line":"* Create the new `nova.objects` models for `Consumer`"},{"line_number":197,"context_line":"* Add new query parameters for the placement allocations REST API to query"},{"line_number":198,"context_line":"  allocations by resource classes, project_id, and user_id"},{"line_number":199,"context_line":""}],"source_content_type":"text/x-rst","patch_set":1,"id":"dfeb2761_1a4fd134","line":196,"updated":"2017-04-04 13:15:02.000000000","message":"Do we need this object? The consumer table in the database provides a way to perform the necessary query and, at least for the time being, the only time we want to write to it is when we write an allocation, so we can do that during the AllocationList.set_all().","commit_id":"f00e7859699520cd86bfc0b3452fb79673c2d052"},{"author":{"_account_id":4690,"name":"melanie witt","display_name":"melwitt","email":"melwittt@gmail.com","username":"melwitt"},"change_message_id":"374d725d39ab2b0446654308cf88cbc85edf2d32","unresolved":false,"context_lines":[{"line_number":193,"context_line":"----------"},{"line_number":194,"context_line":""},{"line_number":195,"context_line":"* Create database migration that creates the `consumers` table"},{"line_number":196,"context_line":"* Create the new `nova.objects` models for `Consumer`"},{"line_number":197,"context_line":"* Add new query parameters for the placement allocations REST API to query"},{"line_number":198,"context_line":"  allocations by resource classes, project_id, and user_id"},{"line_number":199,"context_line":""}],"source_content_type":"text/x-rst","patch_set":1,"id":"dfeb2761_a9d2d8fe","line":196,"in_reply_to":"dfeb2761_1a4fd134","updated":"2017-04-04 16:56:53.000000000","message":"Yes, I think you\u0027re also right on this. I had been thinking accesses to the consumers table would have to go through a matching object, but we don\u0027t have a need for that yet.","commit_id":"f00e7859699520cd86bfc0b3452fb79673c2d052"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"5d16d16fd40f6ee472963d481e0853b7b6ad0c58","unresolved":false,"context_lines":[{"line_number":59,"context_line":""},{"line_number":60,"context_line":"    CREATE TABLE consumers ("},{"line_number":61,"context_line":"        id INT UNSIGNED NOT NULL AUTOINCREMENT PRIMARY KEY,"},{"line_number":62,"context_line":"        consumer_id VARCHAR(64) NOT NULL,"},{"line_number":63,"context_line":"        project_id VARCHAR(64) NOT NULL,"},{"line_number":64,"context_line":"        user_id VARCHAR(64) NOT NULL,"},{"line_number":65,"context_line":"        INDEX (consumer_id),"}],"source_content_type":"text/x-rst","patch_set":2,"id":"dfeb2761_65b3d04e","line":62,"updated":"2017-04-04 19:10:57.000000000","message":"Why wouldn\u0027t this just match the consumer_id column in the allocations table?\n\nhttps://github.com/openstack/nova/blob/master/nova/db/sqlalchemy/api_models.py#L351\n\nconsumer_id \u003d Column(String(36), nullable\u003dFalse)\n\nWhich is 36 because it\u0027s a uuid.","commit_id":"ea04dd10ea026f668584c489dee5aacf3151711e"},{"author":{"_account_id":4690,"name":"melanie witt","display_name":"melwitt","email":"melwittt@gmail.com","username":"melwitt"},"change_message_id":"2bdaf145454a0ac71256ea793275ada76e832c95","unresolved":false,"context_lines":[{"line_number":59,"context_line":""},{"line_number":60,"context_line":"    CREATE TABLE consumers ("},{"line_number":61,"context_line":"        id INT UNSIGNED NOT NULL AUTOINCREMENT PRIMARY KEY,"},{"line_number":62,"context_line":"        consumer_id VARCHAR(64) NOT NULL,"},{"line_number":63,"context_line":"        project_id VARCHAR(64) NOT NULL,"},{"line_number":64,"context_line":"        user_id VARCHAR(64) NOT NULL,"},{"line_number":65,"context_line":"        INDEX (consumer_id),"}],"source_content_type":"text/x-rst","patch_set":2,"id":"dfeb2761_6572b04b","line":62,"in_reply_to":"dfeb2761_65b3d04e","updated":"2017-04-04 19:25:36.000000000","message":"Oops, I had copied what I saw in the spec [1] that described the allocations table and didn\u0027t notice the difference in the model in the code. I intended to make it match.\n\n[1] http://specs.openstack.org/openstack/nova-specs/specs/newton/implemented/resource-providers.html","commit_id":"ea04dd10ea026f668584c489dee5aacf3151711e"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"5d16d16fd40f6ee472963d481e0853b7b6ad0c58","unresolved":false,"context_lines":[{"line_number":60,"context_line":"    CREATE TABLE consumers ("},{"line_number":61,"context_line":"        id INT UNSIGNED NOT NULL AUTOINCREMENT PRIMARY KEY,"},{"line_number":62,"context_line":"        consumer_id VARCHAR(64) NOT NULL,"},{"line_number":63,"context_line":"        project_id VARCHAR(64) NOT NULL,"},{"line_number":64,"context_line":"        user_id VARCHAR(64) NOT NULL,"},{"line_number":65,"context_line":"        INDEX (consumer_id),"},{"line_number":66,"context_line":"        INDEX (project_id, user_id)"},{"line_number":67,"context_line":"    );"}],"source_content_type":"text/x-rst","patch_set":2,"id":"dfeb2761_658cf07e","line":64,"range":{"start_line":63,"start_character":8,"end_line":64,"end_character":37},"updated":"2017-04-04 19:10:57.000000000","message":"Don\u0027t we need to model the same thing as in the project_user_quotas table?\n\nhttps://github.com/openstack/nova/blob/master/nova/db/sqlalchemy/api_models.py#L479-L480\n\nI don\u0027t know why those are 255 since they are normally just uuids, but I know that keystone doesn\u0027t enforce that they are just uuids.\n\nAlthough a user.id in keystone is 64 like you have here:\n\nhttps://github.com/openstack/keystone/blob/master/keystone/identity/backends/sql_model.py#L37","commit_id":"ea04dd10ea026f668584c489dee5aacf3151711e"},{"author":{"_account_id":4690,"name":"melanie witt","display_name":"melwitt","email":"melwittt@gmail.com","username":"melwitt"},"change_message_id":"2bdaf145454a0ac71256ea793275ada76e832c95","unresolved":false,"context_lines":[{"line_number":60,"context_line":"    CREATE TABLE consumers ("},{"line_number":61,"context_line":"        id INT UNSIGNED NOT NULL AUTOINCREMENT PRIMARY KEY,"},{"line_number":62,"context_line":"        consumer_id VARCHAR(64) NOT NULL,"},{"line_number":63,"context_line":"        project_id VARCHAR(64) NOT NULL,"},{"line_number":64,"context_line":"        user_id VARCHAR(64) NOT NULL,"},{"line_number":65,"context_line":"        INDEX (consumer_id),"},{"line_number":66,"context_line":"        INDEX (project_id, user_id)"},{"line_number":67,"context_line":"    );"}],"source_content_type":"text/x-rst","patch_set":2,"id":"dfeb2761_a5a1d896","line":64,"range":{"start_line":63,"start_character":8,"end_line":64,"end_character":37},"in_reply_to":"dfeb2761_658cf07e","updated":"2017-04-04 19:25:36.000000000","message":"Yeah, looks like all of the existing project_id and user_id we have are 255, so we should probably stay consistent with that here.","commit_id":"ea04dd10ea026f668584c489dee5aacf3151711e"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"5d16d16fd40f6ee472963d481e0853b7b6ad0c58","unresolved":false,"context_lines":[{"line_number":171,"context_line":"Other deployer impact"},{"line_number":172,"context_line":"---------------------"},{"line_number":173,"context_line":""},{"line_number":174,"context_line":"None."},{"line_number":175,"context_line":""},{"line_number":176,"context_line":"Developer impact"},{"line_number":177,"context_line":"----------------"}],"source_content_type":"text/x-rst","patch_set":2,"id":"dfeb2761_a549d831","line":174,"updated":"2017-04-04 19:10:57.000000000","message":"How do make up for existing allocations that don\u0027t have project/user id information in the consumers table?","commit_id":"ea04dd10ea026f668584c489dee5aacf3151711e"},{"author":{"_account_id":4690,"name":"melanie witt","display_name":"melwitt","email":"melwittt@gmail.com","username":"melwitt"},"change_message_id":"2bdaf145454a0ac71256ea793275ada76e832c95","unresolved":false,"context_lines":[{"line_number":171,"context_line":"Other deployer impact"},{"line_number":172,"context_line":"---------------------"},{"line_number":173,"context_line":""},{"line_number":174,"context_line":"None."},{"line_number":175,"context_line":""},{"line_number":176,"context_line":"Developer impact"},{"line_number":177,"context_line":"----------------"}],"source_content_type":"text/x-rst","patch_set":2,"id":"dfeb2761_a5f8b87a","line":174,"in_reply_to":"dfeb2761_a549d831","updated":"2017-04-04 19:25:36.000000000","message":"Hm, maybe we could do an online data migration pulls all allocations and for each allocation, look up the instance via consumer_id, and then write the consumer using the project_id and user_id from the instance.","commit_id":"ea04dd10ea026f668584c489dee5aacf3151711e"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"f3103faed77f8fa1448a04b59dda704b05c9e0b5","unresolved":false,"context_lines":[{"line_number":171,"context_line":"Other deployer impact"},{"line_number":172,"context_line":"---------------------"},{"line_number":173,"context_line":""},{"line_number":174,"context_line":"None."},{"line_number":175,"context_line":""},{"line_number":176,"context_line":"Developer impact"},{"line_number":177,"context_line":"----------------"}],"source_content_type":"text/x-rst","patch_set":2,"id":"dfeb2761_e8ba156c","line":174,"in_reply_to":"dfeb2761_a5f8b87a","updated":"2017-04-04 20:04:11.000000000","message":"Yeah I was thinking similar. You should call that out.","commit_id":"ea04dd10ea026f668584c489dee5aacf3151711e"},{"author":{"_account_id":4690,"name":"melanie witt","display_name":"melwitt","email":"melwittt@gmail.com","username":"melwitt"},"change_message_id":"25e826779c04bad3dc19e77f9e7a3f1e7a2a585e","unresolved":false,"context_lines":[{"line_number":171,"context_line":"Other deployer impact"},{"line_number":172,"context_line":"---------------------"},{"line_number":173,"context_line":""},{"line_number":174,"context_line":"None."},{"line_number":175,"context_line":""},{"line_number":176,"context_line":"Developer impact"},{"line_number":177,"context_line":"----------------"}],"source_content_type":"text/x-rst","patch_set":2,"id":"dfeb2761_b3148245","line":174,"in_reply_to":"dfeb2761_e8ba156c","updated":"2017-04-04 20:09:42.000000000","message":"Yup, thanks for catching that.","commit_id":"ea04dd10ea026f668584c489dee5aacf3151711e"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"5d16d16fd40f6ee472963d481e0853b7b6ad0c58","unresolved":false,"context_lines":[{"line_number":190,"context_line":"Other contributors:"},{"line_number":191,"context_line":"  None"},{"line_number":192,"context_line":""},{"line_number":193,"context_line":"Work Items"},{"line_number":194,"context_line":"----------"},{"line_number":195,"context_line":""},{"line_number":196,"context_line":"* Create database migration that creates the `consumers` table"}],"source_content_type":"text/x-rst","patch_set":2,"id":"dfeb2761_e55c80ee","line":193,"updated":"2017-04-04 19:10:57.000000000","message":"We also need to fix this bug first I think:\n\nhttps://bugs.launchpad.net/nova/+bug/1679750","commit_id":"ea04dd10ea026f668584c489dee5aacf3151711e"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"f3103faed77f8fa1448a04b59dda704b05c9e0b5","unresolved":false,"context_lines":[{"line_number":190,"context_line":"Other contributors:"},{"line_number":191,"context_line":"  None"},{"line_number":192,"context_line":""},{"line_number":193,"context_line":"Work Items"},{"line_number":194,"context_line":"----------"},{"line_number":195,"context_line":""},{"line_number":196,"context_line":"* Create database migration that creates the `consumers` table"}],"source_content_type":"text/x-rst","patch_set":2,"id":"dfeb2761_4865811d","line":193,"in_reply_to":"dfeb2761_0581a4c3","updated":"2017-04-04 20:04:11.000000000","message":"Probably should yeah. Basically I think we need to fix the existing problem(s) before adding new code that depends on this.","commit_id":"ea04dd10ea026f668584c489dee5aacf3151711e"},{"author":{"_account_id":4690,"name":"melanie witt","display_name":"melwitt","email":"melwittt@gmail.com","username":"melwitt"},"change_message_id":"25e826779c04bad3dc19e77f9e7a3f1e7a2a585e","unresolved":false,"context_lines":[{"line_number":190,"context_line":"Other contributors:"},{"line_number":191,"context_line":"  None"},{"line_number":192,"context_line":""},{"line_number":193,"context_line":"Work Items"},{"line_number":194,"context_line":"----------"},{"line_number":195,"context_line":""},{"line_number":196,"context_line":"* Create database migration that creates the `consumers` table"}],"source_content_type":"text/x-rst","patch_set":2,"id":"dfeb2761_132fae8e","line":193,"in_reply_to":"dfeb2761_4865811d","updated":"2017-04-04 20:09:42.000000000","message":"Roger that.","commit_id":"ea04dd10ea026f668584c489dee5aacf3151711e"},{"author":{"_account_id":4690,"name":"melanie witt","display_name":"melwitt","email":"melwittt@gmail.com","username":"melwitt"},"change_message_id":"2bdaf145454a0ac71256ea793275ada76e832c95","unresolved":false,"context_lines":[{"line_number":190,"context_line":"Other contributors:"},{"line_number":191,"context_line":"  None"},{"line_number":192,"context_line":""},{"line_number":193,"context_line":"Work Items"},{"line_number":194,"context_line":"----------"},{"line_number":195,"context_line":""},{"line_number":196,"context_line":"* Create database migration that creates the `consumers` table"}],"source_content_type":"text/x-rst","patch_set":2,"id":"dfeb2761_0581a4c3","line":193,"in_reply_to":"dfeb2761_e55c80ee","updated":"2017-04-04 19:25:36.000000000","message":"Okay, are you saying I can write that as the first Work Item in this section?","commit_id":"ea04dd10ea026f668584c489dee5aacf3151711e"},{"author":{"_account_id":7166,"name":"Sylvain Bauza","email":"sbauza@redhat.com","username":"sbauza"},"change_message_id":"5c55bd8197f850bc937ba26d772a1efc29a7c80d","unresolved":false,"context_lines":[{"line_number":42,"context_line":"become omitted from the counted usage for the project/user, allowing them"},{"line_number":43,"context_line":"to allocate additional resources in other cells in the meantime. If and when"},{"line_number":44,"context_line":"the down cell returns, the project/user could then have allocated more"},{"line_number":45,"context_line":"resources than their allowed quota limit."},{"line_number":46,"context_line":""},{"line_number":47,"context_line":"Use Cases"},{"line_number":48,"context_line":"---------"}],"source_content_type":"text/x-rst","patch_set":3,"id":"bff0334d_601eb271","line":45,"updated":"2017-04-05 13:14:11.000000000","message":"It\u0027s generally called \"Split-brain\" but meh.","commit_id":"105c1e9d1bc4c8ba84eaf4115a1a6f5041dda5e7"},{"author":{"_account_id":7,"name":"Jay Pipes","email":"jaypipes@gmail.com","username":"jaypipes"},"change_message_id":"ee045ab4392eaaf3dc95c77e5928cc90b9bb3e34","unresolved":false,"context_lines":[{"line_number":42,"context_line":"become omitted from the counted usage for the project/user, allowing them"},{"line_number":43,"context_line":"to allocate additional resources in other cells in the meantime. If and when"},{"line_number":44,"context_line":"the down cell returns, the project/user could then have allocated more"},{"line_number":45,"context_line":"resources than their allowed quota limit."},{"line_number":46,"context_line":""},{"line_number":47,"context_line":"Use Cases"},{"line_number":48,"context_line":"---------"}],"source_content_type":"text/x-rst","patch_set":3,"id":"bff0334d_865f8ee4","line":45,"updated":"2017-04-05 13:41:51.000000000","message":"No, split-brain refers to when two nodes in a cluster consider themselves the leader.","commit_id":"105c1e9d1bc4c8ba84eaf4115a1a6f5041dda5e7"},{"author":{"_account_id":7166,"name":"Sylvain Bauza","email":"sbauza@redhat.com","username":"sbauza"},"change_message_id":"7acf4f9dd23e22bddfdd4058530b7c4803f639db","unresolved":false,"context_lines":[{"line_number":42,"context_line":"become omitted from the counted usage for the project/user, allowing them"},{"line_number":43,"context_line":"to allocate additional resources in other cells in the meantime. If and when"},{"line_number":44,"context_line":"the down cell returns, the project/user could then have allocated more"},{"line_number":45,"context_line":"resources than their allowed quota limit."},{"line_number":46,"context_line":""},{"line_number":47,"context_line":"Use Cases"},{"line_number":48,"context_line":"---------"}],"source_content_type":"text/x-rst","patch_set":3,"id":"bff0334d_660a4a8b","line":45,"in_reply_to":"bff0334d_865f8ee4","updated":"2017-04-05 13:48:42.000000000","message":"That\u0027s a typical consequence of a split-brain problem, but the term definition is rather inconsistencies between two data sets. Anyway, I don\u0027t want to over-discuss the semantics...","commit_id":"105c1e9d1bc4c8ba84eaf4115a1a6f5041dda5e7"},{"author":{"_account_id":14070,"name":"Eric Fried","email":"openstack@fried.cc","username":"efried"},"change_message_id":"6c2691adefee557fa5f61094d03d6f3794245165","unresolved":false,"context_lines":[{"line_number":54,"context_line":"Proposed change"},{"line_number":55,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"},{"line_number":56,"context_line":""},{"line_number":57,"context_line":"Add a `consumers` table for placement that stores project/user associations"},{"line_number":58,"context_line":"for consumers with fields: consumer_id, project_id, user_id::"},{"line_number":59,"context_line":""},{"line_number":60,"context_line":"    CREATE TABLE consumers ("}],"source_content_type":"text/x-rst","patch_set":3,"id":"dfeb2761_943a84c9","line":57,"range":{"start_line":57,"start_character":6,"end_line":57,"end_character":7},"updated":"2017-04-04 22:21:52.000000000","message":"Nit throughout: single-backquoted strings (otherwise unadorned with underscores) are kinda environment-dependent in RST.  E.g. rst2html.py will use \u003ccite\u003e.  These references might play better with double-backquotes, which renders with \u003ctt\u003e and shows up as monospace.","commit_id":"105c1e9d1bc4c8ba84eaf4115a1a6f5041dda5e7"},{"author":{"_account_id":14070,"name":"Eric Fried","email":"openstack@fried.cc","username":"efried"},"change_message_id":"6c2691adefee557fa5f61094d03d6f3794245165","unresolved":false,"context_lines":[{"line_number":60,"context_line":"    CREATE TABLE consumers ("},{"line_number":61,"context_line":"        id INT UNSIGNED NOT NULL AUTOINCREMENT PRIMARY KEY,"},{"line_number":62,"context_line":"        consumer_id VARCHAR(36) NOT NULL,"},{"line_number":63,"context_line":"        project_id VARCHAR(255) NOT NULL,"},{"line_number":64,"context_line":"        user_id VARCHAR(255) NOT NULL,"},{"line_number":65,"context_line":"        INDEX (consumer_id),"},{"line_number":66,"context_line":"        INDEX (project_id, user_id)"},{"line_number":67,"context_line":"    );"}],"source_content_type":"text/x-rst","patch_set":3,"id":"dfeb2761_94a16497","line":64,"range":{"start_line":63,"start_character":0,"end_line":64,"end_character":38},"updated":"2017-04-04 22:21:52.000000000","message":"I\u0027m not familiar with the database design here in general, but these are foreign keys into some other table(s), yes?  If the refs have their own short ID (INT UNSIGNED PRIMARY KEY kind of thing), the joins will be more efficient if these fields reference those rather than the VARCHARs (which I presume are UUIDs).","commit_id":"105c1e9d1bc4c8ba84eaf4115a1a6f5041dda5e7"},{"author":{"_account_id":11564,"name":"Chris Dent","email":"cdent@anticdent.org","username":"chdent"},"change_message_id":"169115290f15d0584c1fc2f030012b6df84a307b","unresolved":false,"context_lines":[{"line_number":61,"context_line":"        id INT UNSIGNED NOT NULL AUTOINCREMENT PRIMARY KEY,"},{"line_number":62,"context_line":"        consumer_id VARCHAR(36) NOT NULL,"},{"line_number":63,"context_line":"        project_id VARCHAR(255) NOT NULL,"},{"line_number":64,"context_line":"        user_id VARCHAR(255) NOT NULL,"},{"line_number":65,"context_line":"        INDEX (consumer_id),"},{"line_number":66,"context_line":"        INDEX (project_id, user_id)"},{"line_number":67,"context_line":"    );"}],"source_content_type":"text/x-rst","patch_set":3,"id":"dfeb2761_cb5e2643","line":64,"in_reply_to":"dfeb2761_94a16497","updated":"2017-04-05 00:11:04.000000000","message":"They are not foreign keys, at least not in tables that placement will have access to as it evolves into using its own database. From the point of view of placement all three of consumer_id, project_id, and user_id are opaque strings. This table exists to be able to filter the allocations table (which has a consumer_id column) by either or both of project or user _id.","commit_id":"105c1e9d1bc4c8ba84eaf4115a1a6f5041dda5e7"},{"author":{"_account_id":14070,"name":"Eric Fried","email":"openstack@fried.cc","username":"efried"},"change_message_id":"07e1502f725c7c0928ae412a9777ea9f1c30292d","unresolved":false,"context_lines":[{"line_number":61,"context_line":"        id INT UNSIGNED NOT NULL AUTOINCREMENT PRIMARY KEY,"},{"line_number":62,"context_line":"        consumer_id VARCHAR(36) NOT NULL,"},{"line_number":63,"context_line":"        project_id VARCHAR(255) NOT NULL,"},{"line_number":64,"context_line":"        user_id VARCHAR(255) NOT NULL,"},{"line_number":65,"context_line":"        INDEX (consumer_id),"},{"line_number":66,"context_line":"        INDEX (project_id, user_id)"},{"line_number":67,"context_line":"    );"}],"source_content_type":"text/x-rst","patch_set":3,"id":"dfeb2761_97f16198","line":64,"in_reply_to":"dfeb2761_cb5e2643","updated":"2017-04-05 03:05:59.000000000","message":"Roger that, thanks for the education.","commit_id":"105c1e9d1bc4c8ba84eaf4115a1a6f5041dda5e7"},{"author":{"_account_id":7,"name":"Jay Pipes","email":"jaypipes@gmail.com","username":"jaypipes"},"change_message_id":"ee045ab4392eaaf3dc95c77e5928cc90b9bb3e34","unresolved":false,"context_lines":[{"line_number":63,"context_line":"        project_id VARCHAR(255) NOT NULL,"},{"line_number":64,"context_line":"        user_id VARCHAR(255) NOT NULL,"},{"line_number":65,"context_line":"        INDEX (consumer_id),"},{"line_number":66,"context_line":"        INDEX (project_id, user_id)"},{"line_number":67,"context_line":"    );"},{"line_number":68,"context_line":""},{"line_number":69,"context_line":"The records should have the same lifetime as `allocations` records."}],"source_content_type":"text/x-rst","patch_set":3,"id":"bff0334d_66648aba","line":66,"updated":"2017-04-05 13:41:51.000000000","message":"To get full index utilization, it would be best to have the following indexes:\n\n INDEX (project_id, consumer_id)\n INDEX (project_id, user_id, consumer_id)\n\nand get rid of:\n\n INDEX (project_id, user_id)","commit_id":"105c1e9d1bc4c8ba84eaf4115a1a6f5041dda5e7"},{"author":{"_account_id":7,"name":"Jay Pipes","email":"jaypipes@gmail.com","username":"jaypipes"},"change_message_id":"ee045ab4392eaaf3dc95c77e5928cc90b9bb3e34","unresolved":false,"context_lines":[{"line_number":64,"context_line":"        user_id VARCHAR(255) NOT NULL,"},{"line_number":65,"context_line":"        INDEX (consumer_id),"},{"line_number":66,"context_line":"        INDEX (project_id, user_id)"},{"line_number":67,"context_line":"    );"},{"line_number":68,"context_line":""},{"line_number":69,"context_line":"The records should have the same lifetime as `allocations` records."},{"line_number":70,"context_line":""}],"source_content_type":"text/x-rst","patch_set":3,"id":"bff0334d_465986fc","line":67,"updated":"2017-04-05 13:41:51.000000000","message":"Hmm, we probably should have used an integer ID for consumer_id in the allocations table. Hindsight is 20/20 of course... perhaps it\u0027s worth cleaning up in the future so that we store the thin integer ID in allocations.consumer_id and rename consumers.consumer_id to consumers.consumer_uuid (or just consumers.uuid).","commit_id":"105c1e9d1bc4c8ba84eaf4115a1a6f5041dda5e7"},{"author":{"_account_id":7,"name":"Jay Pipes","email":"jaypipes@gmail.com","username":"jaypipes"},"change_message_id":"ee045ab4392eaaf3dc95c77e5928cc90b9bb3e34","unresolved":false,"context_lines":[{"line_number":75,"context_line":""},{"line_number":76,"context_line":"In placement, when a query is received it will look up the consumer_ids and"},{"line_number":77,"context_line":"matching allocations by querying the `consumers` table joined with the"},{"line_number":78,"context_line":"`allocations` table on consumer_id and return the allocations."},{"line_number":79,"context_line":""},{"line_number":80,"context_line":"After the allocations are returned from placement, the quota counting code"},{"line_number":81,"context_line":"can count cores and ram from the allocations."}],"source_content_type":"text/x-rst","patch_set":3,"id":"bff0334d_265e82e6","line":78,"updated":"2017-04-05 13:41:51.000000000","message":"See above note about having consumer_id in the (project_id, consumer_id) and (project_id, user_id, consumer_id) indexes. By doing that, you will allow the index to be used for *both* the filtering (on project_id and user_id) as well as to satisfy the join projection (on consumer_id). This means all queries will likely be served entirely out of memory and no disk reads will be necessary.","commit_id":"105c1e9d1bc4c8ba84eaf4115a1a6f5041dda5e7"},{"author":{"_account_id":7,"name":"Jay Pipes","email":"jaypipes@gmail.com","username":"jaypipes"},"change_message_id":"ee045ab4392eaaf3dc95c77e5928cc90b9bb3e34","unresolved":false,"context_lines":[{"line_number":78,"context_line":"`allocations` table on consumer_id and return the allocations."},{"line_number":79,"context_line":""},{"line_number":80,"context_line":"After the allocations are returned from placement, the quota counting code"},{"line_number":81,"context_line":"can count cores and ram from the allocations."},{"line_number":82,"context_line":""},{"line_number":83,"context_line":"Alternatives"},{"line_number":84,"context_line":"------------"}],"source_content_type":"text/x-rst","patch_set":3,"id":"bff0334d_06737e82","line":81,"updated":"2017-04-05 13:41:51.000000000","message":"Instead of the quota-counting code querying GET /allocations, it should simply query GET /usages (passing the same project_id and user_id parameters) and the placement API will do the aggregation by resource class for it.","commit_id":"105c1e9d1bc4c8ba84eaf4115a1a6f5041dda5e7"},{"author":{"_account_id":7,"name":"Jay Pipes","email":"jaypipes@gmail.com","username":"jaypipes"},"change_message_id":"ee045ab4392eaaf3dc95c77e5928cc90b9bb3e34","unresolved":false,"context_lines":[{"line_number":87,"context_line":"project_id and user_id columns to the `allocations` table. It would be more"},{"line_number":88,"context_line":"direct than creating a `consumers` table with the associations but it wouldn\u0027t"},{"line_number":89,"context_line":"be as generic. A `consumers` table could potentially be used for queries other"},{"line_number":90,"context_line":"than just allocations."},{"line_number":91,"context_line":""},{"line_number":92,"context_line":"Data model impact"},{"line_number":93,"context_line":"-----------------"}],"source_content_type":"text/x-rst","patch_set":3,"id":"bff0334d_e66f7a90","line":90,"updated":"2017-04-05 13:41:51.000000000","message":"++","commit_id":"105c1e9d1bc4c8ba84eaf4115a1a6f5041dda5e7"},{"author":{"_account_id":7166,"name":"Sylvain Bauza","email":"sbauza@redhat.com","username":"sbauza"},"change_message_id":"5c55bd8197f850bc937ba26d772a1efc29a7c80d","unresolved":false,"context_lines":[{"line_number":87,"context_line":"project_id and user_id columns to the `allocations` table. It would be more"},{"line_number":88,"context_line":"direct than creating a `consumers` table with the associations but it wouldn\u0027t"},{"line_number":89,"context_line":"be as generic. A `consumers` table could potentially be used for queries other"},{"line_number":90,"context_line":"than just allocations."},{"line_number":91,"context_line":""},{"line_number":92,"context_line":"Data model impact"},{"line_number":93,"context_line":"-----------------"}],"source_content_type":"text/x-rst","patch_set":3,"id":"bff0334d_436f38c5","line":90,"updated":"2017-04-05 13:14:11.000000000","message":"Yup, good with me.","commit_id":"105c1e9d1bc4c8ba84eaf4115a1a6f5041dda5e7"},{"author":{"_account_id":14070,"name":"Eric Fried","email":"openstack@fried.cc","username":"efried"},"change_message_id":"6c2691adefee557fa5f61094d03d6f3794245165","unresolved":false,"context_lines":[{"line_number":94,"context_line":""},{"line_number":95,"context_line":"The following data model changes will be needed:"},{"line_number":96,"context_line":""},{"line_number":97,"context_line":"* New models for: `Consumer`"},{"line_number":98,"context_line":""},{"line_number":99,"context_line":"* New database table for `Consumer`"},{"line_number":100,"context_line":""}],"source_content_type":"text/x-rst","patch_set":3,"id":"dfeb2761_344f386c","line":97,"range":{"start_line":97,"start_character":19,"end_line":97,"end_character":27},"updated":"2017-04-04 22:21:52.000000000","message":"consumers","commit_id":"105c1e9d1bc4c8ba84eaf4115a1a6f5041dda5e7"},{"author":{"_account_id":14070,"name":"Eric Fried","email":"openstack@fried.cc","username":"efried"},"change_message_id":"6c2691adefee557fa5f61094d03d6f3794245165","unresolved":false,"context_lines":[{"line_number":96,"context_line":""},{"line_number":97,"context_line":"* New models for: `Consumer`"},{"line_number":98,"context_line":""},{"line_number":99,"context_line":"* New database table for `Consumer`"},{"line_number":100,"context_line":""},{"line_number":101,"context_line":"* Database migration will be needed to add the `consumers` table to the schema."},{"line_number":102,"context_line":""}],"source_content_type":"text/x-rst","patch_set":3,"id":"dfeb2761_145434fe","line":99,"range":{"start_line":99,"start_character":26,"end_line":99,"end_character":34},"updated":"2017-04-04 22:21:52.000000000","message":"consumers","commit_id":"105c1e9d1bc4c8ba84eaf4115a1a6f5041dda5e7"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"89dd1ff5fb21cee144882ae34b2d1012bdbb91c7","unresolved":false,"context_lines":[{"line_number":100,"context_line":""},{"line_number":101,"context_line":"* Database migration will be needed to add the `consumers` table to the schema."},{"line_number":102,"context_line":""},{"line_number":103,"context_line":"REST API impact"},{"line_number":104,"context_line":"---------------"},{"line_number":105,"context_line":""},{"line_number":106,"context_line":"The GET method for /allocations will now accept new query strings in the URI"}],"source_content_type":"text/x-rst","patch_set":3,"id":"dfeb2761_b46b48b5","line":103,"updated":"2017-04-04 22:02:48.000000000","message":"We should probably mention this is going to require a microveresion bump for the new query strings. Just FYI in case you respin.","commit_id":"105c1e9d1bc4c8ba84eaf4115a1a6f5041dda5e7"},{"author":{"_account_id":782,"name":"John Garbutt","email":"john@johngarbutt.com","username":"johngarbutt"},"change_message_id":"e0d155d56bec5398a1341148807a24160ca366e0","unresolved":false,"context_lines":[{"line_number":100,"context_line":""},{"line_number":101,"context_line":"* Database migration will be needed to add the `consumers` table to the schema."},{"line_number":102,"context_line":""},{"line_number":103,"context_line":"REST API impact"},{"line_number":104,"context_line":"---------------"},{"line_number":105,"context_line":""},{"line_number":106,"context_line":"The GET method for /allocations will now accept new query strings in the URI"}],"source_content_type":"text/x-rst","patch_set":3,"id":"dfeb2761_05a4e9a4","line":103,"in_reply_to":"dfeb2761_74d8201d","updated":"2017-04-05 11:19:32.000000000","message":"+1 for including a microversion","commit_id":"105c1e9d1bc4c8ba84eaf4115a1a6f5041dda5e7"},{"author":{"_account_id":4690,"name":"melanie witt","display_name":"melwitt","email":"melwittt@gmail.com","username":"melwitt"},"change_message_id":"975671504c6b9caf5d8685522198117ea1eba7ef","unresolved":false,"context_lines":[{"line_number":100,"context_line":""},{"line_number":101,"context_line":"* Database migration will be needed to add the `consumers` table to the schema."},{"line_number":102,"context_line":""},{"line_number":103,"context_line":"REST API impact"},{"line_number":104,"context_line":"---------------"},{"line_number":105,"context_line":""},{"line_number":106,"context_line":"The GET method for /allocations will now accept new query strings in the URI"}],"source_content_type":"text/x-rst","patch_set":3,"id":"dfeb2761_74d8201d","line":103,"in_reply_to":"dfeb2761_b46b48b5","updated":"2017-04-04 22:21:53.000000000","message":"Right, thanks. Will update if respin or another reviewer asks to take care of it now.","commit_id":"105c1e9d1bc4c8ba84eaf4115a1a6f5041dda5e7"},{"author":{"_account_id":7,"name":"Jay Pipes","email":"jaypipes@gmail.com","username":"jaypipes"},"change_message_id":"ee045ab4392eaaf3dc95c77e5928cc90b9bb3e34","unresolved":false,"context_lines":[{"line_number":103,"context_line":"REST API impact"},{"line_number":104,"context_line":"---------------"},{"line_number":105,"context_line":""},{"line_number":106,"context_line":"The GET method for /allocations will now accept new query strings in the URI"},{"line_number":107,"context_line":"called \u0027project_id\u0027 and \u0027user_id\u0027 that will return a list of allocations"},{"line_number":108,"context_line":"matching the project_id and user_id."},{"line_number":109,"context_line":""}],"source_content_type":"text/x-rst","patch_set":3,"id":"bff0334d_c66c769e","line":106,"updated":"2017-04-05 13:41:51.000000000","message":"and GET /usages","commit_id":"105c1e9d1bc4c8ba84eaf4115a1a6f5041dda5e7"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"b39cf6f2a608ba455fe4ef22734e5f5ed16e53f7","unresolved":false,"context_lines":[{"line_number":103,"context_line":"REST API impact"},{"line_number":104,"context_line":"---------------"},{"line_number":105,"context_line":""},{"line_number":106,"context_line":"The GET method for /allocations will now accept new query strings in the URI"},{"line_number":107,"context_line":"called \u0027project_id\u0027 and \u0027user_id\u0027 that will return a list of allocations"},{"line_number":108,"context_line":"matching the project_id and user_id."},{"line_number":109,"context_line":""}],"source_content_type":"text/x-rst","patch_set":3,"id":"bff0334d_c906c921","line":106,"in_reply_to":"bff0334d_c66c769e","updated":"2017-04-05 14:15:17.000000000","message":"Are you saying we\u0027ll do this through both GET /allocations *and* GET /usages or just GET /usages (which is a new handler)?","commit_id":"105c1e9d1bc4c8ba84eaf4115a1a6f5041dda5e7"},{"author":{"_account_id":11564,"name":"Chris Dent","email":"cdent@anticdent.org","username":"chdent"},"change_message_id":"dfbe2a3474cfc15e9438c5b89d511aac02640d33","unresolved":false,"context_lines":[{"line_number":105,"context_line":""},{"line_number":106,"context_line":"The GET method for /allocations will now accept new query strings in the URI"},{"line_number":107,"context_line":"called \u0027project_id\u0027 and \u0027user_id\u0027 that will return a list of allocations"},{"line_number":108,"context_line":"matching the project_id and user_id."},{"line_number":109,"context_line":""},{"line_number":110,"context_line":"Example::"},{"line_number":111,"context_line":""}],"source_content_type":"text/x-rst","patch_set":3,"id":"dfeb2761_a5f77d59","line":108,"updated":"2017-04-05 11:27:54.000000000","message":"Might be worth mentioning, if there\u0027s another version, that the query params are anded, so if both are used, it could result in zero results.\n\nAre procject_id and user_id well-structured enough that we can do schema validation on them?","commit_id":"105c1e9d1bc4c8ba84eaf4115a1a6f5041dda5e7"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"b39cf6f2a608ba455fe4ef22734e5f5ed16e53f7","unresolved":false,"context_lines":[{"line_number":105,"context_line":""},{"line_number":106,"context_line":"The GET method for /allocations will now accept new query strings in the URI"},{"line_number":107,"context_line":"called \u0027project_id\u0027 and \u0027user_id\u0027 that will return a list of allocations"},{"line_number":108,"context_line":"matching the project_id and user_id."},{"line_number":109,"context_line":""},{"line_number":110,"context_line":"Example::"},{"line_number":111,"context_line":""}],"source_content_type":"text/x-rst","patch_set":3,"id":"bff0334d_e9206db3","line":108,"in_reply_to":"dfeb2761_a5f77d59","updated":"2017-04-05 14:15:17.000000000","message":"We\u0027d re-use this:\n\nhttps://github.com/openstack/nova/blob/master/nova/api/validation/parameter_types.py#L273","commit_id":"105c1e9d1bc4c8ba84eaf4115a1a6f5041dda5e7"},{"author":{"_account_id":7,"name":"Jay Pipes","email":"jaypipes@gmail.com","username":"jaypipes"},"change_message_id":"ee045ab4392eaaf3dc95c77e5928cc90b9bb3e34","unresolved":false,"context_lines":[{"line_number":163,"context_line":"API over querying all cells for instances owned by a project and iterating"},{"line_number":164,"context_line":"over them, summing the cores and ram values. Instead of N database queries"},{"line_number":165,"context_line":"for N cells, there will be one database query by placement of consumers"},{"line_number":166,"context_line":"associated with a project/user joined on allocations to get the matching"},{"line_number":167,"context_line":"allocations. Performance will also be improved in that cells being temporarily"},{"line_number":168,"context_line":"down will no longer have the potential for end users to exceed allowed quota"},{"line_number":169,"context_line":"limits."}],"source_content_type":"text/x-rst","patch_set":3,"id":"bff0334d_a669728c","line":166,"updated":"2017-04-05 13:41:51.000000000","message":"Performance will be even better if we did the GET /usages with a project_id and user_id filter. :)","commit_id":"105c1e9d1bc4c8ba84eaf4115a1a6f5041dda5e7"},{"author":{"_account_id":11564,"name":"Chris Dent","email":"cdent@anticdent.org","username":"chdent"},"change_message_id":"dfbe2a3474cfc15e9438c5b89d511aac02640d33","unresolved":false,"context_lines":[{"line_number":200,"context_line":"* Create database migration that creates the `consumers` table"},{"line_number":201,"context_line":"* Update AllocationList object to read/write the `consumers` table"},{"line_number":202,"context_line":"* Add new query parameters for the placement allocations REST API to query"},{"line_number":203,"context_line":"  allocations by project_id and user_id"},{"line_number":204,"context_line":"* Add online data migration that pulls all `allocations` records and for each"},{"line_number":205,"context_line":"  allocation, look up the instance via consumer_id and write its consumer to"},{"line_number":206,"context_line":"  the `consumers` table using the project_id and user_id from the instance"}],"source_content_type":"text/x-rst","patch_set":3,"id":"dfeb2761_4538b1ec","line":203,"updated":"2017-04-05 11:27:54.000000000","message":"* update placement-api-ref (which hopefully will have caught up by the time implementation of this is happening) for new params","commit_id":"105c1e9d1bc4c8ba84eaf4115a1a6f5041dda5e7"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"b39cf6f2a608ba455fe4ef22734e5f5ed16e53f7","unresolved":false,"context_lines":[{"line_number":200,"context_line":"* Create database migration that creates the `consumers` table"},{"line_number":201,"context_line":"* Update AllocationList object to read/write the `consumers` table"},{"line_number":202,"context_line":"* Add new query parameters for the placement allocations REST API to query"},{"line_number":203,"context_line":"  allocations by project_id and user_id"},{"line_number":204,"context_line":"* Add online data migration that pulls all `allocations` records and for each"},{"line_number":205,"context_line":"  allocation, look up the instance via consumer_id and write its consumer to"},{"line_number":206,"context_line":"  the `consumers` table using the project_id and user_id from the instance"}],"source_content_type":"text/x-rst","patch_set":3,"id":"bff0334d_14128459","line":203,"in_reply_to":"dfeb2761_4538b1ec","updated":"2017-04-05 14:15:17.000000000","message":"That could go in the doc impact section below.","commit_id":"105c1e9d1bc4c8ba84eaf4115a1a6f5041dda5e7"}]}