)]}'
{"specs/ocata/approved/cells-count-resources-to-check-quota-in-api.rst":[{"author":{"_account_id":4393,"name":"Dan Smith","email":"dms@danplanet.com","username":"danms"},"change_message_id":"f84a058e0e1a7e131a7336b2517a534879f5afc0","unresolved":false,"context_lines":[{"line_number":101,"context_line":""},{"line_number":102,"context_line":"With the resource counting approach, it will be possible for a project to"},{"line_number":103,"context_line":"consume more resources than they have quota if they are racing near the end"},{"line_number":104,"context_line":"of their quota limits. This is because resource claims currently happen on"},{"line_number":105,"context_line":"the compute host, not the API. So it would be possible for a quota check Y to"},{"line_number":106,"context_line":"pass at the API and shortly after a racing request X also passed quota check"},{"line_number":107,"context_line":"will have consumed the remaining resources allowed for the project, and then"},{"line_number":108,"context_line":"request Y will consume more resources than the quota afterward."}],"source_content_type":"text/x-rst","patch_set":3,"id":"7a77a97e_e1e0a698","line":105,"range":{"start_line":104,"start_character":39,"end_line":105,"end_character":16},"updated":"2016-11-17 21:52:12.000000000","message":"Depends on which resource right? Instances are created in the API now, conductor soon. You can hit a lot of the resource limits with just info about the instance right?","commit_id":"fb4a2a375531a28b20555c64fda4f490ab5cf7c7"},{"author":{"_account_id":4690,"name":"melanie witt","display_name":"melwitt","email":"melwittt@gmail.com","username":"melwitt"},"change_message_id":"7b162fc53232544241310cfb986dd4eb677c4874","unresolved":false,"context_lines":[{"line_number":101,"context_line":""},{"line_number":102,"context_line":"With the resource counting approach, it will be possible for a project to"},{"line_number":103,"context_line":"consume more resources than they have quota if they are racing near the end"},{"line_number":104,"context_line":"of their quota limits. This is because resource claims currently happen on"},{"line_number":105,"context_line":"the compute host, not the API. So it would be possible for a quota check Y to"},{"line_number":106,"context_line":"pass at the API and shortly after a racing request X also passed quota check"},{"line_number":107,"context_line":"will have consumed the remaining resources allowed for the project, and then"},{"line_number":108,"context_line":"request Y will consume more resources than the quota afterward."}],"source_content_type":"text/x-rst","patch_set":3,"id":"7a77a97e_2ce615b0","line":105,"range":{"start_line":104,"start_character":39,"end_line":105,"end_character":16},"in_reply_to":"7a77a97e_e1e0a698","updated":"2016-11-17 22:15:26.000000000","message":"Here I was thinking of when the resource allocations pop up. I had been thinking the instance_claim causes that but I\u0027m not sure now. As we discussed in IRC, this is anyway moot since I had forgotten we can\u0027t just query placement tables and we would have to go through the placement API.","commit_id":"fb4a2a375531a28b20555c64fda4f490ab5cf7c7"},{"author":{"_account_id":4393,"name":"Dan Smith","email":"dms@danplanet.com","username":"danms"},"change_message_id":"f84a058e0e1a7e131a7336b2517a534879f5afc0","unresolved":false,"context_lines":[{"line_number":119,"context_line":""},{"line_number":120,"context_line":"  1. Get all instance uuids for project/user from instance_mappings in API DB."},{"line_number":121,"context_line":"  2. Get all cores/ram/disk by instance uuids (consumer_id) from allocations"},{"line_number":122,"context_line":"     in API DB."},{"line_number":123,"context_line":""},{"line_number":124,"context_line":"All other resources should be able to be counted in one step:"},{"line_number":125,"context_line":""}],"source_content_type":"text/x-rst","patch_set":3,"id":"7a77a97e_019762ec","line":122,"updated":"2016-11-17 21:52:12.000000000","message":"You can\u0027t do this. The allocations table is owned by placement, and the regular API stuff can\u0027t look at it directly (without violating \"the rules\").\n\nUntil the placement stuff knows about tenants and can provide this for us, I think you have to do this via the instance/flavor record.\n\nUnfortunately, it\u0027s not likely to be very performant to check the flavor, although we were hoping to remove/deprecate the instance.vcpu, instance.memory_mb, etc fields soon.","commit_id":"fb4a2a375531a28b20555c64fda4f490ab5cf7c7"},{"author":{"_account_id":4690,"name":"melanie witt","display_name":"melwitt","email":"melwittt@gmail.com","username":"melwitt"},"change_message_id":"7b162fc53232544241310cfb986dd4eb677c4874","unresolved":false,"context_lines":[{"line_number":119,"context_line":""},{"line_number":120,"context_line":"  1. Get all instance uuids for project/user from instance_mappings in API DB."},{"line_number":121,"context_line":"  2. Get all cores/ram/disk by instance uuids (consumer_id) from allocations"},{"line_number":122,"context_line":"     in API DB."},{"line_number":123,"context_line":""},{"line_number":124,"context_line":"All other resources should be able to be counted in one step:"},{"line_number":125,"context_line":""}],"source_content_type":"text/x-rst","patch_set":3,"id":"7a77a97e_8caf2929","line":122,"in_reply_to":"7a77a97e_019762ec","updated":"2016-11-17 22:15:26.000000000","message":"Right, I completely forgot we can\u0027t just read placement tables. :( I\u0027ll update this too.","commit_id":"fb4a2a375531a28b20555c64fda4f490ab5cf7c7"},{"author":{"_account_id":4393,"name":"Dan Smith","email":"dms@danplanet.com","username":"danms"},"change_message_id":"f84a058e0e1a7e131a7336b2517a534879f5afc0","unresolved":false,"context_lines":[{"line_number":157,"context_line":""},{"line_number":158,"context_line":"Nova developers will no longer call quota \"reserve,\" \"commit,\" or \"rollback.\""},{"line_number":159,"context_line":"Instead, they will call quota \"check_resource_count\" or similar when adding a"},{"line_number":160,"context_line":"new API which will consume quota."},{"line_number":161,"context_line":""},{"line_number":162,"context_line":""},{"line_number":163,"context_line":"Implementation"}],"source_content_type":"text/x-rst","patch_set":3,"id":"7a77a97e_21c37eee","line":160,"updated":"2016-11-17 21:52:12.000000000","message":"s/^.*$/Euphoria./","commit_id":"fb4a2a375531a28b20555c64fda4f490ab5cf7c7"},{"author":{"_account_id":4690,"name":"melanie witt","display_name":"melwitt","email":"melwittt@gmail.com","username":"melwitt"},"change_message_id":"698e51a15a52514f9618160beed431353b0dc530","unresolved":false,"context_lines":[{"line_number":113,"context_line":"Performance might be adversely affected in the case of counting resources such"},{"line_number":114,"context_line":"as cores and ram. This is because there is currently no project association"},{"line_number":115,"context_line":"stored in the allocations records. To count these resources, the following"},{"line_number":116,"context_line":"steps are required:"},{"line_number":117,"context_line":""},{"line_number":118,"context_line":"  1. Get all instance uuids for project/user from instance_mappings in API DB."},{"line_number":119,"context_line":"  2. Get all instance records by uuids from all cells and parse their flavors"}],"source_content_type":"text/x-rst","patch_set":4,"id":"7a77a97e_c749fab8","line":116,"updated":"2016-11-17 22:38:36.000000000","message":"I should have updated this part with s/might/will/ and mention that in the future we can query the placement API once it has more data such as project so we could do an efficient query through it. I\u0027ll update at the same time I respin for other impending comments.","commit_id":"de05639fba85bb992d256178f42b13427451c0c2"},{"author":{"_account_id":4393,"name":"Dan Smith","email":"dms@danplanet.com","username":"danms"},"change_message_id":"cf524039af0bf2f54746705098d9b6623689297e","unresolved":false,"context_lines":[{"line_number":115,"context_line":"stored in the allocations records. To count these resources, the following"},{"line_number":116,"context_line":"steps are required:"},{"line_number":117,"context_line":""},{"line_number":118,"context_line":"  1. Get all instance uuids for project/user from instance_mappings in API DB."},{"line_number":119,"context_line":"  2. Get all instance records by uuids from all cells and parse their flavors"},{"line_number":120,"context_line":"     to add up cores and ram consumed for a project."},{"line_number":121,"context_line":""}],"source_content_type":"text/x-rst","patch_set":4,"id":"7a77a97e_adef7fe7","line":118,"updated":"2016-11-17 23:28:46.000000000","message":"I don\u0027t think we need this step for cores/ram. I guess we might already have done it for counting instances, but if we have to do the next step, maybe we can just keep a tally of all the instance objects in all the cells and avoid having to do this?","commit_id":"de05639fba85bb992d256178f42b13427451c0c2"},{"author":{"_account_id":4393,"name":"Dan Smith","email":"dms@danplanet.com","username":"danms"},"change_message_id":"cf524039af0bf2f54746705098d9b6623689297e","unresolved":false,"context_lines":[{"line_number":117,"context_line":""},{"line_number":118,"context_line":"  1. Get all instance uuids for project/user from instance_mappings in API DB."},{"line_number":119,"context_line":"  2. Get all instance records by uuids from all cells and parse their flavors"},{"line_number":120,"context_line":"     to add up cores and ram consumed for a project."},{"line_number":121,"context_line":""},{"line_number":122,"context_line":"All other resources should be able to be counted in one step:"},{"line_number":123,"context_line":""}],"source_content_type":"text/x-rst","patch_set":4,"id":"7a77a97e_8dcd6387","line":120,"updated":"2016-11-17 23:28:46.000000000","message":"We can just do:\n\n instance_get_all_by_filters(filters\u003d{\u0027project_id\u0027: myproj},\n                             expected_attrs\u003d[\u0027flavor\u0027])\n\nagainst each cell database. That will give us the instance objects we need for counting instead of having to pass a giant list of uuids first. Maybe we\u0027ll already have that list, but I think the query will be smaller and faster if we filter by project, and we\u0027ll get anything in the database for the project, which will be more up-to-date than a list we compiled a few seconds ago.","commit_id":"de05639fba85bb992d256178f42b13427451c0c2"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"1e1463b6de8d27f78e8393f2cbef139c48a96f2a","unresolved":false,"context_lines":[{"line_number":8,"context_line":"Count resources to check quota in API for cells"},{"line_number":9,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"},{"line_number":10,"context_line":""},{"line_number":11,"context_line":"https://blueprints.launchpad.net/nova/+spec/cells-count-resources-to-check-quota-in-api"},{"line_number":12,"context_line":""},{"line_number":13,"context_line":"For cellsv2, quota tables are moving to the API database as data global to"},{"line_number":14,"context_line":"a deployment. Currently, for instance delete, quota reservations are made in"}],"source_content_type":"text/x-rst","patch_set":5,"id":"7a77a97e_55fbd1c0","line":11,"updated":"2016-11-18 01:56:14.000000000","message":"We could have also just tracked this against this blueprint:\n\nhttps://blueprints.launchpad.net/nova/+spec/cells-quota-api-db\n\nSince this is going to supersede that one. But we could just also supersede it in LP.","commit_id":"ee696e984abfa20498e5c478a88b1f514fc67755"},{"author":{"_account_id":4690,"name":"melanie witt","display_name":"melwitt","email":"melwittt@gmail.com","username":"melwitt"},"change_message_id":"c47b6f85375a64417018d7b7ccd96abf6f260a88","unresolved":false,"context_lines":[{"line_number":8,"context_line":"Count resources to check quota in API for cells"},{"line_number":9,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"},{"line_number":10,"context_line":""},{"line_number":11,"context_line":"https://blueprints.launchpad.net/nova/+spec/cells-count-resources-to-check-quota-in-api"},{"line_number":12,"context_line":""},{"line_number":13,"context_line":"For cellsv2, quota tables are moving to the API database as data global to"},{"line_number":14,"context_line":"a deployment. Currently, for instance delete, quota reservations are made in"}],"source_content_type":"text/x-rst","patch_set":5,"id":"7a77a97e_e98ec42c","line":11,"in_reply_to":"7a77a97e_55fbd1c0","updated":"2016-11-18 04:22:59.000000000","message":"Okay, I wasn\u0027t thinking of it that way since it\u0027s not completely superseding. We need the quotas and project_user_quotas tables for limits in the case that anyone uses the quota-update API. And I think we also need to have the quota_classes table until we get rid of quota classes.","commit_id":"ee696e984abfa20498e5c478a88b1f514fc67755"},{"author":{"_account_id":4690,"name":"melanie witt","display_name":"melwitt","email":"melwittt@gmail.com","username":"melwitt"},"change_message_id":"58f6137e1a757a2e4d49d7cd9ced403f4c762a28","unresolved":false,"context_lines":[{"line_number":8,"context_line":"Count resources to check quota in API for cells"},{"line_number":9,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"},{"line_number":10,"context_line":""},{"line_number":11,"context_line":"https://blueprints.launchpad.net/nova/+spec/cells-count-resources-to-check-quota-in-api"},{"line_number":12,"context_line":""},{"line_number":13,"context_line":"For cellsv2, quota tables are moving to the API database as data global to"},{"line_number":14,"context_line":"a deployment. Currently, for instance delete, quota reservations are made in"}],"source_content_type":"text/x-rst","patch_set":5,"id":"7a77a97e_99110607","line":11,"in_reply_to":"7a77a97e_a6285d34","updated":"2016-11-18 16:59:53.000000000","message":"We have the tables but we don\u0027t yet have the code that reads from both tables (API and non, for quotas, project_user_quotas, and quota_classes) and the online data migration for the quota limits. If you think that can go under this spec then I think we could close the other one.","commit_id":"ee696e984abfa20498e5c478a88b1f514fc67755"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"469a6460176ae41b4eccd25f18134f0d4b78f5ad","unresolved":false,"context_lines":[{"line_number":8,"context_line":"Count resources to check quota in API for cells"},{"line_number":9,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"},{"line_number":10,"context_line":""},{"line_number":11,"context_line":"https://blueprints.launchpad.net/nova/+spec/cells-count-resources-to-check-quota-in-api"},{"line_number":12,"context_line":""},{"line_number":13,"context_line":"For cellsv2, quota tables are moving to the API database as data global to"},{"line_number":14,"context_line":"a deployment. Currently, for instance delete, quota reservations are made in"}],"source_content_type":"text/x-rst","patch_set":5,"id":"7a77a97e_a6285d34","line":11,"in_reply_to":"7a77a97e_e98ec42c","updated":"2016-11-18 16:27:14.000000000","message":"Since we have those tables, can we just close cells-quota-api-db then?","commit_id":"ee696e984abfa20498e5c478a88b1f514fc67755"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"1e1463b6de8d27f78e8393f2cbef139c48a96f2a","unresolved":false,"context_lines":[{"line_number":20,"context_line":"We propose a new approach of counting consumed resources and checking the"},{"line_number":21,"context_line":"count against the quota limits in the API instead of the current reserve/commit"},{"line_number":22,"context_line":"model where a reservation record is created, quota usage records are created"},{"line_number":23,"context_line":"and marked as \"in_use\" when they are committed, and the reservarion record"},{"line_number":24,"context_line":"deleted."},{"line_number":25,"context_line":""},{"line_number":26,"context_line":""}],"source_content_type":"text/x-rst","patch_set":5,"id":"7a77a97e_751fed07","line":23,"range":{"start_line":23,"start_character":56,"end_line":23,"end_character":67},"updated":"2016-11-18 01:56:14.000000000","message":"reservation","commit_id":"ee696e984abfa20498e5c478a88b1f514fc67755"},{"author":{"_account_id":4690,"name":"melanie witt","display_name":"melwitt","email":"melwittt@gmail.com","username":"melwitt"},"change_message_id":"c47b6f85375a64417018d7b7ccd96abf6f260a88","unresolved":false,"context_lines":[{"line_number":20,"context_line":"We propose a new approach of counting consumed resources and checking the"},{"line_number":21,"context_line":"count against the quota limits in the API instead of the current reserve/commit"},{"line_number":22,"context_line":"model where a reservation record is created, quota usage records are created"},{"line_number":23,"context_line":"and marked as \"in_use\" when they are committed, and the reservarion record"},{"line_number":24,"context_line":"deleted."},{"line_number":25,"context_line":""},{"line_number":26,"context_line":""}],"source_content_type":"text/x-rst","patch_set":5,"id":"7a77a97e_497ed819","line":23,"range":{"start_line":23,"start_character":56,"end_line":23,"end_character":67},"in_reply_to":"7a77a97e_751fed07","updated":"2016-11-18 04:22:59.000000000","message":"Dangit","commit_id":"ee696e984abfa20498e5c478a88b1f514fc67755"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"1e1463b6de8d27f78e8393f2cbef139c48a96f2a","unresolved":false,"context_lines":[{"line_number":36,"context_line":""},{"line_number":37,"context_line":"For instance delete, resources are first reserved in the API when a request is"},{"line_number":38,"context_line":"received and then the reservation is later committed in compute when the"},{"line_number":39,"context_line":"resources are freed. In cellsv2, this means compute cells will write to the API"},{"line_number":40,"context_line":"database for the quota commit. By counting resources in the API to check quota,"},{"line_number":41,"context_line":"we can reduce [*]_ the need for compute cells to write to the API database."},{"line_number":42,"context_line":"At least, we will eliminate the situation where a quota reserve/commit is split"},{"line_number":43,"context_line":"across the API cell and compute cells."}],"source_content_type":"text/x-rst","patch_set":5,"id":"7a77a97e_d5de0110","line":40,"range":{"start_line":39,"start_character":21,"end_line":40,"end_character":30},"updated":"2016-11-18 01:56:14.000000000","message":"compute cells will write to the API database...on L17 it says \"ideally, cells should not need to have the API database connection in their configuration.\" I\u0027m a bit fuzzy on what this is saying.","commit_id":"ee696e984abfa20498e5c478a88b1f514fc67755"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"469a6460176ae41b4eccd25f18134f0d4b78f5ad","unresolved":false,"context_lines":[{"line_number":36,"context_line":""},{"line_number":37,"context_line":"For instance delete, resources are first reserved in the API when a request is"},{"line_number":38,"context_line":"received and then the reservation is later committed in compute when the"},{"line_number":39,"context_line":"resources are freed. In cellsv2, this means compute cells will write to the API"},{"line_number":40,"context_line":"database for the quota commit. By counting resources in the API to check quota,"},{"line_number":41,"context_line":"we can reduce [*]_ the need for compute cells to write to the API database."},{"line_number":42,"context_line":"At least, we will eliminate the situation where a quota reserve/commit is split"},{"line_number":43,"context_line":"across the API cell and compute cells."}],"source_content_type":"text/x-rst","patch_set":5,"id":"7a77a97e_66d9e556","line":40,"range":{"start_line":39,"start_character":21,"end_line":40,"end_character":30},"in_reply_to":"7a77a97e_49a5f89f","updated":"2016-11-18 16:27:14.000000000","message":"You\u0027d probably get dinged for saying \u0027child cells\u0027 too though as that\u0027s a cells v1 concept. :)","commit_id":"ee696e984abfa20498e5c478a88b1f514fc67755"},{"author":{"_account_id":4690,"name":"melanie witt","display_name":"melwitt","email":"melwittt@gmail.com","username":"melwitt"},"change_message_id":"c47b6f85375a64417018d7b7ccd96abf6f260a88","unresolved":false,"context_lines":[{"line_number":36,"context_line":""},{"line_number":37,"context_line":"For instance delete, resources are first reserved in the API when a request is"},{"line_number":38,"context_line":"received and then the reservation is later committed in compute when the"},{"line_number":39,"context_line":"resources are freed. In cellsv2, this means compute cells will write to the API"},{"line_number":40,"context_line":"database for the quota commit. By counting resources in the API to check quota,"},{"line_number":41,"context_line":"we can reduce [*]_ the need for compute cells to write to the API database."},{"line_number":42,"context_line":"At least, we will eliminate the situation where a quota reserve/commit is split"},{"line_number":43,"context_line":"across the API cell and compute cells."}],"source_content_type":"text/x-rst","patch_set":5,"id":"7a77a97e_49a5f89f","line":40,"range":{"start_line":39,"start_character":21,"end_line":40,"end_character":30},"in_reply_to":"7a77a97e_b32fdd86","updated":"2016-11-18 04:22:59.000000000","message":"Yes, sorry I think I should have used the words \"child cells\" instead of compute cells. I meant if we keep the original quota system, child cells will make calls that ultimately read/write in the API database.","commit_id":"ee696e984abfa20498e5c478a88b1f514fc67755"},{"author":{"_account_id":4393,"name":"Dan Smith","email":"dms@danplanet.com","username":"danms"},"change_message_id":"8f3eaf0a764ad5350b8a5844d58bd2d54e72fbef","unresolved":false,"context_lines":[{"line_number":36,"context_line":""},{"line_number":37,"context_line":"For instance delete, resources are first reserved in the API when a request is"},{"line_number":38,"context_line":"received and then the reservation is later committed in compute when the"},{"line_number":39,"context_line":"resources are freed. In cellsv2, this means compute cells will write to the API"},{"line_number":40,"context_line":"database for the quota commit. By counting resources in the API to check quota,"},{"line_number":41,"context_line":"we can reduce [*]_ the need for compute cells to write to the API database."},{"line_number":42,"context_line":"At least, we will eliminate the situation where a quota reserve/commit is split"},{"line_number":43,"context_line":"across the API cell and compute cells."}],"source_content_type":"text/x-rst","patch_set":5,"id":"7a77a97e_d8de3810","line":40,"range":{"start_line":39,"start_character":21,"end_line":40,"end_character":30},"in_reply_to":"7a77a97e_d5de0110","updated":"2016-11-18 02:04:06.000000000","message":"I think this is \"if we went with the transplanted original quota system...\"","commit_id":"ee696e984abfa20498e5c478a88b1f514fc67755"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"3bebbf7031c7012682e4a052c7ed864ad136b908","unresolved":false,"context_lines":[{"line_number":36,"context_line":""},{"line_number":37,"context_line":"For instance delete, resources are first reserved in the API when a request is"},{"line_number":38,"context_line":"received and then the reservation is later committed in compute when the"},{"line_number":39,"context_line":"resources are freed. In cellsv2, this means compute cells will write to the API"},{"line_number":40,"context_line":"database for the quota commit. By counting resources in the API to check quota,"},{"line_number":41,"context_line":"we can reduce [*]_ the need for compute cells to write to the API database."},{"line_number":42,"context_line":"At least, we will eliminate the situation where a quota reserve/commit is split"},{"line_number":43,"context_line":"across the API cell and compute cells."}],"source_content_type":"text/x-rst","patch_set":5,"id":"7a77a97e_b32fdd86","line":40,"range":{"start_line":39,"start_character":21,"end_line":40,"end_character":30},"in_reply_to":"7a77a97e_d8de3810","updated":"2016-11-18 02:28:27.000000000","message":"Ah OK that makes sense. In other words, if we don\u0027t change, the computes in the cells have to call up to the API which we want to avoid.","commit_id":"ee696e984abfa20498e5c478a88b1f514fc67755"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"1e1463b6de8d27f78e8393f2cbef139c48a96f2a","unresolved":false,"context_lines":[{"line_number":60,"context_line":"Consumed resources will be counted to check quota instead of the current"},{"line_number":61,"context_line":"reserve/commit/rollback model."},{"line_number":62,"context_line":""},{"line_number":63,"context_line":"* \"Reserve,\" \"commit,\" and \"rollback\" calls will be removed everywhere."},{"line_number":64,"context_line":""},{"line_number":65,"context_line":"* \"Reserve\" calls will be replaced with something like \"check_resource_count\""},{"line_number":66,"context_line":"  which will query the databases for consumed resources, count them, and raise"}],"source_content_type":"text/x-rst","patch_set":5,"id":"7a77a97e_f5313d5b","line":63,"updated":"2016-11-18 01:56:14.000000000","message":"Except nova-network I guess.","commit_id":"ee696e984abfa20498e5c478a88b1f514fc67755"},{"author":{"_account_id":4690,"name":"melanie witt","display_name":"melwitt","email":"melwittt@gmail.com","username":"melwitt"},"change_message_id":"c47b6f85375a64417018d7b7ccd96abf6f260a88","unresolved":false,"context_lines":[{"line_number":60,"context_line":"Consumed resources will be counted to check quota instead of the current"},{"line_number":61,"context_line":"reserve/commit/rollback model."},{"line_number":62,"context_line":""},{"line_number":63,"context_line":"* \"Reserve,\" \"commit,\" and \"rollback\" calls will be removed everywhere."},{"line_number":64,"context_line":""},{"line_number":65,"context_line":"* \"Reserve\" calls will be replaced with something like \"check_resource_count\""},{"line_number":66,"context_line":"  which will query the databases for consumed resources, count them, and raise"}],"source_content_type":"text/x-rst","patch_set":5,"id":"7a77a97e_e9e324d7","line":63,"in_reply_to":"7a77a97e_7340e534","updated":"2016-11-18 04:22:59.000000000","message":"Sorry, I didn\u0027t mean that nova-network will use reserve/commit/rollback APIs. I meant the nova-network service will have to read quota limits from the API database for fixed_ips and floating_ips allocations until it is removed.","commit_id":"ee696e984abfa20498e5c478a88b1f514fc67755"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"3bebbf7031c7012682e4a052c7ed864ad136b908","unresolved":false,"context_lines":[{"line_number":60,"context_line":"Consumed resources will be counted to check quota instead of the current"},{"line_number":61,"context_line":"reserve/commit/rollback model."},{"line_number":62,"context_line":""},{"line_number":63,"context_line":"* \"Reserve,\" \"commit,\" and \"rollback\" calls will be removed everywhere."},{"line_number":64,"context_line":""},{"line_number":65,"context_line":"* \"Reserve\" calls will be replaced with something like \"check_resource_count\""},{"line_number":66,"context_line":"  which will query the databases for consumed resources, count them, and raise"}],"source_content_type":"text/x-rst","patch_set":5,"id":"7a77a97e_7340e534","line":63,"in_reply_to":"7a77a97e_78d0c4f6","updated":"2016-11-18 02:28:27.000000000","message":"I\u0027m referring to the note on line 44.","commit_id":"ee696e984abfa20498e5c478a88b1f514fc67755"},{"author":{"_account_id":782,"name":"John Garbutt","email":"john@johngarbutt.com","username":"johngarbutt"},"change_message_id":"5e7326d7a26587fe6f9364017904d31f16f856ef","unresolved":false,"context_lines":[{"line_number":60,"context_line":"Consumed resources will be counted to check quota instead of the current"},{"line_number":61,"context_line":"reserve/commit/rollback model."},{"line_number":62,"context_line":""},{"line_number":63,"context_line":"* \"Reserve,\" \"commit,\" and \"rollback\" calls will be removed everywhere."},{"line_number":64,"context_line":""},{"line_number":65,"context_line":"* \"Reserve\" calls will be replaced with something like \"check_resource_count\""},{"line_number":66,"context_line":"  which will query the databases for consumed resources, count them, and raise"}],"source_content_type":"text/x-rst","patch_set":5,"id":"7a77a97e_8b661a4f","line":63,"in_reply_to":"7a77a97e_e9e324d7","updated":"2016-11-18 10:55:10.000000000","message":"ah, I just miss read that bit too. but happy with the answer here.","commit_id":"ee696e984abfa20498e5c478a88b1f514fc67755"},{"author":{"_account_id":4393,"name":"Dan Smith","email":"dms@danplanet.com","username":"danms"},"change_message_id":"8f3eaf0a764ad5350b8a5844d58bd2d54e72fbef","unresolved":false,"context_lines":[{"line_number":60,"context_line":"Consumed resources will be counted to check quota instead of the current"},{"line_number":61,"context_line":"reserve/commit/rollback model."},{"line_number":62,"context_line":""},{"line_number":63,"context_line":"* \"Reserve,\" \"commit,\" and \"rollback\" calls will be removed everywhere."},{"line_number":64,"context_line":""},{"line_number":65,"context_line":"* \"Reserve\" calls will be replaced with something like \"check_resource_count\""},{"line_number":66,"context_line":"  which will query the databases for consumed resources, count them, and raise"}],"source_content_type":"text/x-rst","patch_set":5,"id":"7a77a97e_78d0c4f6","line":63,"in_reply_to":"7a77a97e_f5313d5b","updated":"2016-11-18 02:04:06.000000000","message":"I dunno, they won\u0027t be honored. Why wouldn\u0027t we remove them?","commit_id":"ee696e984abfa20498e5c478a88b1f514fc67755"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"1e1463b6de8d27f78e8393f2cbef139c48a96f2a","unresolved":false,"context_lines":[{"line_number":79,"context_line":"Data model impact"},{"line_number":80,"context_line":"-----------------"},{"line_number":81,"context_line":""},{"line_number":82,"context_line":"None"},{"line_number":83,"context_line":""},{"line_number":84,"context_line":"REST API impact"},{"line_number":85,"context_line":"---------------"}],"source_content_type":"text/x-rst","patch_set":5,"id":"7a77a97e_159dd909","line":82,"updated":"2016-11-18 01:56:14.000000000","message":"Can we drop any of the tables that we moved to the API DB, like do we need the reservations/usages table anymore? Or we still need to have everything while we have nova-network for the old style stuff?","commit_id":"ee696e984abfa20498e5c478a88b1f514fc67755"},{"author":{"_account_id":4393,"name":"Dan Smith","email":"dms@danplanet.com","username":"danms"},"change_message_id":"8f3eaf0a764ad5350b8a5844d58bd2d54e72fbef","unresolved":false,"context_lines":[{"line_number":79,"context_line":"Data model impact"},{"line_number":80,"context_line":"-----------------"},{"line_number":81,"context_line":""},{"line_number":82,"context_line":"None"},{"line_number":83,"context_line":""},{"line_number":84,"context_line":"REST API impact"},{"line_number":85,"context_line":"---------------"}],"source_content_type":"text/x-rst","patch_set":5,"id":"7a77a97e_98d92025","line":82,"in_reply_to":"7a77a97e_159dd909","updated":"2016-11-18 02:04:06.000000000","message":"Yes, we can I think.","commit_id":"ee696e984abfa20498e5c478a88b1f514fc67755"},{"author":{"_account_id":782,"name":"John Garbutt","email":"john@johngarbutt.com","username":"johngarbutt"},"change_message_id":"5e7326d7a26587fe6f9364017904d31f16f856ef","unresolved":false,"context_lines":[{"line_number":79,"context_line":"Data model impact"},{"line_number":80,"context_line":"-----------------"},{"line_number":81,"context_line":""},{"line_number":82,"context_line":"None"},{"line_number":83,"context_line":""},{"line_number":84,"context_line":"REST API impact"},{"line_number":85,"context_line":"---------------"}],"source_content_type":"text/x-rst","patch_set":5,"id":"7a77a97e_8bcd3a33","line":82,"in_reply_to":"7a77a97e_2938bc3c","updated":"2016-11-18 10:55:10.000000000","message":"I guess we can\u0027t drop them till next release anyways. Ho hum.","commit_id":"ee696e984abfa20498e5c478a88b1f514fc67755"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"469a6460176ae41b4eccd25f18134f0d4b78f5ad","unresolved":false,"context_lines":[{"line_number":79,"context_line":"Data model impact"},{"line_number":80,"context_line":"-----------------"},{"line_number":81,"context_line":""},{"line_number":82,"context_line":"None"},{"line_number":83,"context_line":""},{"line_number":84,"context_line":"REST API impact"},{"line_number":85,"context_line":"---------------"}],"source_content_type":"text/x-rst","patch_set":5,"id":"7a77a97e_c6d1d924","line":82,"in_reply_to":"7a77a97e_8bcd3a33","updated":"2016-11-18 16:27:14.000000000","message":"John, why not? Nothing is actually populating those tables in the API DB yet.","commit_id":"ee696e984abfa20498e5c478a88b1f514fc67755"},{"author":{"_account_id":4690,"name":"melanie witt","display_name":"melwitt","email":"melwittt@gmail.com","username":"melwitt"},"change_message_id":"c47b6f85375a64417018d7b7ccd96abf6f260a88","unresolved":false,"context_lines":[{"line_number":79,"context_line":"Data model impact"},{"line_number":80,"context_line":"-----------------"},{"line_number":81,"context_line":""},{"line_number":82,"context_line":"None"},{"line_number":83,"context_line":""},{"line_number":84,"context_line":"REST API impact"},{"line_number":85,"context_line":"---------------"}],"source_content_type":"text/x-rst","patch_set":5,"id":"7a77a97e_2938bc3c","line":82,"in_reply_to":"7a77a97e_98d92025","updated":"2016-11-18 04:22:59.000000000","message":"I think we\u0027ll only need the quotas, project_user_quotas, and quota_classes tables. Nova-network will read the limits from the API database and then count ips locally in the cell database to make the quota pass/fail decision.","commit_id":"ee696e984abfa20498e5c478a88b1f514fc67755"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"1e1463b6de8d27f78e8393f2cbef139c48a96f2a","unresolved":false,"context_lines":[{"line_number":105,"context_line":"across instances in separate databases. So it would be possible for a quota"},{"line_number":106,"context_line":"check Y to pass at the API and shortly after a racing request X also passed"},{"line_number":107,"context_line":"quota check will have consumed the remaining resources allowed for the project,"},{"line_number":108,"context_line":"and then request Y will consume more resources than the quota afterward."},{"line_number":109,"context_line":""},{"line_number":110,"context_line":"Performance Impact"},{"line_number":111,"context_line":"------------------"}],"source_content_type":"text/x-rst","patch_set":5,"id":"7a77a97e_9591e9f3","line":108,"range":{"start_line":108,"start_character":56,"end_line":108,"end_character":61},"updated":"2016-11-18 01:56:14.000000000","message":"s/quota/limit/?","commit_id":"ee696e984abfa20498e5c478a88b1f514fc67755"},{"author":{"_account_id":4690,"name":"melanie witt","display_name":"melwitt","email":"melwittt@gmail.com","username":"melwitt"},"change_message_id":"c47b6f85375a64417018d7b7ccd96abf6f260a88","unresolved":false,"context_lines":[{"line_number":105,"context_line":"across instances in separate databases. So it would be possible for a quota"},{"line_number":106,"context_line":"check Y to pass at the API and shortly after a racing request X also passed"},{"line_number":107,"context_line":"quota check will have consumed the remaining resources allowed for the project,"},{"line_number":108,"context_line":"and then request Y will consume more resources than the quota afterward."},{"line_number":109,"context_line":""},{"line_number":110,"context_line":"Performance Impact"},{"line_number":111,"context_line":"------------------"}],"source_content_type":"text/x-rst","patch_set":5,"id":"7a77a97e_897550fb","line":108,"range":{"start_line":108,"start_character":56,"end_line":108,"end_character":61},"in_reply_to":"7a77a97e_9591e9f3","updated":"2016-11-18 04:22:59.000000000","message":"Yes, quota limit.","commit_id":"ee696e984abfa20498e5c478a88b1f514fc67755"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"3bebbf7031c7012682e4a052c7ed864ad136b908","unresolved":false,"context_lines":[{"line_number":112,"context_line":""},{"line_number":113,"context_line":"Performance will be adversely affected in the case of counting resources such"},{"line_number":114,"context_line":"as cores and ram. This is because there is currently no project association"},{"line_number":115,"context_line":"stored in the allocations records at present. In the future, we will be able"},{"line_number":116,"context_line":"to query the placement API once it has more data and we can do an efficient"},{"line_number":117,"context_line":"query through it. Until then, to count cores and ram, the following approach"},{"line_number":118,"context_line":"is required:"},{"line_number":119,"context_line":""},{"line_number":120,"context_line":"  * Get all instances by project per cell, parsing the flavor JSON blobs and"}],"source_content_type":"text/x-rst","patch_set":5,"id":"7a77a97e_b346fd4c","line":117,"range":{"start_line":115,"start_character":46,"end_line":117,"end_character":17},"updated":"2016-11-18 02:28:27.000000000","message":"I\u0027m also a bit lost on the plan here, maybe this is just saying we should be able to do something better with the placement API at some point, but we just don\u0027t have that in place yet - but I\u0027m not even sure if there is a plan for this?","commit_id":"ee696e984abfa20498e5c478a88b1f514fc67755"},{"author":{"_account_id":4690,"name":"melanie witt","display_name":"melwitt","email":"melwittt@gmail.com","username":"melwitt"},"change_message_id":"c47b6f85375a64417018d7b7ccd96abf6f260a88","unresolved":false,"context_lines":[{"line_number":112,"context_line":""},{"line_number":113,"context_line":"Performance will be adversely affected in the case of counting resources such"},{"line_number":114,"context_line":"as cores and ram. This is because there is currently no project association"},{"line_number":115,"context_line":"stored in the allocations records at present. In the future, we will be able"},{"line_number":116,"context_line":"to query the placement API once it has more data and we can do an efficient"},{"line_number":117,"context_line":"query through it. Until then, to count cores and ram, the following approach"},{"line_number":118,"context_line":"is required:"},{"line_number":119,"context_line":""},{"line_number":120,"context_line":"  * Get all instances by project per cell, parsing the flavor JSON blobs and"}],"source_content_type":"text/x-rst","patch_set":5,"id":"7a77a97e_c961083b","line":117,"range":{"start_line":115,"start_character":46,"end_line":117,"end_character":17},"in_reply_to":"7a77a97e_b346fd4c","updated":"2016-11-18 04:22:59.000000000","message":"Yes, trying to say that we aim to do something better with the placement API at some point. RP folks are aware of the idea as of the summit and a brief IRC chat today, so I think we can make something happen down the road.","commit_id":"ee696e984abfa20498e5c478a88b1f514fc67755"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"1e1463b6de8d27f78e8393f2cbef139c48a96f2a","unresolved":false,"context_lines":[{"line_number":117,"context_line":"query through it. Until then, to count cores and ram, the following approach"},{"line_number":118,"context_line":"is required:"},{"line_number":119,"context_line":""},{"line_number":120,"context_line":"  * Get all instances by project per cell, parsing the flavor JSON blobs and"},{"line_number":121,"context_line":"    adding up the counts. For example::"},{"line_number":122,"context_line":""},{"line_number":123,"context_line":"      instance_get_all_by_filters(filters\u003d{\u0027project_id\u0027: myproj},"}],"source_content_type":"text/x-rst","patch_set":5,"id":"7a77a97e_983200f1","line":120,"updated":"2016-11-18 01:56:14.000000000","message":"So this is really, get all instance_mappings for a given project_id, then for each instance_mapping, get the instance record from the cell DB and then get the cpu/ram from the instance.flavor field, and aggregate those all up for our resource check.\n\nThe default limit for instances is 10 and at first most deployments will be a single cells v2 cell, so it\u0027s a single database to query, but still it\u0027s not fun doing this in python.\n\nI\u0027m wondering if there is anything we can do with the results, like if we\u0027re using 60% CPU quota and are processing a request for 10% more CPU, we find we can do that so we then bump our aggregate usage to 70% and if scheduling fails for whatever reason we roll that back to 60%. While processing the aggregate usage we set some dirty bit which other requests would check when starting this and if set, they basically say we can\u0027t trust the current aggregates, and have to process the values. But if the dirty bit isn\u0027t set, nothing else is processing quota for that project and they can assume that if we\u0027re at 70% quota usage they should just go with that. This is quite a bit of rambling and I\u0027m getting the feeling it sounds a lot like the current reserve/commit/rollback system we have today. It just seems like there should be some way for us to aggregate this data in a smarter way w/o needing to do it with python. For example, we send notifications for each instance that\u0027s created and that has the flavor info and the project. That\u0027s coming from all cells. If we had something listening to notifications from all cells and storing that in something like searchlight (which that\u0027s already going to be the case at some point for listing instances across cells), nova could also mine that centralized pushed data rather than pull it all per request. Again, more rambling, but it\u0027s an idea.","commit_id":"ee696e984abfa20498e5c478a88b1f514fc67755"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"3bebbf7031c7012682e4a052c7ed864ad136b908","unresolved":false,"context_lines":[{"line_number":117,"context_line":"query through it. Until then, to count cores and ram, the following approach"},{"line_number":118,"context_line":"is required:"},{"line_number":119,"context_line":""},{"line_number":120,"context_line":"  * Get all instances by project per cell, parsing the flavor JSON blobs and"},{"line_number":121,"context_line":"    adding up the counts. For example::"},{"line_number":122,"context_line":""},{"line_number":123,"context_line":"      instance_get_all_by_filters(filters\u003d{\u0027project_id\u0027: myproj},"}],"source_content_type":"text/x-rst","patch_set":5,"id":"7a77a97e_53fbe9c0","line":120,"in_reply_to":"7a77a97e_18fc3071","updated":"2016-11-18 02:28:27.000000000","message":"Ah OK yeah that\u0027s better then. I still kind of like the idea of pumping all of those notifications into a cassandra db or something and then nova just querying that thing from the API rather than pulling all instances for a project from all cells. But baby steps.","commit_id":"ee696e984abfa20498e5c478a88b1f514fc67755"},{"author":{"_account_id":782,"name":"John Garbutt","email":"john@johngarbutt.com","username":"johngarbutt"},"change_message_id":"5e7326d7a26587fe6f9364017904d31f16f856ef","unresolved":false,"context_lines":[{"line_number":117,"context_line":"query through it. Until then, to count cores and ram, the following approach"},{"line_number":118,"context_line":"is required:"},{"line_number":119,"context_line":""},{"line_number":120,"context_line":"  * Get all instances by project per cell, parsing the flavor JSON blobs and"},{"line_number":121,"context_line":"    adding up the counts. For example::"},{"line_number":122,"context_line":""},{"line_number":123,"context_line":"      instance_get_all_by_filters(filters\u003d{\u0027project_id\u0027: myproj},"}],"source_content_type":"text/x-rst","patch_set":5,"id":"7a77a97e_6e854c58","line":120,"in_reply_to":"7a77a97e_53fbe9c0","updated":"2016-11-18 10:55:10.000000000","message":"yeah, baby steps, I like how this works really well for the single cell case.","commit_id":"ee696e984abfa20498e5c478a88b1f514fc67755"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"469a6460176ae41b4eccd25f18134f0d4b78f5ad","unresolved":false,"context_lines":[{"line_number":117,"context_line":"query through it. Until then, to count cores and ram, the following approach"},{"line_number":118,"context_line":"is required:"},{"line_number":119,"context_line":""},{"line_number":120,"context_line":"  * Get all instances by project per cell, parsing the flavor JSON blobs and"},{"line_number":121,"context_line":"    adding up the counts. For example::"},{"line_number":122,"context_line":""},{"line_number":123,"context_line":"      instance_get_all_by_filters(filters\u003d{\u0027project_id\u0027: myproj},"}],"source_content_type":"text/x-rst","patch_set":5,"id":"7a77a97e_c698f97a","line":120,"in_reply_to":"7a77a97e_6e854c58","updated":"2016-11-18 16:27:14.000000000","message":"I thought about this later too, but it might be beneficial to run some rally scenarios before and after this change to see how much of a hit it is. It will probably not be much in a small CI environment like we have though, unless we setup like 300 fake instances for a single tenant.","commit_id":"ee696e984abfa20498e5c478a88b1f514fc67755"},{"author":{"_account_id":4393,"name":"Dan Smith","email":"dms@danplanet.com","username":"danms"},"change_message_id":"8f3eaf0a764ad5350b8a5844d58bd2d54e72fbef","unresolved":false,"context_lines":[{"line_number":117,"context_line":"query through it. Until then, to count cores and ram, the following approach"},{"line_number":118,"context_line":"is required:"},{"line_number":119,"context_line":""},{"line_number":120,"context_line":"  * Get all instances by project per cell, parsing the flavor JSON blobs and"},{"line_number":121,"context_line":"    adding up the counts. For example::"},{"line_number":122,"context_line":""},{"line_number":123,"context_line":"      instance_get_all_by_filters(filters\u003d{\u0027project_id\u0027: myproj},"}],"source_content_type":"text/x-rst","patch_set":5,"id":"7a77a97e_18fc3071","line":120,"in_reply_to":"7a77a97e_983200f1","updated":"2016-11-18 02:04:06.000000000","message":"No, we can issue get_all_by_filters to each cell and filter by project. I guess we could do a pre-query to avoid hitting cells that you have no instances in, but that\u0027d be an optimization that will pay off occasionally.","commit_id":"ee696e984abfa20498e5c478a88b1f514fc67755"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"1e1463b6de8d27f78e8393f2cbef139c48a96f2a","unresolved":false,"context_lines":[{"line_number":140,"context_line":"    the cell database so this would be a local DB read until nova-network is"},{"line_number":141,"context_line":"    removed."},{"line_number":142,"context_line":"  * metadata_items: this is a limit on allowed number of image metadata items"},{"line_number":143,"context_line":"    and is checked when image metadata requests come in. No counting of"},{"line_number":144,"context_line":"    resources is necessary."},{"line_number":145,"context_line":"  * injected_files: Similar to metadata_items."},{"line_number":146,"context_line":"  * injected_file_content_bytes: Similar to metadata_items."},{"line_number":147,"context_line":"  * injected_file_path_bytes: Similar to metadata_items."}],"source_content_type":"text/x-rst","patch_set":5,"id":"7a77a97e_7867e4e8","line":144,"range":{"start_line":143,"start_character":57,"end_line":144,"end_character":27},"updated":"2016-11-18 01:56:14.000000000","message":"Because...? We haven\u0027t deprecated the image metadata proxy API yet. That\u0027s supposed to happen in ocata though:\n\nhttps://blueprints.launchpad.net/nova/+spec/deprecate-image-meta-proxy-api\n\nBut that hasn\u0027t started yet, and it wouldn\u0027t really matter anyway since you could hit this with a lower microversion. Unless I\u0027m totally missing something about this.\n\n--\n\nAlso, this isn\u0027t just image metadata, it\u0027s also server metadata.\n\nI think maybe the thing I\u0027m missing is the distinction between AbsoluteResource, ReservableResource and CountableResource. Maybe we should describe those somewhere in here.\n\nReservableResources are the aggregate things we have to process system-wide.\n\nAbsoluteResources are the things that are absolute per request.\n\nI\u0027m a bit lost on the difference between ReservableResources and CountableResources except that CountableResources don\u0027t take into account project_id for some reason...","commit_id":"ee696e984abfa20498e5c478a88b1f514fc67755"},{"author":{"_account_id":4690,"name":"melanie witt","display_name":"melwitt","email":"melwittt@gmail.com","username":"melwitt"},"change_message_id":"d042f855f0a64d872e81927fd5fa9c8d8741df03","unresolved":false,"context_lines":[{"line_number":140,"context_line":"    the cell database so this would be a local DB read until nova-network is"},{"line_number":141,"context_line":"    removed."},{"line_number":142,"context_line":"  * metadata_items: this is a limit on allowed number of image metadata items"},{"line_number":143,"context_line":"    and is checked when image metadata requests come in. No counting of"},{"line_number":144,"context_line":"    resources is necessary."},{"line_number":145,"context_line":"  * injected_files: Similar to metadata_items."},{"line_number":146,"context_line":"  * injected_file_content_bytes: Similar to metadata_items."},{"line_number":147,"context_line":"  * injected_file_path_bytes: Similar to metadata_items."}],"source_content_type":"text/x-rst","patch_set":5,"id":"7a77a97e_ee008736","line":144,"range":{"start_line":143,"start_character":57,"end_line":144,"end_character":27},"in_reply_to":"7a77a97e_697634d8","updated":"2016-11-18 06:11:05.000000000","message":"This is the post from Vek I was thinking of:\n\nhttp://lists.openstack.org/pipermail/openstack-dev/2015-December/081334.html","commit_id":"ee696e984abfa20498e5c478a88b1f514fc67755"},{"author":{"_account_id":4690,"name":"melanie witt","display_name":"melwitt","email":"melwittt@gmail.com","username":"melwitt"},"change_message_id":"c47b6f85375a64417018d7b7ccd96abf6f260a88","unresolved":false,"context_lines":[{"line_number":140,"context_line":"    the cell database so this would be a local DB read until nova-network is"},{"line_number":141,"context_line":"    removed."},{"line_number":142,"context_line":"  * metadata_items: this is a limit on allowed number of image metadata items"},{"line_number":143,"context_line":"    and is checked when image metadata requests come in. No counting of"},{"line_number":144,"context_line":"    resources is necessary."},{"line_number":145,"context_line":"  * injected_files: Similar to metadata_items."},{"line_number":146,"context_line":"  * injected_file_content_bytes: Similar to metadata_items."},{"line_number":147,"context_line":"  * injected_file_path_bytes: Similar to metadata_items."}],"source_content_type":"text/x-rst","patch_set":5,"id":"7a77a97e_697634d8","line":144,"range":{"start_line":143,"start_character":57,"end_line":144,"end_character":27},"in_reply_to":"7a77a97e_7867e4e8","updated":"2016-11-18 04:22:59.000000000","message":"What I meant is when I looked at \"metadata items\" and friends, the way it works is the data to compare against the quota limit comes from the request itself. So there isn\u0027t any counting of resources in the database that needs to occur for the quota check. I\u0027m not sure it\u0027s related to what type of resource it is. I just looked in the code at each check and it didn\u0027t look like any database querying of resources is done.\n\nI can add words about what ReservableResource, CountableResource, and AbsoluteResource are based on what I find out. I remember we had a ML thread about them last year maybe.","commit_id":"ee696e984abfa20498e5c478a88b1f514fc67755"},{"author":{"_account_id":782,"name":"John Garbutt","email":"john@johngarbutt.com","username":"johngarbutt"},"change_message_id":"5e7326d7a26587fe6f9364017904d31f16f856ef","unresolved":false,"context_lines":[{"line_number":140,"context_line":"    the cell database so this would be a local DB read until nova-network is"},{"line_number":141,"context_line":"    removed."},{"line_number":142,"context_line":"  * metadata_items: this is a limit on allowed number of image metadata items"},{"line_number":143,"context_line":"    and is checked when image metadata requests come in. No counting of"},{"line_number":144,"context_line":"    resources is necessary."},{"line_number":145,"context_line":"  * injected_files: Similar to metadata_items."},{"line_number":146,"context_line":"  * injected_file_content_bytes: Similar to metadata_items."},{"line_number":147,"context_line":"  * injected_file_path_bytes: Similar to metadata_items."}],"source_content_type":"text/x-rst","patch_set":5,"id":"7a77a97e_0edcd871","line":144,"range":{"start_line":143,"start_character":57,"end_line":144,"end_character":27},"in_reply_to":"7a77a97e_ee008736","updated":"2016-11-18 10:55:10.000000000","message":"Yeah, its not consumed, its a per instance limit that can be tweaked on a per tenant or per user level","commit_id":"ee696e984abfa20498e5c478a88b1f514fc67755"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"1e1463b6de8d27f78e8393f2cbef139c48a96f2a","unresolved":false,"context_lines":[{"line_number":145,"context_line":"  * injected_files: Similar to metadata_items."},{"line_number":146,"context_line":"  * injected_file_content_bytes: Similar to metadata_items."},{"line_number":147,"context_line":"  * injected_file_path_bytes: Similar to metadata_items."},{"line_number":148,"context_line":"  * security_group_rules: Similar to security_groups."},{"line_number":149,"context_line":"  * key_pairs: this can be obtained from key_pairs table in API DB."},{"line_number":150,"context_line":"  * server_groups: this can be obtained from instance_groups table in API DB."},{"line_number":151,"context_line":"  * server_group_members: this can be obtained from instance_group_member table"}],"source_content_type":"text/x-rst","patch_set":5,"id":"7a77a97e_9853e0e1","line":148,"updated":"2016-11-18 01:56:14.000000000","message":"Except rules are countable resources and security groups are reservable resources, for whatever distinction that means (as noted above).","commit_id":"ee696e984abfa20498e5c478a88b1f514fc67755"},{"author":{"_account_id":4690,"name":"melanie witt","display_name":"melwitt","email":"melwittt@gmail.com","username":"melwitt"},"change_message_id":"c47b6f85375a64417018d7b7ccd96abf6f260a88","unresolved":false,"context_lines":[{"line_number":145,"context_line":"  * injected_files: Similar to metadata_items."},{"line_number":146,"context_line":"  * injected_file_content_bytes: Similar to metadata_items."},{"line_number":147,"context_line":"  * injected_file_path_bytes: Similar to metadata_items."},{"line_number":148,"context_line":"  * security_group_rules: Similar to security_groups."},{"line_number":149,"context_line":"  * key_pairs: this can be obtained from key_pairs table in API DB."},{"line_number":150,"context_line":"  * server_groups: this can be obtained from instance_groups table in API DB."},{"line_number":151,"context_line":"  * server_group_members: this can be obtained from instance_group_member table"}],"source_content_type":"text/x-rst","patch_set":5,"id":"7a77a97e_4927b8c9","line":148,"in_reply_to":"7a77a97e_9853e0e1","updated":"2016-11-18 04:22:59.000000000","message":"I was thinking similar in that the limit would have to be read from the API database but the counting would be local in the cell database.","commit_id":"ee696e984abfa20498e5c478a88b1f514fc67755"},{"author":{"_account_id":782,"name":"John Garbutt","email":"john@johngarbutt.com","username":"johngarbutt"},"change_message_id":"5e7326d7a26587fe6f9364017904d31f16f856ef","unresolved":false,"context_lines":[{"line_number":149,"context_line":"  * key_pairs: this can be obtained from key_pairs table in API DB."},{"line_number":150,"context_line":"  * server_groups: this can be obtained from instance_groups table in API DB."},{"line_number":151,"context_line":"  * server_group_members: this can be obtained from instance_group_member table"},{"line_number":152,"context_line":"    in API DB."},{"line_number":153,"context_line":""},{"line_number":154,"context_line":"Other deployer impact"},{"line_number":155,"context_line":"---------------------"}],"source_content_type":"text/x-rst","patch_set":5,"id":"7a77a97e_ce43c011","line":152,"updated":"2016-11-18 10:55:10.000000000","message":"Its nice when you see our choice of where to put this data paying off nicely.","commit_id":"ee696e984abfa20498e5c478a88b1f514fc67755"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"1e1463b6de8d27f78e8393f2cbef139c48a96f2a","unresolved":false,"context_lines":[{"line_number":154,"context_line":"Other deployer impact"},{"line_number":155,"context_line":"---------------------"},{"line_number":156,"context_line":""},{"line_number":157,"context_line":"None"},{"line_number":158,"context_line":""},{"line_number":159,"context_line":"Developer impact"},{"line_number":160,"context_line":"----------------"}],"source_content_type":"text/x-rst","patch_set":5,"id":"7a77a97e_38ae4c98","line":157,"updated":"2016-11-18 01:56:14.000000000","message":"Would we deprecate the nova-manage project quota_usage_refresh command added in newton:\n\nhttps://review.openstack.org/#/c/299621/\n\n? I assume so as it wouldn\u0027t work the same way.","commit_id":"ee696e984abfa20498e5c478a88b1f514fc67755"},{"author":{"_account_id":4690,"name":"melanie witt","display_name":"melwitt","email":"melwittt@gmail.com","username":"melwitt"},"change_message_id":"c47b6f85375a64417018d7b7ccd96abf6f260a88","unresolved":false,"context_lines":[{"line_number":154,"context_line":"Other deployer impact"},{"line_number":155,"context_line":"---------------------"},{"line_number":156,"context_line":""},{"line_number":157,"context_line":"None"},{"line_number":158,"context_line":""},{"line_number":159,"context_line":"Developer impact"},{"line_number":160,"context_line":"----------------"}],"source_content_type":"text/x-rst","patch_set":5,"id":"7a77a97e_a92a0c8e","line":157,"in_reply_to":"7a77a97e_38ae4c98","updated":"2016-11-18 04:22:59.000000000","message":"That\u0027s a good point. Quota refresh would no longer be a thing.","commit_id":"ee696e984abfa20498e5c478a88b1f514fc67755"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"1e1463b6de8d27f78e8393f2cbef139c48a96f2a","unresolved":false,"context_lines":[{"line_number":183,"context_line":"  consumed resources and raises OverQuota if the request would go over quota"},{"line_number":184,"context_line":"  limits."},{"line_number":185,"context_line":""},{"line_number":186,"context_line":"* Remove reserve/commit/rollback everywhere."},{"line_number":187,"context_line":""},{"line_number":188,"context_line":"* Mark \"reserve,\" \"commit,\" and \"rollback\" methods as DEPRECATED in the"},{"line_number":189,"context_line":"  docstrings to prevent their further use."}],"source_content_type":"text/x-rst","patch_set":5,"id":"7a77a97e_180d10e8","line":186,"updated":"2016-11-18 01:56:14.000000000","message":"I thought this was still needed for resize and nova-network?","commit_id":"ee696e984abfa20498e5c478a88b1f514fc67755"},{"author":{"_account_id":4393,"name":"Dan Smith","email":"dms@danplanet.com","username":"danms"},"change_message_id":"8f3eaf0a764ad5350b8a5844d58bd2d54e72fbef","unresolved":false,"context_lines":[{"line_number":183,"context_line":"  consumed resources and raises OverQuota if the request would go over quota"},{"line_number":184,"context_line":"  limits."},{"line_number":185,"context_line":""},{"line_number":186,"context_line":"* Remove reserve/commit/rollback everywhere."},{"line_number":187,"context_line":""},{"line_number":188,"context_line":"* Mark \"reserve,\" \"commit,\" and \"rollback\" methods as DEPRECATED in the"},{"line_number":189,"context_line":"  docstrings to prevent their further use."}],"source_content_type":"text/x-rst","patch_set":5,"id":"7a77a97e_d82c7804","line":186,"in_reply_to":"7a77a97e_180d10e8","updated":"2016-11-18 02:04:06.000000000","message":"Not for resize, no.\n\nAnd nova-net can do counting-based checking the same as everything else, maybe in allocate_ip_for_instance() or whatever.","commit_id":"ee696e984abfa20498e5c478a88b1f514fc67755"},{"author":{"_account_id":4690,"name":"melanie witt","display_name":"melwitt","email":"melwittt@gmail.com","username":"melwitt"},"change_message_id":"c47b6f85375a64417018d7b7ccd96abf6f260a88","unresolved":false,"context_lines":[{"line_number":183,"context_line":"  consumed resources and raises OverQuota if the request would go over quota"},{"line_number":184,"context_line":"  limits."},{"line_number":185,"context_line":""},{"line_number":186,"context_line":"* Remove reserve/commit/rollback everywhere."},{"line_number":187,"context_line":""},{"line_number":188,"context_line":"* Mark \"reserve,\" \"commit,\" and \"rollback\" methods as DEPRECATED in the"},{"line_number":189,"context_line":"  docstrings to prevent their further use."}],"source_content_type":"text/x-rst","patch_set":5,"id":"7a77a97e_690b942a","line":186,"in_reply_to":"7a77a97e_d82c7804","updated":"2016-11-18 04:22:59.000000000","message":"Yeah, nova-net will read limits from API database and count locally in cell database as I mentioned in an earlier comment.","commit_id":"ee696e984abfa20498e5c478a88b1f514fc67755"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"1e1463b6de8d27f78e8393f2cbef139c48a96f2a","unresolved":false,"context_lines":[{"line_number":218,"context_line":"References"},{"line_number":219,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"},{"line_number":220,"context_line":""},{"line_number":221,"context_line":"None"},{"line_number":222,"context_line":""},{"line_number":223,"context_line":""},{"line_number":224,"context_line":"History"}],"source_content_type":"text/x-rst","patch_set":5,"id":"7a77a97e_980120b2","line":221,"updated":"2016-11-18 01:56:14.000000000","message":"https://etherpad.openstack.org/p/ocata-nova-summit-cellsv2-quotas if that helps, but maybe not.","commit_id":"ee696e984abfa20498e5c478a88b1f514fc67755"},{"author":{"_account_id":4690,"name":"melanie witt","display_name":"melwitt","email":"melwittt@gmail.com","username":"melwitt"},"change_message_id":"c47b6f85375a64417018d7b7ccd96abf6f260a88","unresolved":false,"context_lines":[{"line_number":218,"context_line":"References"},{"line_number":219,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"},{"line_number":220,"context_line":""},{"line_number":221,"context_line":"None"},{"line_number":222,"context_line":""},{"line_number":223,"context_line":""},{"line_number":224,"context_line":"History"}],"source_content_type":"text/x-rst","patch_set":5,"id":"7a77a97e_a9012c08","line":221,"in_reply_to":"7a77a97e_980120b2","updated":"2016-11-18 04:22:59.000000000","message":"Sure, can add that.","commit_id":"ee696e984abfa20498e5c478a88b1f514fc67755"}]}