)]}'
{"/COMMIT_MSG":[{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"43f904d922c3953a51ad32c59f64c22e3b983571","unresolved":false,"context_lines":[{"line_number":6,"context_line":""},{"line_number":7,"context_line":"Add online migration to move quotas to API database"},{"line_number":8,"context_line":""},{"line_number":9,"context_line":"This moves quota limits and classes from the main database to the"},{"line_number":10,"context_line":"API database in an online migration."},{"line_number":11,"context_line":""},{"line_number":12,"context_line":"Part of blueprint cells-quota-api-db"}],"source_content_type":"text/x-gerrit-commit-message","patch_set":59,"id":"ff346bd7_faefebe6","line":9,"range":{"start_line":9,"start_character":45,"end_line":9,"end_character":58},"updated":"2017-07-23 13:52:06.000000000","message":"cell databases? Since I\u0027m assuming this is supporting migrating from multiple cell databases, but I\u0027ve just started the review so surprises might await.\n\n(later)\n\nOh I guess this is not multi-cell. So we just don\u0027t claim multi-cell support until the quotas are in the API DB, which is also how the fleetify devstack patch works (it depends on this series), so OK.\n\nWe\u0027ll need to remember in the release notes (and devref) about multi-cell that you need to have this migration done first before deploying multiple cells.","commit_id":"8cb4c687cbda76508941577d83161395b3b0f30d"}],"nova/objects/quotas.py":[{"author":{"_account_id":4393,"name":"Dan Smith","email":"dms@danplanet.com","username":"danms"},"change_message_id":"1697870bfb747e3d1f47ae97cd389f93677c7786","unresolved":false,"context_lines":[{"line_number":464,"context_line":"    # Migrate per project limits"},{"line_number":465,"context_line":"    main_per_project_limits \u003d _get_main_per_project_limits(context, count)"},{"line_number":466,"context_line":"    done \u003d 0"},{"line_number":467,"context_line":"    for db_limit in main_per_project_limits:"},{"line_number":468,"context_line":"        try:"},{"line_number":469,"context_line":"            Quotas._create_limit_in_db(context, db_limit.project_id,"},{"line_number":470,"context_line":"                                       db_limit.resource, db_limit.hard_limit)"}],"source_content_type":"text/x-python","patch_set":6,"id":"9a57fde8_30ed348c","line":467,"updated":"2017-01-12 15:11:32.000000000","message":"This will create some visible limits in the api db before they\u0027re all done, right? If we did this inside a transaction, I think it\u0027d be safer, and we\u0027d roll them all back if we fail to insert one. With what you have here, I think that if we fail due to network or something in the middle of this loop, we end up with half of them in the api database, and all of them in the main database. I guess re-running will clean that up because it will ignore dupes, but certainly the api will behave differently for a bit as some limits will be missing.\n\nNot sure how much of a deal it really is, but thought I\u0027d bring it up here to maybe see if others agree and/or see if you decided to do it this way because the other is hard or impossible.","commit_id":"6413c2978392161dd74d97519652e5b28c871981"},{"author":{"_account_id":782,"name":"John Garbutt","email":"john@johngarbutt.com","username":"johngarbutt"},"change_message_id":"53742d605d523dcaeb5843f0fc3f203ef8196fb8","unresolved":false,"context_lines":[{"line_number":469,"context_line":"def _create_limits_in_api_db(context, db_limits, per_user\u003dFalse):"},{"line_number":470,"context_line":"    for db_limit in db_limits:"},{"line_number":471,"context_line":"        user_id \u003d db_limit.user_id if per_user else None"},{"line_number":472,"context_line":"        Quotas._create_limit_in_db(context, db_limit.project_id,"},{"line_number":473,"context_line":"                                   db_limit.resource, db_limit.hard_limit,"},{"line_number":474,"context_line":"                                   user_id\u003duser_id)"},{"line_number":475,"context_line":""}],"source_content_type":"text/x-python","patch_set":7,"id":"5a3905b3_cb8a28bf","line":472,"updated":"2017-01-20 16:59:32.000000000","message":"I totally think we could put these all in the same table in the API DB, I would lean towards the Quota table with a nullable user_id field.","commit_id":"fba150056d2b7825e585c5ed22f97df1926cb977"},{"author":{"_account_id":782,"name":"John Garbutt","email":"john@johngarbutt.com","username":"johngarbutt"},"change_message_id":"e21d9921151b87dca56a2acc2f91fc077ab269c5","unresolved":false,"context_lines":[{"line_number":482,"context_line":"    _create_limits_in_api_db(context, main_per_project_limits)"},{"line_number":483,"context_line":"    # Delete the limits separately."},{"line_number":484,"context_line":"    for db_limit in main_per_project_limits:"},{"line_number":485,"context_line":"        _destroy_main_per_project_limits(context, db_limit.project_id,"},{"line_number":486,"context_line":"                                         db_limit.resource)"},{"line_number":487,"context_line":"        done +\u003d 1"},{"line_number":488,"context_line":"    if done \u003d\u003d count:"}],"source_content_type":"text/x-python","patch_set":7,"id":"5a3905b3_cb21484f","line":485,"updated":"2017-01-20 16:57:20.000000000","message":"I don\u0027t think deals with dan\u0027s concerns about doing this in a transaction as such.\n\nBut, given the other code in place here, I think the key thing is that this eventually tidies everything up, even it if gets interrupted half way for some reason.\n\nAny chance we could test that actually? Maybe make _destroy_main_per_project_limits fail, then call it again without the mock to check the tidy up completes OK?\n\nRight now I think QuotaExists exceptions are thrown by _create_limits_in_api_db if we have a failure during the destroy phase.","commit_id":"fba150056d2b7825e585c5ed22f97df1926cb977"},{"author":{"_account_id":4393,"name":"Dan Smith","email":"dms@danplanet.com","username":"danms"},"change_message_id":"19b367ffa899d4a453e92b3e6946290867ed8d4e","unresolved":false,"context_lines":[{"line_number":482,"context_line":"    _create_limits_in_api_db(context, main_per_project_limits)"},{"line_number":483,"context_line":"    # Delete the limits separately."},{"line_number":484,"context_line":"    for db_limit in main_per_project_limits:"},{"line_number":485,"context_line":"        _destroy_main_per_project_limits(context, db_limit.project_id,"},{"line_number":486,"context_line":"                                         db_limit.resource)"},{"line_number":487,"context_line":"        done +\u003d 1"},{"line_number":488,"context_line":"    if done \u003d\u003d count:"}],"source_content_type":"text/x-python","patch_set":7,"id":"3a461143_64a3809f","line":485,"in_reply_to":"5a3905b3_cb21484f","updated":"2017-01-25 18:54:41.000000000","message":"I think it does because it creates all the things or it doesn\u0027t. Because of the way we favor the api database in the object layer, it doesn\u0027t matter if they\u0027re in both places, and we\u0027ll eventually delete from the main when this runs again. I just think it\u0027s less good if we create each limit in a separate transaction and end up with half of them in the api db.","commit_id":"fba150056d2b7825e585c5ed22f97df1926cb977"},{"author":{"_account_id":782,"name":"John Garbutt","email":"john@johngarbutt.com","username":"johngarbutt"},"change_message_id":"e21d9921151b87dca56a2acc2f91fc077ab269c5","unresolved":false,"context_lines":[{"line_number":533,"context_line":"    _create_classes_in_api_db(context, main_quota_classes)"},{"line_number":534,"context_line":"    # Delete the classes separately."},{"line_number":535,"context_line":"    for db_class in main_quota_classes:"},{"line_number":536,"context_line":"        _destroy_main_quota_class(context, db_class.class_name,"},{"line_number":537,"context_line":"                                  db_class.resource, db_class.hard_limit)"},{"line_number":538,"context_line":"        done +\u003d 1"},{"line_number":539,"context_line":"    return len(main_quota_classes), done"}],"source_content_type":"text/x-python","patch_set":7,"id":"5a3905b3_cbf94882","line":536,"updated":"2017-01-20 16:57:20.000000000","message":"Honestly, I think everyone\u0027s DB is empty for this table right now, but I think we should consider if one of the destroy operations fail, that we fix everything up nicely.","commit_id":"fba150056d2b7825e585c5ed22f97df1926cb977"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"43f904d922c3953a51ad32c59f64c22e3b983571","unresolved":false,"context_lines":[{"line_number":156,"context_line":"    @db_api.api_context_manager.reader"},{"line_number":157,"context_line":"    def _get_class_from_db(context, class_name, resource):"},{"line_number":158,"context_line":"        result \u003d context.session.query(api_models.QuotaClass).\\"},{"line_number":159,"context_line":"                        filter_by(class_name\u003dclass_name).\\"},{"line_number":160,"context_line":"                        filter_by(resource\u003dresource).\\"},{"line_number":161,"context_line":"                        first()"},{"line_number":162,"context_line":"        if not result:"},{"line_number":163,"context_line":"            raise exception.QuotaClassNotFound(class_name\u003dclass_name)"}],"source_content_type":"text/x-python","patch_set":59,"id":"ff346bd7_7a0d3b8f","line":160,"range":{"start_line":159,"start_character":24,"end_line":160,"end_character":54},"updated":"2017-07-23 13:52:06.000000000","message":"I guess this is based on quota_class_update from the old DB API.","commit_id":"8cb4c687cbda76508941577d83161395b3b0f30d"},{"author":{"_account_id":4690,"name":"melanie witt","display_name":"melwitt","email":"melwittt@gmail.com","username":"melwitt"},"change_message_id":"8e9932a9c85303e61a10c07382e7671bf78283bd","unresolved":false,"context_lines":[{"line_number":156,"context_line":"    @db_api.api_context_manager.reader"},{"line_number":157,"context_line":"    def _get_class_from_db(context, class_name, resource):"},{"line_number":158,"context_line":"        result \u003d context.session.query(api_models.QuotaClass).\\"},{"line_number":159,"context_line":"                        filter_by(class_name\u003dclass_name).\\"},{"line_number":160,"context_line":"                        filter_by(resource\u003dresource).\\"},{"line_number":161,"context_line":"                        first()"},{"line_number":162,"context_line":"        if not result:"},{"line_number":163,"context_line":"            raise exception.QuotaClassNotFound(class_name\u003dclass_name)"}],"source_content_type":"text/x-python","patch_set":59,"id":"ff346bd7_60a2259d","line":160,"range":{"start_line":159,"start_character":24,"end_line":160,"end_character":54},"in_reply_to":"ff346bd7_7a0d3b8f","updated":"2017-07-24 00:59:13.000000000","message":"It\u0027s based on quota_class_get from the old DB API.","commit_id":"8cb4c687cbda76508941577d83161395b3b0f30d"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"43f904d922c3953a51ad32c59f64c22e3b983571","unresolved":false,"context_lines":[{"line_number":222,"context_line":""},{"line_number":223,"context_line":"    @staticmethod"},{"line_number":224,"context_line":"    @db_api.api_context_manager.writer"},{"line_number":225,"context_line":"    def _create_class_in_db(context, class_name, resource, limit):"},{"line_number":226,"context_line":"        # NOTE(melwitt): There\u0027s no unique constraint on the QuotaClass model,"},{"line_number":227,"context_line":"        # so check for duplicate manually."},{"line_number":228,"context_line":"        try:"}],"source_content_type":"text/x-python","patch_set":59,"id":"ff346bd7_3a3e9329","line":225,"range":{"start_line":225,"start_character":8,"end_line":225,"end_character":27},"updated":"2017-07-23 13:52:06.000000000","message":"This could have been a better name as _create_class_entry_in_db since it\u0027s a single row in the quota_classes table, not the entire thing.","commit_id":"8cb4c687cbda76508941577d83161395b3b0f30d"},{"author":{"_account_id":4690,"name":"melanie witt","display_name":"melwitt","email":"melwittt@gmail.com","username":"melwitt"},"change_message_id":"42cef2a6817610074ba3d77d02601198f1092d7c","unresolved":false,"context_lines":[{"line_number":222,"context_line":""},{"line_number":223,"context_line":"    @staticmethod"},{"line_number":224,"context_line":"    @db_api.api_context_manager.writer"},{"line_number":225,"context_line":"    def _create_class_in_db(context, class_name, resource, limit):"},{"line_number":226,"context_line":"        # NOTE(melwitt): There\u0027s no unique constraint on the QuotaClass model,"},{"line_number":227,"context_line":"        # so check for duplicate manually."},{"line_number":228,"context_line":"        try:"}],"source_content_type":"text/x-python","patch_set":59,"id":"ff346bd7_9140bd09","line":225,"range":{"start_line":225,"start_character":8,"end_line":225,"end_character":27},"in_reply_to":"ff346bd7_00a8f91c","updated":"2017-07-24 04:34:42.000000000","message":"Yeah, agreed on that.","commit_id":"8cb4c687cbda76508941577d83161395b3b0f30d"},{"author":{"_account_id":4690,"name":"melanie witt","display_name":"melwitt","email":"melwittt@gmail.com","username":"melwitt"},"change_message_id":"8e9932a9c85303e61a10c07382e7671bf78283bd","unresolved":false,"context_lines":[{"line_number":222,"context_line":""},{"line_number":223,"context_line":"    @staticmethod"},{"line_number":224,"context_line":"    @db_api.api_context_manager.writer"},{"line_number":225,"context_line":"    def _create_class_in_db(context, class_name, resource, limit):"},{"line_number":226,"context_line":"        # NOTE(melwitt): There\u0027s no unique constraint on the QuotaClass model,"},{"line_number":227,"context_line":"        # so check for duplicate manually."},{"line_number":228,"context_line":"        try:"}],"source_content_type":"text/x-python","patch_set":59,"id":"ff346bd7_e08e1526","line":225,"range":{"start_line":225,"start_character":8,"end_line":225,"end_character":27},"in_reply_to":"ff346bd7_3a3e9329","updated":"2017-07-24 00:59:13.000000000","message":"I think I don\u0027t understand what you mean here. I thought a single row in the quota_classes table is a \"class\" and this is creating a \"class\" i.e. one row. What do you mean by \"the entire thing?\"\n\n(later)\n\nI think I understand now. One quota classes will probably be represented by multiple rows, one for each resource.","commit_id":"8cb4c687cbda76508941577d83161395b3b0f30d"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"b74018b299c9c0436f0c1c3fcdb189aab0f8e18e","unresolved":false,"context_lines":[{"line_number":222,"context_line":""},{"line_number":223,"context_line":"    @staticmethod"},{"line_number":224,"context_line":"    @db_api.api_context_manager.writer"},{"line_number":225,"context_line":"    def _create_class_in_db(context, class_name, resource, limit):"},{"line_number":226,"context_line":"        # NOTE(melwitt): There\u0027s no unique constraint on the QuotaClass model,"},{"line_number":227,"context_line":"        # so check for duplicate manually."},{"line_number":228,"context_line":"        try:"}],"source_content_type":"text/x-python","patch_set":59,"id":"ff346bd7_00a8f91c","line":225,"range":{"start_line":225,"start_character":8,"end_line":225,"end_character":27},"in_reply_to":"ff346bd7_e08e1526","updated":"2017-07-24 01:15:15.000000000","message":"Right, in my mind the class is the set of resource limits for the same class name, the entries are the resource records within that class name. Ideally we\u0027d have a unique constraint on the class name + resource since it doesn\u0027t really make sense to have multiple rows for the same class name and resource but with different limits. But that\u0027s not really something we care about here.","commit_id":"8cb4c687cbda76508941577d83161395b3b0f30d"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"43f904d922c3953a51ad32c59f64c22e3b983571","unresolved":false,"context_lines":[{"line_number":548,"context_line":""},{"line_number":549,"context_line":"@db_api.require_context"},{"line_number":550,"context_line":"@db_api.pick_context_manager_writer"},{"line_number":551,"context_line":"def _destroy_main_per_user_limits(context, project_id, resource, user_id):"},{"line_number":552,"context_line":"    context.session.query(main_models.ProjectUserQuota).\\"},{"line_number":553,"context_line":"        filter_by(deleted\u003d0).\\"},{"line_number":554,"context_line":"        filter_by(project_id\u003dproject_id).\\"}],"source_content_type":"text/x-python","patch_set":59,"id":"ff346bd7_ba896327","line":551,"range":{"start_line":551,"start_character":43,"end_line":551,"end_character":72},"updated":"2017-07-23 13:52:06.000000000","message":"kind of weird to have project_id and user_id args not next to each other like they usually always are, but that\u0027s not worth changing here","commit_id":"8cb4c687cbda76508941577d83161395b3b0f30d"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"b74018b299c9c0436f0c1c3fcdb189aab0f8e18e","unresolved":false,"context_lines":[{"line_number":548,"context_line":""},{"line_number":549,"context_line":"@db_api.require_context"},{"line_number":550,"context_line":"@db_api.pick_context_manager_writer"},{"line_number":551,"context_line":"def _destroy_main_per_user_limits(context, project_id, resource, user_id):"},{"line_number":552,"context_line":"    context.session.query(main_models.ProjectUserQuota).\\"},{"line_number":553,"context_line":"        filter_by(deleted\u003d0).\\"},{"line_number":554,"context_line":"        filter_by(project_id\u003dproject_id).\\"}],"source_content_type":"text/x-python","patch_set":59,"id":"ff346bd7_e0629516","line":551,"range":{"start_line":551,"start_character":43,"end_line":551,"end_character":72},"in_reply_to":"ff346bd7_00a9197a","updated":"2017-07-24 01:15:15.000000000","message":"Yeah it\u0027s fine.","commit_id":"8cb4c687cbda76508941577d83161395b3b0f30d"},{"author":{"_account_id":4690,"name":"melanie witt","display_name":"melwitt","email":"melwittt@gmail.com","username":"melwitt"},"change_message_id":"8e9932a9c85303e61a10c07382e7671bf78283bd","unresolved":false,"context_lines":[{"line_number":548,"context_line":""},{"line_number":549,"context_line":"@db_api.require_context"},{"line_number":550,"context_line":"@db_api.pick_context_manager_writer"},{"line_number":551,"context_line":"def _destroy_main_per_user_limits(context, project_id, resource, user_id):"},{"line_number":552,"context_line":"    context.session.query(main_models.ProjectUserQuota).\\"},{"line_number":553,"context_line":"        filter_by(deleted\u003d0).\\"},{"line_number":554,"context_line":"        filter_by(project_id\u003dproject_id).\\"}],"source_content_type":"text/x-python","patch_set":59,"id":"ff346bd7_00a9197a","line":551,"range":{"start_line":551,"start_character":43,"end_line":551,"end_character":72},"in_reply_to":"ff346bd7_ba896327","updated":"2017-07-24 00:59:13.000000000","message":"Not sure why I did that other than the make the signatures of _destroy_main_per_project_limits and _destroy_main_per_user_limits match except for the one way in which they\u0027re different (user_id).","commit_id":"8cb4c687cbda76508941577d83161395b3b0f30d"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"d21fe69e04cd6ef79339cc069123e14cffdd6918","unresolved":false,"context_lines":[{"line_number":586,"context_line":"    if done \u003d\u003d count:"},{"line_number":587,"context_line":"        return len(main_per_project_limits), done"},{"line_number":588,"context_line":"    # Migrate per user limits"},{"line_number":589,"context_line":"    count -\u003d done"},{"line_number":590,"context_line":"    main_per_user_limits \u003d _get_main_per_user_limits(context, count)"},{"line_number":591,"context_line":"    try:"},{"line_number":592,"context_line":"        # Create all the limits in a single transaction."}],"source_content_type":"text/x-python","patch_set":59,"id":"ff346bd7_3a88d3c4","line":589,"range":{"start_line":589,"start_character":4,"end_line":589,"end_character":17},"updated":"2017-07-23 13:55:00.000000000","message":"Do you have test coverage where we migrate per-project limits and still have room to migrate per-user limits?","commit_id":"8cb4c687cbda76508941577d83161395b3b0f30d"},{"author":{"_account_id":4690,"name":"melanie witt","display_name":"melwitt","email":"melwittt@gmail.com","username":"melwitt"},"change_message_id":"8e9932a9c85303e61a10c07382e7671bf78283bd","unresolved":false,"context_lines":[{"line_number":586,"context_line":"    if done \u003d\u003d count:"},{"line_number":587,"context_line":"        return len(main_per_project_limits), done"},{"line_number":588,"context_line":"    # Migrate per user limits"},{"line_number":589,"context_line":"    count -\u003d done"},{"line_number":590,"context_line":"    main_per_user_limits \u003d _get_main_per_user_limits(context, count)"},{"line_number":591,"context_line":"    try:"},{"line_number":592,"context_line":"        # Create all the limits in a single transaction."}],"source_content_type":"text/x-python","patch_set":59,"id":"ff346bd7_e0dcd519","line":589,"range":{"start_line":589,"start_character":4,"end_line":589,"end_character":17},"in_reply_to":"ff346bd7_3a88d3c4","updated":"2017-07-24 00:59:13.000000000","message":"Looks like not. In test_migrate_quota_limits it migrates two per user limits first and then migrates a per project limit separately afterward.","commit_id":"8cb4c687cbda76508941577d83161395b3b0f30d"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"43f904d922c3953a51ad32c59f64c22e3b983571","unresolved":false,"context_lines":[{"line_number":604,"context_line":"    return len(main_per_project_limits) + len(main_per_user_limits), done"},{"line_number":605,"context_line":""},{"line_number":606,"context_line":""},{"line_number":607,"context_line":"@db_api.require_context"},{"line_number":608,"context_line":"@db_api.pick_context_manager_reader"},{"line_number":609,"context_line":"def _get_main_quota_classes(context, limit):"},{"line_number":610,"context_line":"    return context.session.query(main_models.QuotaClass).\\"}],"source_content_type":"text/x-python","patch_set":59,"id":"ff346bd7_fac48b42","line":607,"range":{"start_line":607,"start_character":0,"end_line":607,"end_character":23},"updated":"2017-07-23 13:52:06.000000000","message":"Why is this sometimes used and sometimes not? Like it\u0027s used on _destroy_main_per_project_limits and the other one above, but not _destroy_main_quota_class.","commit_id":"8cb4c687cbda76508941577d83161395b3b0f30d"},{"author":{"_account_id":4690,"name":"melanie witt","display_name":"melwitt","email":"melwittt@gmail.com","username":"melwitt"},"change_message_id":"8e9932a9c85303e61a10c07382e7671bf78283bd","unresolved":false,"context_lines":[{"line_number":604,"context_line":"    return len(main_per_project_limits) + len(main_per_user_limits), done"},{"line_number":605,"context_line":""},{"line_number":606,"context_line":""},{"line_number":607,"context_line":"@db_api.require_context"},{"line_number":608,"context_line":"@db_api.pick_context_manager_reader"},{"line_number":609,"context_line":"def _get_main_quota_classes(context, limit):"},{"line_number":610,"context_line":"    return context.session.query(main_models.QuotaClass).\\"}],"source_content_type":"text/x-python","patch_set":59,"id":"ff346bd7_c0e1d1d2","line":607,"range":{"start_line":607,"start_character":0,"end_line":607,"end_character":23},"in_reply_to":"ff346bd7_fac48b42","updated":"2017-07-24 00:59:13.000000000","message":"I don\u0027t know. I tried to keep parity with what\u0027s in the old DB API, in case it was necessary to keep the behavior consistent after the move.","commit_id":"8cb4c687cbda76508941577d83161395b3b0f30d"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"43f904d922c3953a51ad32c59f64c22e3b983571","unresolved":false,"context_lines":[{"line_number":618,"context_line":"    return context.session.query(main_models.QuotaClass).\\"},{"line_number":619,"context_line":"        filter_by(deleted\u003d0).\\"},{"line_number":620,"context_line":"        filter_by(class_name\u003dclass_name).\\"},{"line_number":621,"context_line":"        filter_by(resource\u003dresource).\\"},{"line_number":622,"context_line":"        filter_by(hard_limit\u003dlimit).\\"},{"line_number":623,"context_line":"        soft_delete(synchronize_session\u003dFalse)"},{"line_number":624,"context_line":""}],"source_content_type":"text/x-python","patch_set":59,"id":"ff346bd7_daef87ae","line":621,"range":{"start_line":621,"start_character":8,"end_line":621,"end_character":38},"updated":"2017-07-23 13:52:06.000000000","message":"Why are we filtering on resource here? Can\u0027t we just delete the quota class by name, which is essentially it\u0027s unique key (even though we don\u0027t have a unique constraint on it?).","commit_id":"8cb4c687cbda76508941577d83161395b3b0f30d"},{"author":{"_account_id":4690,"name":"melanie witt","display_name":"melwitt","email":"melwittt@gmail.com","username":"melwitt"},"change_message_id":"9e31c8e5071e464221fd08e3cef3eda64acb4a7d","unresolved":false,"context_lines":[{"line_number":618,"context_line":"    return context.session.query(main_models.QuotaClass).\\"},{"line_number":619,"context_line":"        filter_by(deleted\u003d0).\\"},{"line_number":620,"context_line":"        filter_by(class_name\u003dclass_name).\\"},{"line_number":621,"context_line":"        filter_by(resource\u003dresource).\\"},{"line_number":622,"context_line":"        filter_by(hard_limit\u003dlimit).\\"},{"line_number":623,"context_line":"        soft_delete(synchronize_session\u003dFalse)"},{"line_number":624,"context_line":""}],"source_content_type":"text/x-python","patch_set":59,"id":"ff346bd7_3fbd49b7","line":621,"range":{"start_line":621,"start_character":8,"end_line":621,"end_character":38},"in_reply_to":"ff346bd7_1154cdc9","updated":"2017-07-24 15:48:40.000000000","message":"Okay, now that I went to go do this, I see why it was done one-by-one. Because when you do the migration, you can pass \"count\" to limit how many are migrated at the same time. So not all the quota classes may have been migrated, maybe only a partial set of them. So we should delete them by id.","commit_id":"8cb4c687cbda76508941577d83161395b3b0f30d"},{"author":{"_account_id":4690,"name":"melanie witt","display_name":"melwitt","email":"melwittt@gmail.com","username":"melwitt"},"change_message_id":"42cef2a6817610074ba3d77d02601198f1092d7c","unresolved":false,"context_lines":[{"line_number":618,"context_line":"    return context.session.query(main_models.QuotaClass).\\"},{"line_number":619,"context_line":"        filter_by(deleted\u003d0).\\"},{"line_number":620,"context_line":"        filter_by(class_name\u003dclass_name).\\"},{"line_number":621,"context_line":"        filter_by(resource\u003dresource).\\"},{"line_number":622,"context_line":"        filter_by(hard_limit\u003dlimit).\\"},{"line_number":623,"context_line":"        soft_delete(synchronize_session\u003dFalse)"},{"line_number":624,"context_line":""}],"source_content_type":"text/x-python","patch_set":59,"id":"ff346bd7_1154cdc9","line":621,"range":{"start_line":621,"start_character":8,"end_line":621,"end_character":38},"in_reply_to":"ff346bd7_a03a4de4","updated":"2017-07-24 04:34:42.000000000","message":"In the first version of this, the \"create in API DB\" were done one-by-one and so these were deleted one-by-one. But now they\u0027re created in the same transaction, so it doesn\u0027t seem like the delete needs to be one-by-one anymore. I can change it to a delete all at the same time and also add the missing test coverage.","commit_id":"8cb4c687cbda76508941577d83161395b3b0f30d"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"b74018b299c9c0436f0c1c3fcdb189aab0f8e18e","unresolved":false,"context_lines":[{"line_number":618,"context_line":"    return context.session.query(main_models.QuotaClass).\\"},{"line_number":619,"context_line":"        filter_by(deleted\u003d0).\\"},{"line_number":620,"context_line":"        filter_by(class_name\u003dclass_name).\\"},{"line_number":621,"context_line":"        filter_by(resource\u003dresource).\\"},{"line_number":622,"context_line":"        filter_by(hard_limit\u003dlimit).\\"},{"line_number":623,"context_line":"        soft_delete(synchronize_session\u003dFalse)"},{"line_number":624,"context_line":""}],"source_content_type":"text/x-python","patch_set":59,"id":"ff346bd7_a03a4de4","line":621,"range":{"start_line":621,"start_character":8,"end_line":621,"end_character":38},"in_reply_to":"ff346bd7_a0ffed65","updated":"2017-07-24 01:15:15.000000000","message":"I thought about this later and was thinking maybe you were doing it row by row to match the counting stuff exactly, so deleting all records for a single class wouldn\u0027t make sense with how the online data migrations work if you\u0027re migrating the records row by row, especially if the ones fetched are different from the ones deleted, order-wise, although I don\u0027t think that would probably happen (the order should be consistent).\n\nIf we are going to delete one by one, we could just do it by id rather than name/resource/hard_limit, since the id would be unique.","commit_id":"8cb4c687cbda76508941577d83161395b3b0f30d"},{"author":{"_account_id":4690,"name":"melanie witt","display_name":"melwitt","email":"melwittt@gmail.com","username":"melwitt"},"change_message_id":"8e9932a9c85303e61a10c07382e7671bf78283bd","unresolved":false,"context_lines":[{"line_number":618,"context_line":"    return context.session.query(main_models.QuotaClass).\\"},{"line_number":619,"context_line":"        filter_by(deleted\u003d0).\\"},{"line_number":620,"context_line":"        filter_by(class_name\u003dclass_name).\\"},{"line_number":621,"context_line":"        filter_by(resource\u003dresource).\\"},{"line_number":622,"context_line":"        filter_by(hard_limit\u003dlimit).\\"},{"line_number":623,"context_line":"        soft_delete(synchronize_session\u003dFalse)"},{"line_number":624,"context_line":""}],"source_content_type":"text/x-python","patch_set":59,"id":"ff346bd7_a0ffed65","line":621,"range":{"start_line":621,"start_character":8,"end_line":621,"end_character":38},"in_reply_to":"ff346bd7_daef87ae","updated":"2017-07-24 00:59:13.000000000","message":"I guess the way this is currently working, _get_main_quota_classes() returns all of the rows in the main DB, and for each row in the main DB, we\u0027re deleting them one-by-one, filtering by each row-identifying thing.\n\nIn earlier versions of this, we weren\u0027t creating all of the quota classes in the API DB in a single transaction, but now we are. So really, could or should we just delete them all in one go? Like:\n\n context.session.query(main_models.QuotaClass).\\\n        filter_by(deleted\u003d0).\\\n        limit(limit).\\\n        soft_delete(synchronize_session\u003dFalse)\n\n?","commit_id":"8cb4c687cbda76508941577d83161395b3b0f30d"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"43f904d922c3953a51ad32c59f64c22e3b983571","unresolved":false,"context_lines":[{"line_number":619,"context_line":"        filter_by(deleted\u003d0).\\"},{"line_number":620,"context_line":"        filter_by(class_name\u003dclass_name).\\"},{"line_number":621,"context_line":"        filter_by(resource\u003dresource).\\"},{"line_number":622,"context_line":"        filter_by(hard_limit\u003dlimit).\\"},{"line_number":623,"context_line":"        soft_delete(synchronize_session\u003dFalse)"},{"line_number":624,"context_line":""},{"line_number":625,"context_line":""}],"source_content_type":"text/x-python","patch_set":59,"id":"ff346bd7_ba2a83e4","line":622,"range":{"start_line":622,"start_character":8,"end_line":622,"end_character":37},"updated":"2017-07-23 13:52:06.000000000","message":"same, this seems unnecessary when we can just delete the class by name","commit_id":"8cb4c687cbda76508941577d83161395b3b0f30d"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"43f904d922c3953a51ad32c59f64c22e3b983571","unresolved":false,"context_lines":[{"line_number":626,"context_line":"@db_api.api_context_manager.writer"},{"line_number":627,"context_line":"def _create_classes_in_api_db(context, db_classes):"},{"line_number":628,"context_line":"    for db_class in db_classes:"},{"line_number":629,"context_line":"        Quotas._create_class_in_db(context, db_class.class_name,"},{"line_number":630,"context_line":"                                   db_class.resource, db_class.hard_limit)"},{"line_number":631,"context_line":""},{"line_number":632,"context_line":""}],"source_content_type":"text/x-python","patch_set":59,"id":"ff346bd7_5a04b77b","line":629,"range":{"start_line":629,"start_character":15,"end_line":629,"end_character":34},"updated":"2017-07-23 13:52:06.000000000","message":"Well this is kind of not fun since it\u0027s going to get a \u0027class exists\u0027 for every resource but the first one, but it\u0027s a one time thing so no biggy, and it\u0027s only one class (default).","commit_id":"8cb4c687cbda76508941577d83161395b3b0f30d"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"b74018b299c9c0436f0c1c3fcdb189aab0f8e18e","unresolved":false,"context_lines":[{"line_number":626,"context_line":"@db_api.api_context_manager.writer"},{"line_number":627,"context_line":"def _create_classes_in_api_db(context, db_classes):"},{"line_number":628,"context_line":"    for db_class in db_classes:"},{"line_number":629,"context_line":"        Quotas._create_class_in_db(context, db_class.class_name,"},{"line_number":630,"context_line":"                                   db_class.resource, db_class.hard_limit)"},{"line_number":631,"context_line":""},{"line_number":632,"context_line":""}],"source_content_type":"text/x-python","patch_set":59,"id":"ff346bd7_002b5994","line":629,"range":{"start_line":629,"start_character":15,"end_line":629,"end_character":34},"in_reply_to":"ff346bd7_001eb9b9","updated":"2017-07-24 01:15:15.000000000","message":"Maybe I wasn\u0027t looking at the new code correctly, I was thinking about how this worked in the API with the old DB APIs:\n\nhttps://github.com/openstack/nova/blob/master/nova/api/openstack/compute/quota_classes.py#L95-L99","commit_id":"8cb4c687cbda76508941577d83161395b3b0f30d"},{"author":{"_account_id":4690,"name":"melanie witt","display_name":"melwitt","email":"melwittt@gmail.com","username":"melwitt"},"change_message_id":"42cef2a6817610074ba3d77d02601198f1092d7c","unresolved":false,"context_lines":[{"line_number":626,"context_line":"@db_api.api_context_manager.writer"},{"line_number":627,"context_line":"def _create_classes_in_api_db(context, db_classes):"},{"line_number":628,"context_line":"    for db_class in db_classes:"},{"line_number":629,"context_line":"        Quotas._create_class_in_db(context, db_class.class_name,"},{"line_number":630,"context_line":"                                   db_class.resource, db_class.hard_limit)"},{"line_number":631,"context_line":""},{"line_number":632,"context_line":""}],"source_content_type":"text/x-python","patch_set":59,"id":"ff346bd7_d12d2547","line":629,"range":{"start_line":629,"start_character":15,"end_line":629,"end_character":34},"in_reply_to":"ff346bd7_002b5994","updated":"2017-07-24 04:34:42.000000000","message":"Hm, yeah I\u0027m not sure how that\u0027s related. Maybe I\u0027m missing something. The new stuff will only raise \"class exists\" if the name + resource already exists in the API DB. So if there are dupes of quota class \"mriedem\" and \"instances\" in the main DB, they will get \"class exists\" and not be created more than once with potentially different limits. Only the first \"mriedem\" + \"instances\" combo will be accepted.","commit_id":"8cb4c687cbda76508941577d83161395b3b0f30d"},{"author":{"_account_id":4690,"name":"melanie witt","display_name":"melwitt","email":"melwittt@gmail.com","username":"melwitt"},"change_message_id":"8e9932a9c85303e61a10c07382e7671bf78283bd","unresolved":false,"context_lines":[{"line_number":626,"context_line":"@db_api.api_context_manager.writer"},{"line_number":627,"context_line":"def _create_classes_in_api_db(context, db_classes):"},{"line_number":628,"context_line":"    for db_class in db_classes:"},{"line_number":629,"context_line":"        Quotas._create_class_in_db(context, db_class.class_name,"},{"line_number":630,"context_line":"                                   db_class.resource, db_class.hard_limit)"},{"line_number":631,"context_line":""},{"line_number":632,"context_line":""}],"source_content_type":"text/x-python","patch_set":59,"id":"ff346bd7_001eb9b9","line":629,"range":{"start_line":629,"start_character":15,"end_line":629,"end_character":34},"in_reply_to":"ff346bd7_5a04b77b","updated":"2017-07-24 00:59:13.000000000","message":"Why do you say that? I have things considering a class_name+resource combo as being unique. Why would there be a \"class exists\" unless the user has duplicate class_name+resource combos?","commit_id":"8cb4c687cbda76508941577d83161395b3b0f30d"}],"nova/tests/functional/db/test_quotas.py":[{"author":{"_account_id":782,"name":"John Garbutt","email":"john@johngarbutt.com","username":"johngarbutt"},"change_message_id":"e21d9921151b87dca56a2acc2f91fc077ab269c5","unresolved":false,"context_lines":[{"line_number":237,"context_line":"                         quotas.migrate_quota_limits_to_api_db("},{"line_number":238,"context_line":"                                self.context, 100))"},{"line_number":239,"context_line":"        self.assertEqual((0, 0),"},{"line_number":240,"context_line":"                         quotas.migrate_quota_limits_to_api_db("},{"line_number":241,"context_line":"                                self.context, 100))"},{"line_number":242,"context_line":""},{"line_number":243,"context_line":"    def test_migrate_quota_classes(self):"}],"source_content_type":"text/x-python","patch_set":7,"id":"5a3905b3_8b366050","line":240,"updated":"2017-01-20 16:57:20.000000000","message":"Loving these tests, BTW.","commit_id":"fba150056d2b7825e585c5ed22f97df1926cb977"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"08b746a5f3a43a1c29143f447d25b7dc119bd920","unresolved":false,"context_lines":[{"line_number":227,"context_line":""},{"line_number":228,"context_line":"        # This only fetches from the api db. There should now be 4 limits."},{"line_number":229,"context_line":"        api_user_limits \u003d quotas.Quotas._get_all_from_db(self.context,"},{"line_number":230,"context_line":"                                                             \u0027fake-project\u0027)"},{"line_number":231,"context_line":"        api_proj_limits_dict \u003d quotas.Quotas._get_all_from_db_by_project("},{"line_number":232,"context_line":"                self.context, \u0027fake-project\u0027)"},{"line_number":233,"context_line":"        api_proj_limits_dict.pop(\u0027project_id\u0027, None)"}],"source_content_type":"text/x-python","patch_set":60,"id":"ff346bd7_32dd7314","line":230,"range":{"start_line":230,"start_character":61,"end_line":230,"end_character":75},"updated":"2017-07-24 22:31:32.000000000","message":"alignment","commit_id":"33cdf121126cd56bf4370db189bd807329972ce7"}],"releasenotes/notes/quota-limits-classes-api-db-88e5d0d2426d2dba.yaml":[{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"08b746a5f3a43a1c29143f447d25b7dc119bd920","unresolved":false,"context_lines":[{"line_number":1,"context_line":"---"},{"line_number":2,"context_line":"upgrade:"},{"line_number":3,"context_line":"  - Quota limits and classes are being moved to teh API database for Cells v2."},{"line_number":4,"context_line":"    In this release, the online data migrations will move any quota limits and"},{"line_number":5,"context_line":"    classes you have in your main database to the API database, retaining all"},{"line_number":6,"context_line":"    attributes."}],"source_content_type":"text/x-yaml","patch_set":60,"id":"ff346bd7_b2d043f9","line":3,"range":{"start_line":3,"start_character":48,"end_line":3,"end_character":51},"updated":"2017-07-24 22:31:32.000000000","message":"the","commit_id":"33cdf121126cd56bf4370db189bd807329972ce7"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"08b746a5f3a43a1c29143f447d25b7dc119bd920","unresolved":false,"context_lines":[{"line_number":4,"context_line":"    In this release, the online data migrations will move any quota limits and"},{"line_number":5,"context_line":"    classes you have in your main database to the API database, retaining all"},{"line_number":6,"context_line":"    attributes."},{"line_number":7,"context_line":"  - Note that quota limits and classes can no longer be soft-deleted as the API"},{"line_number":8,"context_line":"    database does not replicate the legacy soft-delete functionality from the"},{"line_number":9,"context_line":"    main database. As such, deleted quota limits and classes are not migrated"},{"line_number":10,"context_line":"    and the behavior users will experience will be the same as if a purge of"}],"source_content_type":"text/x-yaml","patch_set":60,"id":"ff346bd7_72dadb17","line":7,"range":{"start_line":7,"start_character":2,"end_line":7,"end_character":8},"updated":"2017-07-24 22:31:32.000000000","message":"Do you want this to be a separate release note entry? Or a .. note:: under the existing one you have above?","commit_id":"33cdf121126cd56bf4370db189bd807329972ce7"}]}