)]}'
{"cinder/service.py":[{"author":{"_account_id":9535,"name":"Gorka Eguileor","email":"geguileo@redhat.com","username":"Gorka"},"change_message_id":"c1b93f26a9721ded8d7c77d24174b54be54b473d","unresolved":false,"context_lines":[{"line_number":327,"context_line":""},{"line_number":328,"context_line":"        # NOTE(jsbryant) Other DB errors can happen in HA configurations."},{"line_number":329,"context_line":"        # such errors shouldn\u0027t kill this thread, so we handle them here."},{"line_number":330,"context_line":"        except db_exc.DBError as e:"},{"line_number":331,"context_line":"            if not getattr(self, \u0027model_disconnected\u0027, False):"},{"line_number":332,"context_line":"                self.model_disconnected \u003d True"},{"line_number":333,"context_line":"                LOG.exception(_LE(\u0027DBError encountered: %s\u0027), e)"}],"source_content_type":"text/x-python","patch_set":1,"id":"fa32b979_a54f2fbd","line":330,"updated":"2015-06-22 18:31:26.000000000","message":"We should filter on specific errors and re raise any other errors.\n\nOne way would be to check exception details here, but I think we should do it the same way that retry_on_deadlock works.  Creating an exception in oslo_db using its filters to group all temporary takeover exceptions on the different DBs and then create a decorator that allows either retry with delay or just silently ignore those errors and re rase others.\n\nI think this kind of solution will be useful because we\u0027ll probably have more methods that will require it.","commit_id":"3a1aed0913b37200d96f99778b00fe13ced2ec8e"},{"author":{"_account_id":7198,"name":"Jay Bryant","email":"jungleboyj@electronicjungle.net","username":"jsbryant"},"change_message_id":"6bdbc7c0a58ae94e964d592fe6ee3f3c939c9217","unresolved":false,"context_lines":[{"line_number":327,"context_line":""},{"line_number":328,"context_line":"        # NOTE(jsbryant) Other DB errors can happen in HA configurations."},{"line_number":329,"context_line":"        # such errors shouldn\u0027t kill this thread, so we handle them here."},{"line_number":330,"context_line":"        except db_exc.DBError as e:"},{"line_number":331,"context_line":"            if not getattr(self, \u0027model_disconnected\u0027, False):"},{"line_number":332,"context_line":"                self.model_disconnected \u003d True"},{"line_number":333,"context_line":"                LOG.exception(_LE(\u0027DBError encountered: %s\u0027), e)"}],"source_content_type":"text/x-python","patch_set":1,"id":"fa32b979_a59c045e","line":330,"in_reply_to":"fa32b979_a54f2fbd","updated":"2015-06-22 23:10:42.000000000","message":"Gorka,\n\nI was trying to keep this as small a patch as possible for backport into Kilo.  Would like to get an improvement in place now.  We can consider something more extensive for Liberty.\n\nIs that an acceptable compromise?\n\nJay","commit_id":"3a1aed0913b37200d96f99778b00fe13ced2ec8e"},{"author":{"_account_id":9535,"name":"Gorka Eguileor","email":"geguileo@redhat.com","username":"Gorka"},"change_message_id":"da6a46b19dc0b9825628669abf2d4e4128c483bf","unresolved":false,"context_lines":[{"line_number":327,"context_line":""},{"line_number":328,"context_line":"        # NOTE(jsbryant) Other DB errors can happen in HA configurations."},{"line_number":329,"context_line":"        # such errors shouldn\u0027t kill this thread, so we handle them here."},{"line_number":330,"context_line":"        except db_exc.DBError as e:"},{"line_number":331,"context_line":"            if not getattr(self, \u0027model_disconnected\u0027, False):"},{"line_number":332,"context_line":"                self.model_disconnected \u003d True"},{"line_number":333,"context_line":"                LOG.exception(_LE(\u0027DBError encountered: %s\u0027), e)"}],"source_content_type":"text/x-python","patch_set":1,"id":"fa32b979_7879b7fb","line":330,"in_reply_to":"fa32b979_a59c045e","updated":"2015-06-24 10:10:58.000000000","message":"Yes, it\u0027s acceptable. That\u0027s a solid reason I can get behind to keep it simple.","commit_id":"3a1aed0913b37200d96f99778b00fe13ced2ec8e"},{"author":{"_account_id":4523,"name":"Eric Harney","email":"eharney@redhat.com","username":"eharney"},"change_message_id":"90fca54474cc224066be58167f25e22d2366f252","unresolved":false,"context_lines":[{"line_number":330,"context_line":"        except db_exc.DBError as e:"},{"line_number":331,"context_line":"            if not getattr(self, \u0027model_disconnected\u0027, False):"},{"line_number":332,"context_line":"                self.model_disconnected \u003d True"},{"line_number":333,"context_line":"                LOG.exception(_LE(\u0027DBError encountered: %s\u0027), e)"},{"line_number":334,"context_line":""},{"line_number":335,"context_line":""},{"line_number":336,"context_line":"class WSGIService(object):"}],"source_content_type":"text/x-python","patch_set":1,"id":"fa32b979_bd70ada5","line":333,"updated":"2015-06-22 17:07:01.000000000","message":"I\u0027m concerned that this could lead to silent dropping of some errors we\u0027d want to see that aren\u0027t related to just the connection... what else does this cover?","commit_id":"3a1aed0913b37200d96f99778b00fe13ced2ec8e"},{"author":{"_account_id":11904,"name":"Sean McGinnis","email":"sean.mcginnis@gmail.com","username":"SeanM"},"change_message_id":"162b633be9ba28e37838b027addd4acc282e2f75","unresolved":false,"context_lines":[{"line_number":330,"context_line":"        except db_exc.DBError as e:"},{"line_number":331,"context_line":"            if not getattr(self, \u0027model_disconnected\u0027, False):"},{"line_number":332,"context_line":"                self.model_disconnected \u003d True"},{"line_number":333,"context_line":"                LOG.exception(_LE(\u0027DBError encountered: %s\u0027), e)"},{"line_number":334,"context_line":""},{"line_number":335,"context_line":""},{"line_number":336,"context_line":"class WSGIService(object):"}],"source_content_type":"text/x-python","patch_set":1,"id":"fa32b979_b20e9730","line":333,"updated":"2015-06-22 17:48:18.000000000","message":"We wouldn\u0027t want to pass in e here since this is LOG.exception. LOG.exception is just LOG.error with the added behavior that it will also dump out whatever exception is in the current scope. So either:\n\nLOG.exception(_LE(\u0027DBError encountered: \u0027)\n\nor\n\nLOG.error(_LE(\u0027DBError encountered: %s\u0027), e)","commit_id":"3a1aed0913b37200d96f99778b00fe13ced2ec8e"},{"author":{"_account_id":4523,"name":"Eric Harney","email":"eharney@redhat.com","username":"eharney"},"change_message_id":"b5ed583b9d49ceefcbf77cc108f34fc167ecf7a2","unresolved":false,"context_lines":[{"line_number":330,"context_line":"        except db_exc.DBError as e:"},{"line_number":331,"context_line":"            if not getattr(self, \u0027model_disconnected\u0027, False):"},{"line_number":332,"context_line":"                self.model_disconnected \u003d True"},{"line_number":333,"context_line":"                LOG.exception(_LE(\u0027DBError encountered: %s\u0027), e)"},{"line_number":334,"context_line":""},{"line_number":335,"context_line":""},{"line_number":336,"context_line":"class WSGIService(object):"}],"source_content_type":"text/x-python","patch_set":1,"id":"fa32b979_e80e6e8b","line":333,"in_reply_to":"fa32b979_328b6720","updated":"2015-06-22 18:52:07.000000000","message":"My worry was that DBError covers a lot of other errors that aren\u0027t related to what you\u0027re describing.\n\nHowever, since this is a periodic \"task\", I think this is ok, because this method is basically expected to log and drop exceptions.","commit_id":"3a1aed0913b37200d96f99778b00fe13ced2ec8e"},{"author":{"_account_id":7198,"name":"Jay Bryant","email":"jungleboyj@electronicjungle.net","username":"jsbryant"},"change_message_id":"6bdbc7c0a58ae94e964d592fe6ee3f3c939c9217","unresolved":false,"context_lines":[{"line_number":330,"context_line":"        except db_exc.DBError as e:"},{"line_number":331,"context_line":"            if not getattr(self, \u0027model_disconnected\u0027, False):"},{"line_number":332,"context_line":"                self.model_disconnected \u003d True"},{"line_number":333,"context_line":"                LOG.exception(_LE(\u0027DBError encountered: %s\u0027), e)"},{"line_number":334,"context_line":""},{"line_number":335,"context_line":""},{"line_number":336,"context_line":"class WSGIService(object):"}],"source_content_type":"text/x-python","patch_set":1,"id":"fa32b979_a5eac4aa","line":333,"in_reply_to":"fa32b979_b20e9730","updated":"2015-06-22 23:10:42.000000000","message":"Done","commit_id":"3a1aed0913b37200d96f99778b00fe13ced2ec8e"},{"author":{"_account_id":7198,"name":"Jay Bryant","email":"jungleboyj@electronicjungle.net","username":"jsbryant"},"change_message_id":"05fc544b6dfc1d8baaa652ffd74622e90a9c3828","unresolved":false,"context_lines":[{"line_number":330,"context_line":"        except db_exc.DBError as e:"},{"line_number":331,"context_line":"            if not getattr(self, \u0027model_disconnected\u0027, False):"},{"line_number":332,"context_line":"                self.model_disconnected \u003d True"},{"line_number":333,"context_line":"                LOG.exception(_LE(\u0027DBError encountered: %s\u0027), e)"},{"line_number":334,"context_line":""},{"line_number":335,"context_line":""},{"line_number":336,"context_line":"class WSGIService(object):"}],"source_content_type":"text/x-python","patch_set":1,"id":"fa32b979_f243bf11","line":333,"in_reply_to":"fa32b979_b20e9730","updated":"2015-06-22 18:00:37.000000000","message":"Sorry Sean.  You are right.  I will just do the LOG.exception.  I think that is the best approach.","commit_id":"3a1aed0913b37200d96f99778b00fe13ced2ec8e"},{"author":{"_account_id":7198,"name":"Jay Bryant","email":"jungleboyj@electronicjungle.net","username":"jsbryant"},"change_message_id":"05fc544b6dfc1d8baaa652ffd74622e90a9c3828","unresolved":false,"context_lines":[{"line_number":330,"context_line":"        except db_exc.DBError as e:"},{"line_number":331,"context_line":"            if not getattr(self, \u0027model_disconnected\u0027, False):"},{"line_number":332,"context_line":"                self.model_disconnected \u003d True"},{"line_number":333,"context_line":"                LOG.exception(_LE(\u0027DBError encountered: %s\u0027), e)"},{"line_number":334,"context_line":""},{"line_number":335,"context_line":""},{"line_number":336,"context_line":"class WSGIService(object):"}],"source_content_type":"text/x-python","patch_set":1,"id":"fa32b979_328b6720","line":333,"in_reply_to":"fa32b979_bd70ada5","updated":"2015-06-22 18:00:37.000000000","message":"Eric,\n\nIn the case of the failure we are seeing it is this:\n\n DBError: (Error) ibm_db_dbi::Error: [IBM][CLI Driver] SQL1776N  The command cannot be issued on an HADR database. Reason code \u003d \"1\". SQLCODE\u003d-1776 None None\n\nHere is an explanation of what that means:\nIn our HA implementation, cinder and other services access the database via a virtual IP.  The virtual IP normally resides with the DB2 HADR primary database; the HADR secondary databases do not support client connections.  If there is a failure or maintenance action that results in the virtual IP and HADR primary moving to another server, there is a period of time where DB2 on the new primary is taking over the role of primary for each of the OpenStack databases.  During that time, clients may be connecting to a database for which takeover has not yet completed, and database operations may fail with the exception you noted.  Once the takeover has completed, connections using the virtual IP work normally.\n\nSo, catching DBErrors here will catch situations like this as we are moving to an HA environment where the failure is transient, like a DBConnectionError.  In the case that the failure is not transient we are in a better place that we currently are.  Currently what happens is the polling thread throws an unhandled exceptions and exits the thread.  Services are no longer updating the database and that is not clear.\n\nWith this patch, if the error is not transient it will be put into the logs with each pass of the polling thread and hopefully make it clear as to why the state of that service hasn\u0027t been updated in the database.\n\nMake sense?","commit_id":"3a1aed0913b37200d96f99778b00fe13ced2ec8e"}]}