)]}'
{"oslo_messaging/_drivers/impl_rabbit.py":[{"author":{"_account_id":7109,"name":"Dmitry Mescheryakov (dmitryme)","email":"dmescheryakov@mirantis.com","username":"dmitryme"},"change_message_id":"e9db425c9108988d9cd99bd108345611d74b2fb3","unresolved":false,"context_lines":[{"line_number":307,"context_line":"            # condition itself can be fixed."},{"line_number":308,"context_line":"            # See https://bugs.launchpad.net/neutron/+bug/1318721 for details."},{"line_number":309,"context_line":"            if exc.code \u003d\u003d 404:"},{"line_number":310,"context_line":"                self.queue.declare()"},{"line_number":311,"context_line":"            else:"},{"line_number":312,"context_line":"                raise"},{"line_number":313,"context_line":""}],"source_content_type":"text/x-python","patch_set":1,"id":"bacf61ea_69065acf","line":310,"updated":"2016-08-04 16:29:00.000000000","message":"This place is missing \"self._declared_on \u003d conn.channel\". Actually it can be done in one place instead of two - right after try..except statement.","commit_id":"aa156ea542af395337cb09ed186ccc7444656cb9"},{"author":{"_account_id":19520,"name":"Kirill Bespalov","email":"kbespalov@mirantis.com","username":"k.bespalov"},"change_message_id":"040cab82c4b2cd9b11dbb0a6a5e6f4202919229b","unresolved":false,"context_lines":[{"line_number":307,"context_line":"            # condition itself can be fixed."},{"line_number":308,"context_line":"            # See https://bugs.launchpad.net/neutron/+bug/1318721 for details."},{"line_number":309,"context_line":"            if exc.code \u003d\u003d 404:"},{"line_number":310,"context_line":"                self.queue.declare()"},{"line_number":311,"context_line":"            else:"},{"line_number":312,"context_line":"                raise"},{"line_number":313,"context_line":""}],"source_content_type":"text/x-python","patch_set":1,"id":"bacf61ea_aaa024f5","line":310,"in_reply_to":"bacf61ea_69065acf","updated":"2016-08-05 08:03:33.000000000","message":"Done","commit_id":"aa156ea542af395337cb09ed186ccc7444656cb9"},{"author":{"_account_id":21191,"name":"shen.zhixing","email":"fooy5460@sina.com"},"change_message_id":"8a639134cd92c2f729de71a58b8fd69642fce5f0","unresolved":false,"context_lines":[{"line_number":1052,"context_line":"                for consumer, tag in self._consumers.items():"},{"line_number":1053,"context_line":"                    if tag in self._new_tags:"},{"line_number":1054,"context_line":"                        try:"},{"line_number":1055,"context_line":"                            if not consumer.is_declared(self):"},{"line_number":1056,"context_line":"                                consumer.declare(self)"},{"line_number":1057,"context_line":"                            consumer.consume(tag\u003dtag)"},{"line_number":1058,"context_line":"                            self._new_tags.remove(tag)"},{"line_number":1059,"context_line":"                        except self.connection.channel_errors as exc:"}],"source_content_type":"text/x-python","patch_set":2,"id":"9ad45d7e_76fcfe3e","line":1056,"range":{"start_line":1055,"start_character":0,"end_line":1056,"end_character":54},"updated":"2016-08-08 10:33:02.000000000","message":"I think correcting it at below \"Exception\" position is better;\nMaybe exchange is aute-deleted, we can catch it here at the first time.\n\nunfortunately, We have done nothing for this Exception.\n\nAMQPDriverBase._send\n\nif wait_for_reply:\n    result \u003d self._waiter.wait(msg_id, timeout)\n    if isinstance(result, Exception):\n        raise result\n    return result","commit_id":"c3ef60116da6dd456fd551fbbbf0ccadb5fe896d"},{"author":{"_account_id":2813,"name":"Mehdi Abaakouk","email":"sileht@sileht.net","username":"sileht"},"change_message_id":"ef4bedfef8f21fb2a84b5811e5e6f5f1d73416a9","unresolved":false,"context_lines":[{"line_number":1053,"context_line":"                    if tag in self._new_tags:"},{"line_number":1054,"context_line":"                        try:"},{"line_number":1055,"context_line":"                            if not consumer.is_declared(self):"},{"line_number":1056,"context_line":"                                consumer.declare(self)"},{"line_number":1057,"context_line":"                            consumer.consume(tag\u003dtag)"},{"line_number":1058,"context_line":"                            self._new_tags.remove(tag)"},{"line_number":1059,"context_line":"                        except self.connection.channel_errors as exc:"}],"source_content_type":"text/x-python","patch_set":2,"id":"bacf61ea_e559adfe","line":1056,"updated":"2016-09-06 09:10:28.000000000","message":"The on_reconnection kombu callback already do this. If the queue is unbound while no reconnection occurs, I\u0027m really confuse, rabbit have unbound the queue itself without notice us ? Ok that\u0027s not the first time I see this kind of race when network issue occurs...\n\nI thought the describe race was mitigated by L302-L309. But yeah the 404 can be raised on consume too.\n\nWe have recently added L1060-L1074 to hide the error from log when the race occurs by directly redeclaring the consumer instead of triggering on_reconnection and redeclare everything.\n\nMy thought is that when the 404 occurs on one consumer, we have a good chance that more than 2 queues have been unbounded. So the code that retries only twice will not redeclare all queues that have disappear but only two and then raise a exception, on_reconnection is triggered and everything is back to normal state, but deployer see ton of error.\n\nIn this end when on_reconnection is triggered everything is fine. But we want to show to deployer on thing we can\u0027t recover in an transparency manner.\n\nI have proposed a similar fix (based on yours) that should handle this kind of races more broadly: https://review.openstack.org/#/c/365959/\n\nI have added a bunch proactive check to ensure queue/exchange are binded. I would appreciate your opinion on this.","commit_id":"c3ef60116da6dd456fd551fbbbf0ccadb5fe896d"},{"author":{"_account_id":2813,"name":"Mehdi Abaakouk","email":"sileht@sileht.net","username":"sileht"},"change_message_id":"66498e40327b055507fb16a83ab427b45d6893c4","unresolved":false,"context_lines":[{"line_number":1053,"context_line":"                    if tag in self._new_tags:"},{"line_number":1054,"context_line":"                        try:"},{"line_number":1055,"context_line":"                            if not consumer.is_declared(self):"},{"line_number":1056,"context_line":"                                consumer.declare(self)"},{"line_number":1057,"context_line":"                            consumer.consume(tag\u003dtag)"},{"line_number":1058,"context_line":"                            self._new_tags.remove(tag)"},{"line_number":1059,"context_line":"                        except self.connection.channel_errors as exc:"}],"source_content_type":"text/x-python","patch_set":2,"id":"9a89bdaa_3c5a1150","line":1056,"in_reply_to":"9a89bdaa_abeee2b1","updated":"2016-09-06 12:45:21.000000000","message":"Ok I got it, thx for the detail.","commit_id":"c3ef60116da6dd456fd551fbbbf0ccadb5fe896d"},{"author":{"_account_id":19520,"name":"Kirill Bespalov","email":"kbespalov@mirantis.com","username":"k.bespalov"},"change_message_id":"dd2799c76c07b8981a799464b40c15e48276ad7c","unresolved":false,"context_lines":[{"line_number":1052,"context_line":"                for consumer, tag in self._consumers.items():"},{"line_number":1053,"context_line":"                    if tag in self._new_tags:"},{"line_number":1054,"context_line":"                        try:"},{"line_number":1055,"context_line":"                            if not consumer.is_declared(self):"},{"line_number":1056,"context_line":"                                consumer.declare(self)"},{"line_number":1057,"context_line":"                            consumer.consume(tag\u003dtag)"},{"line_number":1058,"context_line":"                            self._new_tags.remove(tag)"},{"line_number":1059,"context_line":"                        except self.connection.channel_errors as exc:"}],"source_content_type":"text/x-python","patch_set":2,"id":"9ad45d7e_f32abb2b","line":1056,"range":{"start_line":1055,"start_character":0,"end_line":1056,"end_character":54},"in_reply_to":"9ad45d7e_76fcfe3e","updated":"2016-08-08 12:12:34.000000000","message":"AMQPDriverBase._send does not catch this Exception if we do raise instead of redeclaration. The _send() method catch only remote rpc and timeout exceptions. Connection.consume() used by AMQPListener, not AMQPDriverBase. \n\nAnyway if an exchange is auto-deleted we must redeclare and bind ours consumers (because they are alive) and the better place to do it is here - just before consuming starting.","commit_id":"c3ef60116da6dd456fd551fbbbf0ccadb5fe896d"},{"author":{"_account_id":21191,"name":"shen.zhixing","email":"fooy5460@sina.com"},"change_message_id":"3facb5fde4c5434f0f0ef10fe0da9f41c3e444bc","unresolved":false,"context_lines":[{"line_number":1052,"context_line":"                for consumer, tag in self._consumers.items():"},{"line_number":1053,"context_line":"                    if tag in self._new_tags:"},{"line_number":1054,"context_line":"                        try:"},{"line_number":1055,"context_line":"                            if not consumer.is_declared(self):"},{"line_number":1056,"context_line":"                                consumer.declare(self)"},{"line_number":1057,"context_line":"                            consumer.consume(tag\u003dtag)"},{"line_number":1058,"context_line":"                            self._new_tags.remove(tag)"},{"line_number":1059,"context_line":"                        except self.connection.channel_errors as exc:"}],"source_content_type":"text/x-python","patch_set":2,"id":"9ad45d7e_c6d25734","line":1056,"range":{"start_line":1055,"start_character":0,"end_line":1056,"end_character":54},"in_reply_to":"9ad45d7e_76fcfe3e","updated":"2016-08-08 12:03:32.000000000","message":"The Exception is raised from \"_waiter.wait\"; not only isinstance(result, Exception).","commit_id":"c3ef60116da6dd456fd551fbbbf0ccadb5fe896d"},{"author":{"_account_id":21191,"name":"shen.zhixing","email":"fooy5460@sina.com"},"change_message_id":"73e80434782122848c7ffd6093b40d564ed150d4","unresolved":false,"context_lines":[{"line_number":1052,"context_line":"                for consumer, tag in self._consumers.items():"},{"line_number":1053,"context_line":"                    if tag in self._new_tags:"},{"line_number":1054,"context_line":"                        try:"},{"line_number":1055,"context_line":"                            if not consumer.is_declared(self):"},{"line_number":1056,"context_line":"                                consumer.declare(self)"},{"line_number":1057,"context_line":"                            consumer.consume(tag\u003dtag)"},{"line_number":1058,"context_line":"                            self._new_tags.remove(tag)"},{"line_number":1059,"context_line":"                        except self.connection.channel_errors as exc:"}],"source_content_type":"text/x-python","patch_set":2,"id":"9ad45d7e_7f954185","line":1056,"range":{"start_line":1055,"start_character":0,"end_line":1056,"end_character":54},"in_reply_to":"9ad45d7e_f32abb2b","updated":"2016-08-11 07:08:06.000000000","message":"I can\u0027t reappear this problem in my compute,I have Added comment at below page.\n\nhttps://bugs.launchpad.net/oslo.messaging/+bug/1609766","commit_id":"c3ef60116da6dd456fd551fbbbf0ccadb5fe896d"},{"author":{"_account_id":19520,"name":"Kirill Bespalov","email":"kbespalov@mirantis.com","username":"k.bespalov"},"change_message_id":"da10586308dc07ab4a9c28a834a04e2022c18bf4","unresolved":false,"context_lines":[{"line_number":1053,"context_line":"                    if tag in self._new_tags:"},{"line_number":1054,"context_line":"                        try:"},{"line_number":1055,"context_line":"                            if not consumer.is_declared(self):"},{"line_number":1056,"context_line":"                                consumer.declare(self)"},{"line_number":1057,"context_line":"                            consumer.consume(tag\u003dtag)"},{"line_number":1058,"context_line":"                            self._new_tags.remove(tag)"},{"line_number":1059,"context_line":"                        except self.connection.channel_errors as exc:"}],"source_content_type":"text/x-python","patch_set":2,"id":"9a89bdaa_abeee2b1","line":1056,"in_reply_to":"bacf61ea_e559adfe","updated":"2016-09-06 12:27:35.000000000","message":"\u003eThe on_reconnection kombu callback already do this.\n\nActually not. We no have any guarantees that on_reconnection is successfully done. If any exception will be raised here (e.g. timeout exception when you trying to re-bind a queue to an exchange) then we just throw out from self.ensure(_consume) and return to the main consuming loop of AMQPListener:\n\n while True:\n  try:\n    connection.consume()\n  except Exception as e:\n     # we just ignore here any exceptions\n     # if even it exception from on_reconnection_callback\n\nSo, the bug: \n\nhttps://bugs.launchpad.net/oslo.messaging/+bug/1609766\n\nactually happens when on_reconnection fail due to timeout on a queue binding to the exchange. Timeout - NOT leads to the on_reconnection triggering, because the connection still alive, it just an operation fail like queue.bind, queue.declare, etc. \n\nThe race condition described at L1060-L1074 has another reason.\nIn this case on_reconnection is successfully done, but then the home node for the queue is fail and in this moment we connected to the other node (which no fails). So, the on_reconnection is not triggered, because we still connected, but the queue is lost on another node.","commit_id":"c3ef60116da6dd456fd551fbbbf0ccadb5fe896d"}]}
