)]}'
{"/PATCHSET_LEVEL":[{"author":{"_account_id":28356,"name":"Rafael Weingartner","email":"rafael@apache.org","username":"rafaelweingartner"},"change_message_id":"6dc66a2cd1f8e64108ed834a97fd435d9e21f467","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":2,"id":"7573ed72_d150a286","updated":"2022-02-26 00:00:03.000000000","message":"Hello guys, \nDo we have an update on this patch?","commit_id":"8e3cb833b43b1d895dc644e002fdb75e00f115d4"},{"author":{"_account_id":28356,"name":"Rafael Weingartner","email":"rafael@apache.org","username":"rafaelweingartner"},"change_message_id":"9007d68df752da32c88d58c30d9b4eaa2c0d985b","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":2,"id":"98f4770b_6f747831","updated":"2022-01-26 15:33:30.000000000","message":"Hello guys, \nDo we have an update on this patch?","commit_id":"8e3cb833b43b1d895dc644e002fdb75e00f115d4"},{"author":{"_account_id":28356,"name":"Rafael Weingartner","email":"rafael@apache.org","username":"rafaelweingartner"},"change_message_id":"0933dd04e41c91932f2a992c09b9d46e2b587181","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":2,"id":"721ae45a_f241ed16","updated":"2022-11-30 11:06:43.000000000","message":"I have rebased the patch. Can you guys review it?","commit_id":"8e3cb833b43b1d895dc644e002fdb75e00f115d4"},{"author":{"_account_id":28356,"name":"Rafael Weingartner","email":"rafael@apache.org","username":"rafaelweingartner"},"change_message_id":"34b99d250bf24c434cf3c0ec8a7a6733709232c9","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":2,"id":"1b32c37d_a169cf4b","updated":"2021-12-03 12:24:24.000000000","message":"Pavlo, I think you misunderstood my patch. I am not trying to solve filtering projects by domain_id for the case described by you. What I am solving is the locking issue, that was causing the exception I reported. \n\nI am not ignoring the `filter_project`, if you look a the code, \n```\nLOG.warning(\u0027filtered project not found in keystone,\u0027\n                       \u0027 ignoring the filter_project \u0027\n                      \u0027option\u0027)\nself.filter_project \u003d None\n```\n\nWhen the `filter_project` is not found, it is set to `None`, and this causes a problem when multiple concurrent executions are queued by the locking (`with self._gnocchi_project_id_lock`). That is the problem I am solving here. Therefore, the patch you pointed out is not an alternative to this one. They (the patches) are not concurrent, each one of them is solving a different problem. The code continues to do the same as it was before, I just fixed that improper locking code that was coded.\n\nIf you ignore this patch in favor of the other you pointed out, the problem still persists. ","commit_id":"8e3cb833b43b1d895dc644e002fdb75e00f115d4"},{"author":{"_account_id":9542,"name":"Pavlo Shchelokovskyy","email":"pshchelokovskyy@mirantis.com","username":"pshchelo"},"change_message_id":"e7e97b4b30c304ff0946fe5365fb94c017e0462c","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":2,"id":"7ded03b8_58ad0aae","updated":"2021-12-03 10:04:19.000000000","message":"not sure about the problem with locking, but with this patch you basically ignore the filter_domain.\n\nplease see https://review.opendev.org/c/openstack/ceilometer/+/820355 for the proper fix for filtering projects by domain_id, this solves the NoUniqueMatch for the case when e.g. several projects named \u0027service\u0027 are created in different domains.","commit_id":"8e3cb833b43b1d895dc644e002fdb75e00f115d4"},{"author":{"_account_id":9542,"name":"Pavlo Shchelokovskyy","email":"pshchelokovskyy@mirantis.com","username":"pshchelo"},"change_message_id":"9e1ed6b40a99a426bad70b95e9798b7f3f95556c","unresolved":true,"context_lines":[],"source_content_type":"","patch_set":2,"id":"517fc544_1013e54c","in_reply_to":"0e1e053b_8ff1a941","updated":"2022-07-13 20:08:17.000000000","message":"I don\u0027t think your explanation of meaning of \u0027filter_project\u0027 is correct. It not so much here to support \u0027hosting gnocchi VMs inside openstack\u0027. It is for events that gnocchi as as service itself generates. If we do not filter these out, we will get a self-sustaining loop of \"gnocchi generates event/sample -\u003e ceilometer catches it and stores in Gnocchi -\u003e gnocchi emits event/sample that something has been stored -\u003e ...\"\n\nSo the filter_project should never be set to None, and the \"service\" is a standard/default project name where service users (e.g. used by ceilometer to access gnocchi) reside.","commit_id":"8e3cb833b43b1d895dc644e002fdb75e00f115d4"},{"author":{"_account_id":28356,"name":"Rafael Weingartner","email":"rafael@apache.org","username":"rafaelweingartner"},"change_message_id":"114cc56829d49e8cf87dfb9684e379ce64ad29fd","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":2,"id":"0e1e053b_8ff1a941","in_reply_to":"1b32c37d_a169cf4b","updated":"2022-06-16 15:55:15.000000000","message":"Hello guys, can somebody help here?\n\nI mean, I have explained the situation, but the person that gave the -1 did not return my response. Is it suppose to be definitive like this? Somebody just slap a -1 somewhere, and then the author promptly responds the issue and explain that it is a misunderstanding, and nobody else jumps in to discuss this further.","commit_id":"8e3cb833b43b1d895dc644e002fdb75e00f115d4"},{"author":{"_account_id":28356,"name":"Rafael Weingartner","email":"rafael@apache.org","username":"rafaelweingartner"},"change_message_id":"cfa273c160b0a0534135faad2766661a94edc5f1","unresolved":true,"context_lines":[],"source_content_type":"","patch_set":2,"id":"c9c638c3_744bc9f5","in_reply_to":"517fc544_1013e54c","updated":"2022-11-30 10:55:08.000000000","message":"Let me stress what I am doing here. \u003cb\u003eI am not changing the logic\u003cb/\u003e. If one takes a look at [1], she/he will be able to see that the variable `self.filter_project\u003dNone` is set to None if the project name is not found in Keystone. The default `filter_project` is loaded at [2]. Then, when it is set to None because of code in [1], line [3] will return `False`, which in turn will make the code in [1] not to be called again.\n\nTherefore, I am not changing the logic. The process of setting the \"filter_project\" to None was already there, and it makes sense. Again, let me repeat; I am not changing the logic.\n\nNow, what am I changing/addressing?\n\nI am fixing concurrency issues that might happen as soon as the first boot/startup of Ceilometer. During that moment different thread might be stopped at line [4], then the first thread that is executed, if it does not find the \"filter_project\" in Keystone, it will set it to None. Afterwards, the others threads, when they execute, the \"filter_project\"  is not going to be a name anymore, it is going to be the value None, which will make Keystone to list all projects, which in turn creates an exception because we were expecting a single entry.\n\nTherefore, that is what I am addressing with this patch. I am simply addressing this very specific concurrency issue. No logic/process is changed. The behaviour that was there is maitained.\n\nDo you guys see/understand that? Should I try to explain it with some other words or some more detailed use case?\n\n\n\n\n[1] https://github.com/openstack/ceilometer/blob/e496b5d212ab141a156bc57a4894d80805d1b40a/ceilometer/publisher/gnocchi.py#L291\n\n[2] https://github.com/openstack/ceilometer/blob/e496b5d212ab141a156bc57a4894d80805d1b40a/ceilometer/publisher/gnocchi.py#L200\n\n[3] https://github.com/openstack/ceilometer/blob/e496b5d212ab141a156bc57a4894d80805d1b40a/ceilometer/publisher/gnocchi.py#L308\n\n[4] https://github.com/openstack/ceilometer/blob/e496b5d212ab141a156bc57a4894d80805d1b40a/ceilometer/publisher/gnocchi.py#L281","commit_id":"8e3cb833b43b1d895dc644e002fdb75e00f115d4"},{"author":{"_account_id":28356,"name":"Rafael Weingartner","email":"rafael@apache.org","username":"rafaelweingartner"},"change_message_id":"b963aa507f7c700dd361068cb3295236d5283901","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":2,"id":"269d3116_df7a5a36","in_reply_to":"c9c638c3_744bc9f5","updated":"2022-12-08 15:22:24.000000000","message":"Done","commit_id":"8e3cb833b43b1d895dc644e002fdb75e00f115d4"},{"author":{"_account_id":4264,"name":"Matthias Runge","email":"mrunge@redhat.com","username":"mrunge"},"change_message_id":"1587d16d7103c4394a374ab0db786017514a7159","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":4,"id":"4d1414f8_5d869579","updated":"2022-12-21 11:09:40.000000000","message":"sorry this took so long, thank you for the patch!","commit_id":"463594b229017b30a1c457aaf6d4cdfef2bb421c"},{"author":{"_account_id":28356,"name":"Rafael Weingartner","email":"rafael@apache.org","username":"rafaelweingartner"},"change_message_id":"3adc4ec6a9c14e8db2ab2ef8e9bf4a7f340b743b","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":4,"id":"6079d6be_2656e6a1","in_reply_to":"4d1414f8_5d869579","updated":"2022-12-21 11:41:12.000000000","message":"No worries, it happens. \n\nI thank you 😊\n\nIt has been a pleasure to work with you guys for all these years.","commit_id":"463594b229017b30a1c457aaf6d4cdfef2bb421c"}],"ceilometer/publisher/gnocchi.py":[{"author":{"_account_id":32240,"name":"Yadnesh Kulkarni","email":"yadnesh_kulkarni@proton.me","username":"ykulkarn"},"change_message_id":"615e0f4b08e9d82fd0756973338a0d76c14d570c","unresolved":true,"context_lines":[{"line_number":279,"context_line":"        if self._gnocchi_project_id is not None:"},{"line_number":280,"context_line":"            return self._gnocchi_project_id"},{"line_number":281,"context_line":"        with self._gnocchi_project_id_lock:"},{"line_number":282,"context_line":"            if self._gnocchi_project_id is None:"},{"line_number":283,"context_line":"                try:"},{"line_number":284,"context_line":"                    project \u003d self._ks_client.projects.find("},{"line_number":285,"context_line":"                        name\u003dself.filter_project,"}],"source_content_type":"text/x-python","patch_set":3,"id":"75fb45e4_3f53f02b","side":"PARENT","line":282,"range":{"start_line":282,"start_character":47,"end_line":282,"end_character":48},"updated":"2022-12-07 11:47:36.000000000","message":"I agree with concurrency issue here (only the first thread gets the value of filter_project, rest all get None). Instead of moving the below code to new function (_internal_gnocchi_project_discovery) we could fix it by checking whether filter_project is not None here like\n```\nif self._gnocchi_project_id is None and self.filter_project:\n```\n\nOnly difference here, the changes proposed here checks filter_project first and then _gnocchi_project_id in another function.","commit_id":"060a13455cbcf759eac451c2eb923af31b08c625"},{"author":{"_account_id":28356,"name":"Rafael Weingartner","email":"rafael@apache.org","username":"rafaelweingartner"},"change_message_id":"56412afcbc3251a41e977d28ea10ee8c71cdd1f7","unresolved":true,"context_lines":[{"line_number":279,"context_line":"        if self._gnocchi_project_id is not None:"},{"line_number":280,"context_line":"            return self._gnocchi_project_id"},{"line_number":281,"context_line":"        with self._gnocchi_project_id_lock:"},{"line_number":282,"context_line":"            if self._gnocchi_project_id is None:"},{"line_number":283,"context_line":"                try:"},{"line_number":284,"context_line":"                    project \u003d self._ks_client.projects.find("},{"line_number":285,"context_line":"                        name\u003dself.filter_project,"}],"source_content_type":"text/x-python","patch_set":3,"id":"b2979577_b8dead9c","side":"PARENT","line":282,"range":{"start_line":282,"start_character":47,"end_line":282,"end_character":48},"in_reply_to":"75fb45e4_3f53f02b","updated":"2022-12-07 12:00:57.000000000","message":"Yes, we could. However, I moved to a new method to reduce the nesting and to make things a bit more readable that we have there. You would like to me use the other approach?","commit_id":"060a13455cbcf759eac451c2eb923af31b08c625"},{"author":{"_account_id":28356,"name":"Rafael Weingartner","email":"rafael@apache.org","username":"rafaelweingartner"},"change_message_id":"b5cf33e00d6a193394bbfbbb613ff4007a9c02e4","unresolved":false,"context_lines":[{"line_number":279,"context_line":"        if self._gnocchi_project_id is not None:"},{"line_number":280,"context_line":"            return self._gnocchi_project_id"},{"line_number":281,"context_line":"        with self._gnocchi_project_id_lock:"},{"line_number":282,"context_line":"            if self._gnocchi_project_id is None:"},{"line_number":283,"context_line":"                try:"},{"line_number":284,"context_line":"                    project \u003d self._ks_client.projects.find("},{"line_number":285,"context_line":"                        name\u003dself.filter_project,"}],"source_content_type":"text/x-python","patch_set":3,"id":"981dca55_29efb6e4","side":"PARENT","line":282,"range":{"start_line":282,"start_character":47,"end_line":282,"end_character":48},"in_reply_to":"96843e6e_870dda29","updated":"2022-12-08 13:39:38.000000000","message":"ok","commit_id":"060a13455cbcf759eac451c2eb923af31b08c625"},{"author":{"_account_id":32240,"name":"Yadnesh Kulkarni","email":"yadnesh_kulkarni@proton.me","username":"ykulkarn"},"change_message_id":"d8ad3f9d7309b045d503d1667612fafe91dc0505","unresolved":true,"context_lines":[{"line_number":279,"context_line":"        if self._gnocchi_project_id is not None:"},{"line_number":280,"context_line":"            return self._gnocchi_project_id"},{"line_number":281,"context_line":"        with self._gnocchi_project_id_lock:"},{"line_number":282,"context_line":"            if self._gnocchi_project_id is None:"},{"line_number":283,"context_line":"                try:"},{"line_number":284,"context_line":"                    project \u003d self._ks_client.projects.find("},{"line_number":285,"context_line":"                        name\u003dself.filter_project,"}],"source_content_type":"text/x-python","patch_set":3,"id":"96843e6e_870dda29","side":"PARENT","line":282,"range":{"start_line":282,"start_character":47,"end_line":282,"end_character":48},"in_reply_to":"b2979577_b8dead9c","updated":"2022-12-07 16:16:24.000000000","message":"Yes, I\u0027d prefer checking filter_project alongside _gnocchi_project_id.","commit_id":"060a13455cbcf759eac451c2eb923af31b08c625"},{"author":{"_account_id":32240,"name":"Yadnesh Kulkarni","email":"yadnesh_kulkarni@proton.me","username":"ykulkarn"},"change_message_id":"d8ad3f9d7309b045d503d1667612fafe91dc0505","unresolved":true,"context_lines":[{"line_number":289,"context_line":""},{"line_number":290,"context_line":"            return self._gnocchi_project_id"},{"line_number":291,"context_line":""},{"line_number":292,"context_line":"    def _internal_gnocchi_project_discovery(self):"},{"line_number":293,"context_line":"        if self._gnocchi_project_id is None:"},{"line_number":294,"context_line":"            try:"},{"line_number":295,"context_line":"                project \u003d self._ks_client.projects.find("}],"source_content_type":"text/x-python","patch_set":3,"id":"280c3778_1c8723bb","line":292,"range":{"start_line":292,"start_character":8,"end_line":292,"end_character":43},"updated":"2022-12-07 16:16:24.000000000","message":"Unless finding projects is a repetitive task in gnocchi publisher, we should move it back into gnocchi_project_id() and not have a new function.","commit_id":"a6c00d3f6c7afbe2c6b79d48dc0f4cd53c1df3c2"},{"author":{"_account_id":28356,"name":"Rafael Weingartner","email":"rafael@apache.org","username":"rafaelweingartner"},"change_message_id":"b5cf33e00d6a193394bbfbbb613ff4007a9c02e4","unresolved":false,"context_lines":[{"line_number":289,"context_line":""},{"line_number":290,"context_line":"            return self._gnocchi_project_id"},{"line_number":291,"context_line":""},{"line_number":292,"context_line":"    def _internal_gnocchi_project_discovery(self):"},{"line_number":293,"context_line":"        if self._gnocchi_project_id is None:"},{"line_number":294,"context_line":"            try:"},{"line_number":295,"context_line":"                project \u003d self._ks_client.projects.find("}],"source_content_type":"text/x-python","patch_set":3,"id":"576cd3de_8ccd1a53","line":292,"range":{"start_line":292,"start_character":8,"end_line":292,"end_character":43},"in_reply_to":"280c3778_1c8723bb","updated":"2022-12-08 13:39:38.000000000","message":"Not always like that. Otherwise, you end-up with crazy long methods. It is a good practice to reduce the method body, which in turn allow testing, and proper documentation.","commit_id":"a6c00d3f6c7afbe2c6b79d48dc0f4cd53c1df3c2"},{"author":{"_account_id":32240,"name":"Yadnesh Kulkarni","email":"yadnesh_kulkarni@proton.me","username":"ykulkarn"},"change_message_id":"d8ad3f9d7309b045d503d1667612fafe91dc0505","unresolved":true,"context_lines":[{"line_number":297,"context_line":"            except ka_exceptions.NotFound:"},{"line_number":298,"context_line":"                LOG.warning(\u0027filtered project [%s] not found in keystone,\u0027"},{"line_number":299,"context_line":"                            \u0027 ignoring the filter_project \u0027"},{"line_number":300,"context_line":"                            \u0027option\u0027 % self.filter_project)"},{"line_number":301,"context_line":"                self.filter_project \u003d None"},{"line_number":302,"context_line":"                return None"},{"line_number":303,"context_line":"            except Exception:"}],"source_content_type":"text/x-python","patch_set":3,"id":"b70792e7_2ef6e6d8","line":300,"range":{"start_line":300,"start_character":39,"end_line":300,"end_character":58},"updated":"2022-12-07 16:16:24.000000000","message":"Logging the name of the project that couldn\u0027t be found would be helpful.","commit_id":"a6c00d3f6c7afbe2c6b79d48dc0f4cd53c1df3c2"},{"author":{"_account_id":28356,"name":"Rafael Weingartner","email":"rafael@apache.org","username":"rafaelweingartner"},"change_message_id":"b963aa507f7c700dd361068cb3295236d5283901","unresolved":false,"context_lines":[{"line_number":297,"context_line":"            except ka_exceptions.NotFound:"},{"line_number":298,"context_line":"                LOG.warning(\u0027filtered project [%s] not found in keystone,\u0027"},{"line_number":299,"context_line":"                            \u0027 ignoring the filter_project \u0027"},{"line_number":300,"context_line":"                            \u0027option\u0027 % self.filter_project)"},{"line_number":301,"context_line":"                self.filter_project \u003d None"},{"line_number":302,"context_line":"                return None"},{"line_number":303,"context_line":"            except Exception:"}],"source_content_type":"text/x-python","patch_set":3,"id":"fc7731c6_f705cfd7","line":300,"range":{"start_line":300,"start_character":39,"end_line":300,"end_character":58},"in_reply_to":"45c19dcc_6a9a9a6f","updated":"2022-12-08 15:22:24.000000000","message":"Ack","commit_id":"a6c00d3f6c7afbe2c6b79d48dc0f4cd53c1df3c2"},{"author":{"_account_id":32240,"name":"Yadnesh Kulkarni","email":"yadnesh_kulkarni@proton.me","username":"ykulkarn"},"change_message_id":"10f62931984bdc35eaba279769d53d43c32b0914","unresolved":true,"context_lines":[{"line_number":297,"context_line":"            except ka_exceptions.NotFound:"},{"line_number":298,"context_line":"                LOG.warning(\u0027filtered project [%s] not found in keystone,\u0027"},{"line_number":299,"context_line":"                            \u0027 ignoring the filter_project \u0027"},{"line_number":300,"context_line":"                            \u0027option\u0027 % self.filter_project)"},{"line_number":301,"context_line":"                self.filter_project \u003d None"},{"line_number":302,"context_line":"                return None"},{"line_number":303,"context_line":"            except Exception:"}],"source_content_type":"text/x-python","patch_set":3,"id":"45c19dcc_6a9a9a6f","line":300,"range":{"start_line":300,"start_character":39,"end_line":300,"end_character":58},"in_reply_to":"6a3301db_0cf25e1e","updated":"2022-12-08 15:06:25.000000000","message":"Sorry about the way I framed this. I meant to agree with your change for having better logs if project is not found.","commit_id":"a6c00d3f6c7afbe2c6b79d48dc0f4cd53c1df3c2"},{"author":{"_account_id":28356,"name":"Rafael Weingartner","email":"rafael@apache.org","username":"rafaelweingartner"},"change_message_id":"b5cf33e00d6a193394bbfbbb613ff4007a9c02e4","unresolved":true,"context_lines":[{"line_number":297,"context_line":"            except ka_exceptions.NotFound:"},{"line_number":298,"context_line":"                LOG.warning(\u0027filtered project [%s] not found in keystone,\u0027"},{"line_number":299,"context_line":"                            \u0027 ignoring the filter_project \u0027"},{"line_number":300,"context_line":"                            \u0027option\u0027 % self.filter_project)"},{"line_number":301,"context_line":"                self.filter_project \u003d None"},{"line_number":302,"context_line":"                return None"},{"line_number":303,"context_line":"            except Exception:"}],"source_content_type":"text/x-python","patch_set":3,"id":"6a3301db_0cf25e1e","line":300,"range":{"start_line":300,"start_character":39,"end_line":300,"end_character":58},"in_reply_to":"b70792e7_2ef6e6d8","updated":"2022-12-08 13:39:38.000000000","message":"I do not understand the comment. I am already doing that here.","commit_id":"a6c00d3f6c7afbe2c6b79d48dc0f4cd53c1df3c2"},{"author":{"_account_id":32240,"name":"Yadnesh Kulkarni","email":"yadnesh_kulkarni@proton.me","username":"ykulkarn"},"change_message_id":"615e0f4b08e9d82fd0756973338a0d76c14d570c","unresolved":true,"context_lines":[{"line_number":331,"context_line":"            if operation:"},{"line_number":332,"context_line":"                return rd, operation"},{"line_number":333,"context_line":""},{"line_number":334,"context_line":"    def filter_gnocchi_activity_openstack(self, samples):"},{"line_number":335,"context_line":"        \"\"\"Skip sample generated by gnocchi itself"},{"line_number":336,"context_line":""},{"line_number":337,"context_line":"        This method will filter out the samples that are generated by"}],"source_content_type":"text/x-python","patch_set":3,"id":"5ea72d68_69c382f2","line":334,"updated":"2022-12-07 11:47:36.000000000","message":"If I understand correctly, this function seems to be removing gnocchi activity samples using \"_is_gnocchi_activity()\" and adds the rest to filtered_samples list. Not sure how this is different from the current code which is \n```\n        # NOTE(sileht): skip sample generated by gnocchi itself\n        data \u003d [s for s in data if not self._is_gnocchi_activity(s)]\n```\nThey seem to be doing the same.\n\n\nAlso, if gnocchi is hosted inside a VM running on OSP, how would those metrics get picked up by ceilometer ? In such a scenario, I think the only connection between these 2 services would be pipeline.yaml would have the a gnocchi publisher with ip of the vm.","commit_id":"a6c00d3f6c7afbe2c6b79d48dc0f4cd53c1df3c2"},{"author":{"_account_id":28356,"name":"Rafael Weingartner","email":"rafael@apache.org","username":"rafaelweingartner"},"change_message_id":"56412afcbc3251a41e977d28ea10ee8c71cdd1f7","unresolved":true,"context_lines":[{"line_number":331,"context_line":"            if operation:"},{"line_number":332,"context_line":"                return rd, operation"},{"line_number":333,"context_line":""},{"line_number":334,"context_line":"    def filter_gnocchi_activity_openstack(self, samples):"},{"line_number":335,"context_line":"        \"\"\"Skip sample generated by gnocchi itself"},{"line_number":336,"context_line":""},{"line_number":337,"context_line":"        This method will filter out the samples that are generated by"}],"source_content_type":"text/x-python","patch_set":3,"id":"79fe2560_78401ad7","line":334,"in_reply_to":"5ea72d68_69c382f2","updated":"2022-12-07 12:00:57.000000000","message":"Exactly. The code is the same. I just extracted to a method to document it. \n\nThe thing regarding \"filtering out gnocchi measures for Gnocchi VMs running in OpenStack\" was may assumption after reading the code \u003d). I have no idea why this is here, and where people use it. I just assumed that. I can remove that comment to avoid confusion then.\n\nAs I explained, the behavior is maintained,I just fixed the concurrency issue, and extracted some code to methods to make the blocks a bit smaller, and tried to document a bit the code being extracted.","commit_id":"a6c00d3f6c7afbe2c6b79d48dc0f4cd53c1df3c2"},{"author":{"_account_id":32240,"name":"Yadnesh Kulkarni","email":"yadnesh_kulkarni@proton.me","username":"ykulkarn"},"change_message_id":"d8ad3f9d7309b045d503d1667612fafe91dc0505","unresolved":true,"context_lines":[{"line_number":331,"context_line":"            if operation:"},{"line_number":332,"context_line":"                return rd, operation"},{"line_number":333,"context_line":""},{"line_number":334,"context_line":"    def filter_gnocchi_activity_openstack(self, samples):"},{"line_number":335,"context_line":"        \"\"\"Skip sample generated by gnocchi itself"},{"line_number":336,"context_line":""},{"line_number":337,"context_line":"        This method will filter out the samples that are generated by"}],"source_content_type":"text/x-python","patch_set":3,"id":"86a62d7c_89190add","line":334,"in_reply_to":"79fe2560_78401ad7","updated":"2022-12-07 16:16:24.000000000","message":"Afaik we exclude metrics from \"service\" project because it contains the service users of all OSP services and we do not want the metrics generated when these users consume cloud resources. For eg, some osp services are known to create objects in swift which is needed by them to function correctly and we don\u0027t want these metrics to show up in tenant user\u0027s metrics.","commit_id":"a6c00d3f6c7afbe2c6b79d48dc0f4cd53c1df3c2"},{"author":{"_account_id":28356,"name":"Rafael Weingartner","email":"rafael@apache.org","username":"rafaelweingartner"},"change_message_id":"b5cf33e00d6a193394bbfbbb613ff4007a9c02e4","unresolved":false,"context_lines":[{"line_number":331,"context_line":"            if operation:"},{"line_number":332,"context_line":"                return rd, operation"},{"line_number":333,"context_line":""},{"line_number":334,"context_line":"    def filter_gnocchi_activity_openstack(self, samples):"},{"line_number":335,"context_line":"        \"\"\"Skip sample generated by gnocchi itself"},{"line_number":336,"context_line":""},{"line_number":337,"context_line":"        This method will filter out the samples that are generated by"}],"source_content_type":"text/x-python","patch_set":3,"id":"2f13b96c_11c7be2b","line":334,"in_reply_to":"86a62d7c_89190add","updated":"2022-12-08 13:39:38.000000000","message":"Done","commit_id":"a6c00d3f6c7afbe2c6b79d48dc0f4cd53c1df3c2"},{"author":{"_account_id":32240,"name":"Yadnesh Kulkarni","email":"yadnesh_kulkarni@proton.me","username":"ykulkarn"},"change_message_id":"d8ad3f9d7309b045d503d1667612fafe91dc0505","unresolved":true,"context_lines":[{"line_number":335,"context_line":"        \"\"\"Skip sample generated by gnocchi itself"},{"line_number":336,"context_line":""},{"line_number":337,"context_line":"        This method will filter out the samples that are generated by"},{"line_number":338,"context_line":"        Gnocchi itself. This will happen when Gnocchi is running in virtual"},{"line_number":339,"context_line":"        machine instances that are hosted in the OpenStack environment that"},{"line_number":340,"context_line":"        we gather data from."},{"line_number":341,"context_line":"        \"\"\""},{"line_number":342,"context_line":"        filtered_samples \u003d []"},{"line_number":343,"context_line":"        for sample in samples:"}],"source_content_type":"text/x-python","patch_set":3,"id":"76704dda_04602661","line":340,"range":{"start_line":338,"start_character":24,"end_line":340,"end_character":28},"updated":"2022-12-07 16:16:24.000000000","message":"It would better to not have this.","commit_id":"a6c00d3f6c7afbe2c6b79d48dc0f4cd53c1df3c2"},{"author":{"_account_id":28356,"name":"Rafael Weingartner","email":"rafael@apache.org","username":"rafaelweingartner"},"change_message_id":"b5cf33e00d6a193394bbfbbb613ff4007a9c02e4","unresolved":false,"context_lines":[{"line_number":335,"context_line":"        \"\"\"Skip sample generated by gnocchi itself"},{"line_number":336,"context_line":""},{"line_number":337,"context_line":"        This method will filter out the samples that are generated by"},{"line_number":338,"context_line":"        Gnocchi itself. This will happen when Gnocchi is running in virtual"},{"line_number":339,"context_line":"        machine instances that are hosted in the OpenStack environment that"},{"line_number":340,"context_line":"        we gather data from."},{"line_number":341,"context_line":"        \"\"\""},{"line_number":342,"context_line":"        filtered_samples \u003d []"},{"line_number":343,"context_line":"        for sample in samples:"}],"source_content_type":"text/x-python","patch_set":3,"id":"b0c19282_5725c74d","line":340,"range":{"start_line":338,"start_character":24,"end_line":340,"end_character":28},"in_reply_to":"76704dda_04602661","updated":"2022-12-08 13:39:38.000000000","message":"Done","commit_id":"a6c00d3f6c7afbe2c6b79d48dc0f4cd53c1df3c2"},{"author":{"_account_id":32240,"name":"Yadnesh Kulkarni","email":"yadnesh_kulkarni@proton.me","username":"ykulkarn"},"change_message_id":"d8ad3f9d7309b045d503d1667612fafe91dc0505","unresolved":true,"context_lines":[{"line_number":343,"context_line":"        for sample in samples:"},{"line_number":344,"context_line":"            if not self._is_gnocchi_activity(sample):"},{"line_number":345,"context_line":"                filtered_samples.append(sample)"},{"line_number":346,"context_line":"                LOG.debug(\"Sample [%s] is not a Gnocchi activity; therefore, \""},{"line_number":347,"context_line":"                          \"we do not filter it out and push it to Gnocchi.\","},{"line_number":348,"context_line":"                          sample)"},{"line_number":349,"context_line":"            else:"},{"line_number":350,"context_line":"                LOG.debug(\"Sample [%s] is a Gnocchi activity; therefore, \""},{"line_number":351,"context_line":"                          \"we filter it out and do not push it to Gnocchi.\","}],"source_content_type":"text/x-python","patch_set":3,"id":"89dda0f0_26ca0877","line":348,"range":{"start_line":346,"start_character":16,"end_line":348,"end_character":33},"updated":"2022-12-07 16:16:24.000000000","message":"Logging samples before pushing to gnocchi seems unnecessary. Depending upon the scale of cloud and the samples generated, this could easily flood ceilometer log files if debug is kept enabled for a while.","commit_id":"a6c00d3f6c7afbe2c6b79d48dc0f4cd53c1df3c2"},{"author":{"_account_id":28356,"name":"Rafael Weingartner","email":"rafael@apache.org","username":"rafaelweingartner"},"change_message_id":"b5cf33e00d6a193394bbfbbb613ff4007a9c02e4","unresolved":false,"context_lines":[{"line_number":343,"context_line":"        for sample in samples:"},{"line_number":344,"context_line":"            if not self._is_gnocchi_activity(sample):"},{"line_number":345,"context_line":"                filtered_samples.append(sample)"},{"line_number":346,"context_line":"                LOG.debug(\"Sample [%s] is not a Gnocchi activity; therefore, \""},{"line_number":347,"context_line":"                          \"we do not filter it out and push it to Gnocchi.\","},{"line_number":348,"context_line":"                          sample)"},{"line_number":349,"context_line":"            else:"},{"line_number":350,"context_line":"                LOG.debug(\"Sample [%s] is a Gnocchi activity; therefore, \""},{"line_number":351,"context_line":"                          \"we filter it out and do not push it to Gnocchi.\","}],"source_content_type":"text/x-python","patch_set":3,"id":"9ba20b36_d796595a","line":348,"range":{"start_line":346,"start_character":16,"end_line":348,"end_character":33},"in_reply_to":"89dda0f0_26ca0877","updated":"2022-12-08 13:39:38.000000000","message":"There is a simple fix for that. Do not enable the DEBUG log level. Only use it when troubleshooting something.\n\nIf you check the Dynamic pollster implementation, logs help operators to troubleshoot the system without needing to scale the issue. Other parts of Ceilometer, do not help; when we set the LOG level to DEBUG, nothing extra appears. Logs are a very important element of systems, and sadly when there are no extra logs in DEBUG, we make the life of operators a living hell sometimes when troubleshooting issues with the system.\n\nI would rather keep them.","commit_id":"a6c00d3f6c7afbe2c6b79d48dc0f4cd53c1df3c2"},{"author":{"_account_id":32240,"name":"Yadnesh Kulkarni","email":"yadnesh_kulkarni@proton.me","username":"ykulkarn"},"change_message_id":"d8ad3f9d7309b045d503d1667612fafe91dc0505","unresolved":true,"context_lines":[{"line_number":347,"context_line":"                          \"we do not filter it out and push it to Gnocchi.\","},{"line_number":348,"context_line":"                          sample)"},{"line_number":349,"context_line":"            else:"},{"line_number":350,"context_line":"                LOG.debug(\"Sample [%s] is a Gnocchi activity; therefore, \""},{"line_number":351,"context_line":"                          \"we filter it out and do not push it to Gnocchi.\","},{"line_number":352,"context_line":"                          sample)"},{"line_number":353,"context_line":"        return filtered_samples"},{"line_number":354,"context_line":""},{"line_number":355,"context_line":"    def publish_samples(self, data):"}],"source_content_type":"text/x-python","patch_set":3,"id":"f38e814d_1e27db42","line":352,"range":{"start_line":350,"start_character":0,"end_line":352,"end_character":33},"updated":"2022-12-07 16:16:24.000000000","message":"Here as well.","commit_id":"a6c00d3f6c7afbe2c6b79d48dc0f4cd53c1df3c2"},{"author":{"_account_id":28356,"name":"Rafael Weingartner","email":"rafael@apache.org","username":"rafaelweingartner"},"change_message_id":"b5cf33e00d6a193394bbfbbb613ff4007a9c02e4","unresolved":false,"context_lines":[{"line_number":347,"context_line":"                          \"we do not filter it out and push it to Gnocchi.\","},{"line_number":348,"context_line":"                          sample)"},{"line_number":349,"context_line":"            else:"},{"line_number":350,"context_line":"                LOG.debug(\"Sample [%s] is a Gnocchi activity; therefore, \""},{"line_number":351,"context_line":"                          \"we filter it out and do not push it to Gnocchi.\","},{"line_number":352,"context_line":"                          sample)"},{"line_number":353,"context_line":"        return filtered_samples"},{"line_number":354,"context_line":""},{"line_number":355,"context_line":"    def publish_samples(self, data):"}],"source_content_type":"text/x-python","patch_set":3,"id":"87c5d6b4_faa3d7f0","line":352,"range":{"start_line":350,"start_character":0,"end_line":352,"end_character":33},"in_reply_to":"f38e814d_1e27db42","updated":"2022-12-08 13:39:38.000000000","message":"same here","commit_id":"a6c00d3f6c7afbe2c6b79d48dc0f4cd53c1df3c2"},{"author":{"_account_id":32240,"name":"Yadnesh Kulkarni","email":"yadnesh_kulkarni@proton.me","username":"ykulkarn"},"change_message_id":"d8ad3f9d7309b045d503d1667612fafe91dc0505","unresolved":true,"context_lines":[{"line_number":355,"context_line":"    def publish_samples(self, data):"},{"line_number":356,"context_line":"        self.ensures_archives_policies()"},{"line_number":357,"context_line":""},{"line_number":358,"context_line":"        data \u003d self.filter_gnocchi_activity_openstack(data)"},{"line_number":359,"context_line":""},{"line_number":360,"context_line":"        def value_to_sort(object_to_sort):"},{"line_number":361,"context_line":"            value \u003d object_to_sort.resource_id"}],"source_content_type":"text/x-python","patch_set":3,"id":"36e405e6_5dfac589","line":358,"updated":"2022-12-07 16:16:24.000000000","message":"I do not have any issues with introducing a new function here, what we have now just feels good and compact.","commit_id":"a6c00d3f6c7afbe2c6b79d48dc0f4cd53c1df3c2"},{"author":{"_account_id":28356,"name":"Rafael Weingartner","email":"rafael@apache.org","username":"rafaelweingartner"},"change_message_id":"b5cf33e00d6a193394bbfbbb613ff4007a9c02e4","unresolved":false,"context_lines":[{"line_number":355,"context_line":"    def publish_samples(self, data):"},{"line_number":356,"context_line":"        self.ensures_archives_policies()"},{"line_number":357,"context_line":""},{"line_number":358,"context_line":"        data \u003d self.filter_gnocchi_activity_openstack(data)"},{"line_number":359,"context_line":""},{"line_number":360,"context_line":"        def value_to_sort(object_to_sort):"},{"line_number":361,"context_line":"            value \u003d object_to_sort.resource_id"}],"source_content_type":"text/x-python","patch_set":3,"id":"b93203ea_2aa80000","line":358,"in_reply_to":"36e405e6_5dfac589","updated":"2022-12-08 13:39:38.000000000","message":"Ack","commit_id":"a6c00d3f6c7afbe2c6b79d48dc0f4cd53c1df3c2"},{"author":{"_account_id":32240,"name":"Yadnesh Kulkarni","email":"yadnesh_kulkarni@proton.me","username":"ykulkarn"},"change_message_id":"10f62931984bdc35eaba279769d53d43c32b0914","unresolved":true,"context_lines":[{"line_number":281,"context_line":"        with self._gnocchi_project_id_lock:"},{"line_number":282,"context_line":"            if self._gnocchi_project_id is None:"},{"line_number":283,"context_line":"                if not self.filter_project:"},{"line_number":284,"context_line":"                    LOG.debug("},{"line_number":285,"context_line":"                        \"Multiple executions were locked on \""},{"line_number":286,"context_line":"                        \"self._gnocchi_project_id_lock`. This execution \""},{"line_number":287,"context_line":"                        \"should no call `_internal_gnocchi_project_discovery` \""},{"line_number":288,"context_line":"                        \"as `self.filter_project` is None.\")"},{"line_number":289,"context_line":"                    return None"},{"line_number":290,"context_line":"                try:"},{"line_number":291,"context_line":"                    project \u003d self._ks_client.projects.find("}],"source_content_type":"text/x-python","patch_set":4,"id":"077d3285_2e5f7ea8","line":288,"range":{"start_line":284,"start_character":0,"end_line":288,"end_character":60},"updated":"2022-12-08 15:06:25.000000000","message":"As we got rid of _internal_gnocchi_project_discovery we don\u0027t need this debug log.","commit_id":"463594b229017b30a1c457aaf6d4cdfef2bb421c"},{"author":{"_account_id":28356,"name":"Rafael Weingartner","email":"rafael@apache.org","username":"rafaelweingartner"},"change_message_id":"b963aa507f7c700dd361068cb3295236d5283901","unresolved":true,"context_lines":[{"line_number":281,"context_line":"        with self._gnocchi_project_id_lock:"},{"line_number":282,"context_line":"            if self._gnocchi_project_id is None:"},{"line_number":283,"context_line":"                if not self.filter_project:"},{"line_number":284,"context_line":"                    LOG.debug("},{"line_number":285,"context_line":"                        \"Multiple executions were locked on \""},{"line_number":286,"context_line":"                        \"self._gnocchi_project_id_lock`. This execution \""},{"line_number":287,"context_line":"                        \"should no call `_internal_gnocchi_project_discovery` \""},{"line_number":288,"context_line":"                        \"as `self.filter_project` is None.\")"},{"line_number":289,"context_line":"                    return None"},{"line_number":290,"context_line":"                try:"},{"line_number":291,"context_line":"                    project \u003d self._ks_client.projects.find("}],"source_content_type":"text/x-python","patch_set":4,"id":"0e1e81b1_d9b92a42","line":288,"range":{"start_line":284,"start_character":0,"end_line":288,"end_character":60},"in_reply_to":"077d3285_2e5f7ea8","updated":"2022-12-08 15:22:24.000000000","message":"I find this useful. It is going to show there exist race-conditions happening, and that the code is addressing them. It is a DEBUG log message that is not displayed by default, only when enabled during a troubleshoot.\n\nI prefer to maintain it.","commit_id":"463594b229017b30a1c457aaf6d4cdfef2bb421c"},{"author":{"_account_id":28356,"name":"Rafael Weingartner","email":"rafael@apache.org","username":"rafaelweingartner"},"change_message_id":"8b6b5ce0a5fef227b6898fffddbe24182aa8549d","unresolved":false,"context_lines":[{"line_number":281,"context_line":"        with self._gnocchi_project_id_lock:"},{"line_number":282,"context_line":"            if self._gnocchi_project_id is None:"},{"line_number":283,"context_line":"                if not self.filter_project:"},{"line_number":284,"context_line":"                    LOG.debug("},{"line_number":285,"context_line":"                        \"Multiple executions were locked on \""},{"line_number":286,"context_line":"                        \"self._gnocchi_project_id_lock`. This execution \""},{"line_number":287,"context_line":"                        \"should no call `_internal_gnocchi_project_discovery` \""},{"line_number":288,"context_line":"                        \"as `self.filter_project` is None.\")"},{"line_number":289,"context_line":"                    return None"},{"line_number":290,"context_line":"                try:"},{"line_number":291,"context_line":"                    project \u003d self._ks_client.projects.find("}],"source_content_type":"text/x-python","patch_set":4,"id":"8aeeaf87_52d8646c","line":288,"range":{"start_line":284,"start_character":0,"end_line":288,"end_character":60},"in_reply_to":"0e1e81b1_d9b92a42","updated":"2022-12-09 11:45:17.000000000","message":"Done","commit_id":"463594b229017b30a1c457aaf6d4cdfef2bb421c"},{"author":{"_account_id":32240,"name":"Yadnesh Kulkarni","email":"yadnesh_kulkarni@proton.me","username":"ykulkarn"},"change_message_id":"10f62931984bdc35eaba279769d53d43c32b0914","unresolved":true,"context_lines":[{"line_number":339,"context_line":"        for sample in samples:"},{"line_number":340,"context_line":"            if not self._is_gnocchi_activity(sample):"},{"line_number":341,"context_line":"                filtered_samples.append(sample)"},{"line_number":342,"context_line":"                LOG.debug(\"Sample [%s] is not a Gnocchi activity; therefore, \""},{"line_number":343,"context_line":"                          \"we do not filter it out and push it to Gnocchi.\","},{"line_number":344,"context_line":"                          sample)"},{"line_number":345,"context_line":"            else:"},{"line_number":346,"context_line":"                LOG.debug(\"Sample [%s] is a Gnocchi activity; therefore, \""},{"line_number":347,"context_line":"                          \"we filter it out and do not push it to Gnocchi.\","}],"source_content_type":"text/x-python","patch_set":4,"id":"641a9c9f_79ae0c22","line":344,"range":{"start_line":342,"start_character":0,"end_line":344,"end_character":33},"updated":"2022-12-08 15:06:25.000000000","message":"I\u0027d recommend to remove this debug and the one below this. I had added a reason in the previous review.","commit_id":"463594b229017b30a1c457aaf6d4cdfef2bb421c"},{"author":{"_account_id":28356,"name":"Rafael Weingartner","email":"rafael@apache.org","username":"rafaelweingartner"},"change_message_id":"b963aa507f7c700dd361068cb3295236d5283901","unresolved":true,"context_lines":[{"line_number":339,"context_line":"        for sample in samples:"},{"line_number":340,"context_line":"            if not self._is_gnocchi_activity(sample):"},{"line_number":341,"context_line":"                filtered_samples.append(sample)"},{"line_number":342,"context_line":"                LOG.debug(\"Sample [%s] is not a Gnocchi activity; therefore, \""},{"line_number":343,"context_line":"                          \"we do not filter it out and push it to Gnocchi.\","},{"line_number":344,"context_line":"                          sample)"},{"line_number":345,"context_line":"            else:"},{"line_number":346,"context_line":"                LOG.debug(\"Sample [%s] is a Gnocchi activity; therefore, \""},{"line_number":347,"context_line":"                          \"we filter it out and do not push it to Gnocchi.\","}],"source_content_type":"text/x-python","patch_set":4,"id":"a6899531_54a76eef","line":344,"range":{"start_line":342,"start_character":0,"end_line":344,"end_character":33},"in_reply_to":"641a9c9f_79ae0c22","updated":"2022-12-08 15:22:24.000000000","message":"I replied and explain why it is useful. I also presented valid reasons to maintain as is now.\n\n- DEBUG is not enable by default\n- When troubleshooting, these information can be handy to understand how the processing of the system behaves","commit_id":"463594b229017b30a1c457aaf6d4cdfef2bb421c"},{"author":{"_account_id":28356,"name":"Rafael Weingartner","email":"rafael@apache.org","username":"rafaelweingartner"},"change_message_id":"8b6b5ce0a5fef227b6898fffddbe24182aa8549d","unresolved":false,"context_lines":[{"line_number":339,"context_line":"        for sample in samples:"},{"line_number":340,"context_line":"            if not self._is_gnocchi_activity(sample):"},{"line_number":341,"context_line":"                filtered_samples.append(sample)"},{"line_number":342,"context_line":"                LOG.debug(\"Sample [%s] is not a Gnocchi activity; therefore, \""},{"line_number":343,"context_line":"                          \"we do not filter it out and push it to Gnocchi.\","},{"line_number":344,"context_line":"                          sample)"},{"line_number":345,"context_line":"            else:"},{"line_number":346,"context_line":"                LOG.debug(\"Sample [%s] is a Gnocchi activity; therefore, \""},{"line_number":347,"context_line":"                          \"we filter it out and do not push it to Gnocchi.\","}],"source_content_type":"text/x-python","patch_set":4,"id":"1393ddc9_84a359fc","line":344,"range":{"start_line":342,"start_character":0,"end_line":344,"end_character":33},"in_reply_to":"a6899531_54a76eef","updated":"2022-12-09 11:45:17.000000000","message":"Done","commit_id":"463594b229017b30a1c457aaf6d4cdfef2bb421c"}]}