)]}'
{"/PATCHSET_LEVEL":[{"author":{"_account_id":13252,"name":"Dr. Jens Harbott","display_name":"Jens Harbott (frickler)","email":"frickler@offenerstapel.de","username":"jrosenboom"},"change_message_id":"edd15a09fe1801666442a1824a49b66372ef4894","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":2,"id":"b6ebbe95_85c11168","updated":"2022-06-15 12:53:06.000000000","message":"Do you have a reference to these tuning parameters? The bug report that Brian links doesn\u0027t mention these, either. While tuning devstack certainly is a valid thing, I would ask why python and/or the OpenStack services don\u0027t have sensible defaults. IMO it doesn\u0027t help much if devstack tests works fine but every real world deployment will blow up later anyway.","commit_id":"d0de8c783192fbe5a278b396b130b92a476e062c"},{"author":{"_account_id":5314,"name":"Brian Rosmaita","email":"rosmaita.fossdev@gmail.com","username":"brian-rosmaita"},"change_message_id":"ea421eb60369927d782d914d763cac4a0c9f1873","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":2,"id":"01c39085_448d9955","updated":"2022-06-15 12:22:19.000000000","message":"Might want to add \"Related-bug: #1908805\".  This might give operators some pointers on things to look into in their infrastructure configuration.","commit_id":"d0de8c783192fbe5a278b396b130b92a476e062c"},{"author":{"_account_id":9535,"name":"Gorka Eguileor","email":"geguileo@redhat.com","username":"Gorka"},"change_message_id":"63f3255b46d61bcca067e65c3242acd38e5527a5","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":2,"id":"90eae70e_3af21475","in_reply_to":"01c39085_448d9955","updated":"2022-06-15 18:32:04.000000000","message":"Done","commit_id":"d0de8c783192fbe5a278b396b130b92a476e062c"},{"author":{"_account_id":9535,"name":"Gorka Eguileor","email":"geguileo@redhat.com","username":"Gorka"},"change_message_id":"b8758ec0c57c0cf4cf360742d26da2db53552bcf","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":2,"id":"b4622a08_8b726ba1","in_reply_to":"113a3a64_d90314ab","updated":"2022-06-15 18:33:08.000000000","message":"In the last mallopt call it\u0027s not M_MMAP_THRESHOLD but M_TRIM_THRESHOLD","commit_id":"d0de8c783192fbe5a278b396b130b92a476e062c"},{"author":{"_account_id":9535,"name":"Gorka Eguileor","email":"geguileo@redhat.com","username":"Gorka"},"change_message_id":"63f3255b46d61bcca067e65c3242acd38e5527a5","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":2,"id":"113a3a64_d90314ab","in_reply_to":"b6ebbe95_85c11168","updated":"2022-06-15 18:32:04.000000000","message":"These tuning parameters are presented here https://www.gnu.org/software/libc/manual/html_node/Memory-Allocation-Tunables.html\n\nTuning these options is needed because glibc changed and it now creates 1 memory arena for each native thread instead of sharing it between them like before.\n\nI evaluated setting these in the Python Services themselves and discarded it because it is a bit \"hackish\", as we can\u0027t set these env vars in Python ourselves because by the time the Python code is evaluated glibc has already been loaded and seen the environmental variable values.  The alternative is to call C code directly like this:\n\n  # These values are defined in /usr/include/malloc.h\n  M_TRIM_THRESHOLD \u003d -1\n  M_MMAP_THRESHOLD \u003d -3\n  M_ARENA_MAX \u003d -8\n\n  import ctypes\n  libc \u003d ctypes.CDLL(\u0027libc.so.6\u0027)\n  libc.mallopt(ctypes.c_int(M_ARENA_MAX), ctypes.c_int(1))\n  libc.mallopt(ctypes.c_int(M_MMAP_THRESHOLD), ctypes.c_int(131072))\n  libc.mallopt(ctypes.c_int(M_MMAP_THRESHOLD), ctypes.c_int(262144))\n\nI\u0027m also changing these values in TripleO so that these changes are also applied in those deployments: https://review.opendev.org/c/openstack/tripleo-common/+/845807","commit_id":"d0de8c783192fbe5a278b396b130b92a476e062c"},{"author":{"_account_id":29071,"name":"norman shen","email":"yshxxsjt715@gmail.com","username":"ushen"},"change_message_id":"543edcc375747946d9af16583dc1afaeb0ef9b72","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":3,"id":"38a3ced9_4555f25b","updated":"2022-06-16 01:15:28.000000000","message":"Does it have any performance impact? Is it save to use it for production? Thank you.","commit_id":"a2af4c3dfb0e24a3b81e5b9d8cc6d52901a3e050"},{"author":{"_account_id":9535,"name":"Gorka Eguileor","email":"geguileo@redhat.com","username":"Gorka"},"change_message_id":"b6547082a012919be5c208163facc6302b149bd0","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":3,"id":"f2938971_89f2d631","updated":"2022-06-15 19:20:12.000000000","message":"Have to check performance.json for other services","commit_id":"a2af4c3dfb0e24a3b81e5b9d8cc6d52901a3e050"},{"author":{"_account_id":9535,"name":"Gorka Eguileor","email":"geguileo@redhat.com","username":"Gorka"},"change_message_id":"082e65288ce8abf6ba5d6c403616b2fd55162b6d","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":3,"id":"bf6dee84_7bd9080a","in_reply_to":"38a3ced9_4555f25b","updated":"2022-06-17 10:24:39.000000000","message":"I tested it on the cinder-backup service and there doesn\u0027t seem to have a significant performance impact.\n\nIn my tests I run 2000 backups and 2000 restores with concurrency 10 both with and without the environmental variables and the results were:\n\n- Peak memory usage dropped from 2.9GB to 1.2GB.\n- Memory at rest after the whole process dropped from 2GB to 140MB.\n\nThere were additional tests, but this was the largest one, and in all of them the high water mark issue (memory at rest) was resolved.\n\nPerformance wise the full tests took 1.44% longer with the changes (from 11 hours and 3 minutes to 11 hours and 12 minutes), but these numbers could be caused by external factors such as room temperature (one was run at night while the other during the day, in summer), etc.\n\nI would recommend you do your own independent tests before going to production with the changes, and we would love to hear your results.","commit_id":"a2af4c3dfb0e24a3b81e5b9d8cc6d52901a3e050"},{"author":{"_account_id":29071,"name":"norman shen","email":"yshxxsjt715@gmail.com","username":"ushen"},"change_message_id":"b76cd4a0d8b031eb836f4a22cae6ae9835ff9517","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":3,"id":"b79035ca_7fd5e6ff","in_reply_to":"bf6dee84_7bd9080a","updated":"2022-06-17 10:53:22.000000000","message":"thank you very much for the feedback! We indeed are testing this patch ourselves. We are happy to find the memory usage is dropping. I will report to you later if performance test result comes out.","commit_id":"a2af4c3dfb0e24a3b81e5b9d8cc6d52901a3e050"},{"author":{"_account_id":9535,"name":"Gorka Eguileor","email":"geguileo@redhat.com","username":"Gorka"},"change_message_id":"04c848a5c75369f5189971c2ee89f8280e5081c5","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":4,"id":"93a9030a_f5bb284e","updated":"2022-06-17 10:11:47.000000000","message":"I can\u0027t see any huge memory consumption difference for other services, so changing the patch to only modify c-vol and c-bak.  I\u0027ll do the same for the tripleo patch.","commit_id":"758588af386d983b7e83d3ef23f2d8cc659c7840"},{"author":{"_account_id":9535,"name":"Gorka Eguileor","email":"geguileo@redhat.com","username":"Gorka"},"change_message_id":"c61a5f72d00e08311fea99ae9a7fe27a1c50fc62","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":4,"id":"fcc0d9e4_592a8434","updated":"2022-06-21 10:16:56.000000000","message":"Thanks for the review.","commit_id":"758588af386d983b7e83d3ef23f2d8cc659c7840"},{"author":{"_account_id":10459,"name":"Luigi Toscano","email":"ltoscano@redhat.com","username":"ltoscano"},"change_message_id":"efa3bfc4f25d1d2e4cf3027651aa122b25602efe","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":5,"id":"8b802cec_64a32892","updated":"2022-06-30 13:04:00.000000000","message":"As it seems to be fine and I\u0027m seeing memory issue on older branches, I took the liberty of backporting this to all non-EM branches.","commit_id":"d5af514ac9485009229f3b594bccc09e905782fb"},{"author":{"_account_id":16137,"name":"Tobias Urdin","email":"tobias.urdin@binero.com","username":"tobasco"},"change_message_id":"54d6602a81d0f80b5fdad17db0f52168e83ef2a3","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":5,"id":"cdd51a76_53eaf1ff","updated":"2023-02-21 16:18:23.000000000","message":"Just a quick heads up, this gave us insanely good results with cinder-backup\u0027s memory usage. Going from 90% memory usage down to 10% or lower in some cases, so thanks! 😊","commit_id":"d5af514ac9485009229f3b594bccc09e905782fb"},{"author":{"_account_id":13252,"name":"Dr. Jens Harbott","display_name":"Jens Harbott (frickler)","email":"frickler@offenerstapel.de","username":"jrosenboom"},"change_message_id":"84a89ce6916cdcf049ba3b0716216f5f783d180c","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":5,"id":"cca1fc2e_b101ee2a","updated":"2022-06-21 15:10:00.000000000","message":"Thx. Failure is unrelated, will hopefully be fixed soon","commit_id":"d5af514ac9485009229f3b594bccc09e905782fb"},{"author":{"_account_id":13252,"name":"Dr. Jens Harbott","display_name":"Jens Harbott (frickler)","email":"frickler@offenerstapel.de","username":"jrosenboom"},"change_message_id":"b1faa7d1fcc2cded1af6aec5ca2fcf9ba1babc34","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":5,"id":"5e8aa626_51bebea3","updated":"2022-06-22 04:38:39.000000000","message":"recheck c9s n-v","commit_id":"d5af514ac9485009229f3b594bccc09e905782fb"},{"author":{"_account_id":22873,"name":"Martin Kopec","email":"mkopec@redhat.com","username":"mkopec"},"change_message_id":"87e721ac00c73c253972202f0896d02798870a52","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":5,"id":"f2ea4d83_a52586e7","updated":"2022-06-21 15:36:22.000000000","message":"very nice improvement, thanks, I didn\u0027t +w as cs-9 job is currently being marked as n-v, feel free to +w after that\u0027s in the gate","commit_id":"d5af514ac9485009229f3b594bccc09e905782fb"}],"functions-common":[{"author":{"_account_id":13252,"name":"Dr. Jens Harbott","display_name":"Jens Harbott (frickler)","email":"frickler@offenerstapel.de","username":"jrosenboom"},"change_message_id":"135426532341b9ffa476cffb691f02256ea6bae3","unresolved":true,"context_lines":[{"line_number":1661,"context_line":"    local group\u003d$3"},{"line_number":1662,"context_line":"    if [[ -z \"$user\" ]]; then"},{"line_number":1663,"context_line":"        user\u003d$STACK_USER"},{"line_number":1664,"context_line":"    fi"},{"line_number":1665,"context_line":"    local env_vars\u003d\"$5\""},{"line_number":1666,"context_line":"    if [[ \"$command\" \u003d~ \"uwsgi\" ]] ; then"},{"line_number":1667,"context_line":"        write_uwsgi_user_unit_file $systemd_service \"$cmd\" \"$group\" \"$user\" \"$env_vars\""}],"source_content_type":"application/x-shellscript","patch_set":4,"id":"94ff49e9_e1ded815","line":1664,"updated":"2022-06-17 14:56:47.000000000","message":"Why are you changing this?","commit_id":"758588af386d983b7e83d3ef23f2d8cc659c7840"},{"author":{"_account_id":9535,"name":"Gorka Eguileor","email":"geguileo@redhat.com","username":"Gorka"},"change_message_id":"c61a5f72d00e08311fea99ae9a7fe27a1c50fc62","unresolved":false,"context_lines":[{"line_number":1661,"context_line":"    local group\u003d$3"},{"line_number":1662,"context_line":"    if [[ -z \"$user\" ]]; then"},{"line_number":1663,"context_line":"        user\u003d$STACK_USER"},{"line_number":1664,"context_line":"    fi"},{"line_number":1665,"context_line":"    local env_vars\u003d\"$5\""},{"line_number":1666,"context_line":"    if [[ \"$command\" \u003d~ \"uwsgi\" ]] ; then"},{"line_number":1667,"context_line":"        write_uwsgi_user_unit_file $systemd_service \"$cmd\" \"$group\" \"$user\" \"$env_vars\""}],"source_content_type":"application/x-shellscript","patch_set":4,"id":"64e5dd8d_b962524c","line":1664,"in_reply_to":"94ff49e9_e1ded815","updated":"2022-06-21 10:16:56.000000000","message":"The reason to add this code is because we have cases now where we want to pass the 5th parameter \"env_vars\" and still use the default for user, but the code is horribly wrong.\n\nIt should not replace the existing code but be an addition.\n\nSo the idea is to be able to pass an empty string as the user and make it behave like the group parameter.  It is shown in the lib/cinder change","commit_id":"758588af386d983b7e83d3ef23f2d8cc659c7840"},{"author":{"_account_id":13252,"name":"Dr. Jens Harbott","display_name":"Jens Harbott (frickler)","email":"frickler@offenerstapel.de","username":"jrosenboom"},"change_message_id":"135426532341b9ffa476cffb691f02256ea6bae3","unresolved":true,"context_lines":[{"line_number":1687,"context_line":"# If the command includes shell metachatacters (;\u003c\u003e*) it must be run using a shell"},{"line_number":1688,"context_line":"# If an optional group is provided sg will be used to run the"},{"line_number":1689,"context_line":"# command as that group."},{"line_number":1690,"context_line":"# run_process service \"command-line\" [group] [user]"},{"line_number":1691,"context_line":"function run_process {"},{"line_number":1692,"context_line":"    local service\u003d$1"},{"line_number":1693,"context_line":"    local command\u003d\"$2\""}],"source_content_type":"application/x-shellscript","patch_set":4,"id":"9cf9e33f_657fc452","line":1690,"updated":"2022-06-17 14:56:47.000000000","message":"Please add a short note about the new parameter here","commit_id":"758588af386d983b7e83d3ef23f2d8cc659c7840"},{"author":{"_account_id":9535,"name":"Gorka Eguileor","email":"geguileo@redhat.com","username":"Gorka"},"change_message_id":"c61a5f72d00e08311fea99ae9a7fe27a1c50fc62","unresolved":false,"context_lines":[{"line_number":1687,"context_line":"# If the command includes shell metachatacters (;\u003c\u003e*) it must be run using a shell"},{"line_number":1688,"context_line":"# If an optional group is provided sg will be used to run the"},{"line_number":1689,"context_line":"# command as that group."},{"line_number":1690,"context_line":"# run_process service \"command-line\" [group] [user]"},{"line_number":1691,"context_line":"function run_process {"},{"line_number":1692,"context_line":"    local service\u003d$1"},{"line_number":1693,"context_line":"    local command\u003d\"$2\""}],"source_content_type":"application/x-shellscript","patch_set":4,"id":"d32a08e5_0ac59962","line":1690,"in_reply_to":"9cf9e33f_657fc452","updated":"2022-06-21 10:16:56.000000000","message":"Done","commit_id":"758588af386d983b7e83d3ef23f2d8cc659c7840"}]}