)]}'
{"/COMMIT_MSG":[{"author":{"_account_id":15343,"name":"Tim Burke","email":"tburke@nvidia.com","username":"tburke"},"change_message_id":"1892ae2ff409500b9d03444a9ba38f743eb82c9f","unresolved":true,"context_lines":[{"line_number":4,"context_line":"Commit:     Christian Schwede \u003ccschwede@mailbox.org\u003e"},{"line_number":5,"context_line":"CommitDate: 2026-08-14 17:48:40 +0200"},{"line_number":6,"context_line":""},{"line_number":7,"context_line":"Release the GIL during liberasurecode encode/decode/reconstruct"},{"line_number":8,"context_line":""},{"line_number":9,"context_line":"Holding the GIL for the whole C computation kept callers from overlapping"},{"line_number":10,"context_line":"erasure coding with other work (e.g. Swift\u0027s proxy on an eventlet hub) or"}],"source_content_type":"text/x-gerrit-commit-message","patch_set":1,"id":"e8078103_26757a61","line":7,"updated":"2026-08-14 17:16:30.000000000","message":"But there\u0027s still a global lock down in liberasurecode (\u003e\u003d1.8.0) that\u0027s held for each of these operations, yeah? See https://review.opendev.org/c/openstack/liberasurecode/+/929847\n\nI think we need something like https://review.opendev.org/c/openstack/liberasurecode/+/982505 and https://review.opendev.org/c/openstack/pyeclib/+/983666 to get full advantage of this change.\n\nhttps://review.opendev.org/c/openstack/pyeclib/+/991506 is another good one to prove out that we really get the concurrent workload we\u0027re hoping for.","commit_id":"6db7a108ce64fdc21e86ad1fd53352321992d0d7"},{"author":{"_account_id":15343,"name":"Tim Burke","email":"tburke@nvidia.com","username":"tburke"},"change_message_id":"c52104c1a1cf4bf69cb11202f1f2359215c17311","unresolved":true,"context_lines":[{"line_number":4,"context_line":"Commit:     Christian Schwede \u003ccschwede@mailbox.org\u003e"},{"line_number":5,"context_line":"CommitDate: 2026-08-14 17:48:40 +0200"},{"line_number":6,"context_line":""},{"line_number":7,"context_line":"Release the GIL during liberasurecode encode/decode/reconstruct"},{"line_number":8,"context_line":""},{"line_number":9,"context_line":"Holding the GIL for the whole C computation kept callers from overlapping"},{"line_number":10,"context_line":"erasure coding with other work (e.g. Swift\u0027s proxy on an eventlet hub) or"}],"source_content_type":"text/x-gerrit-commit-message","patch_set":1,"id":"52eedf33_7268a8fc","line":7,"in_reply_to":"7e884f51_6eb463df","updated":"2026-08-18 01:37:04.000000000","message":"Oh, and on that note -- confirmed that the GIL limits us *without* this patch:\n```\n% python -m pyeclib.cli.bench --ec-type liberasurecode_rs_vand -t 2\nUsing 10 data + 5 parity with 2 unavailable frags\nliberasurecode_rs_vand (encode): 414.7MB/s\nliberasurecode_rs_vand (decode): 394.5MB/s\n% python -m pyeclib.cli.bench --ec-type liberasurecode_rs_vand -t 4\nUsing 10 data + 5 parity with 2 unavailable frags\nliberasurecode_rs_vand (encode): 423.7MB/s\nliberasurecode_rs_vand (decode): 401.3MB/s\n```","commit_id":"6db7a108ce64fdc21e86ad1fd53352321992d0d7"},{"author":{"_account_id":15343,"name":"Tim Burke","email":"tburke@nvidia.com","username":"tburke"},"change_message_id":"e505205d0f000dbf61d6c731eb010c7de32ed3af","unresolved":true,"context_lines":[{"line_number":4,"context_line":"Commit:     Christian Schwede \u003ccschwede@mailbox.org\u003e"},{"line_number":5,"context_line":"CommitDate: 2026-08-14 17:48:40 +0200"},{"line_number":6,"context_line":""},{"line_number":7,"context_line":"Release the GIL during liberasurecode encode/decode/reconstruct"},{"line_number":8,"context_line":""},{"line_number":9,"context_line":"Holding the GIL for the whole C computation kept callers from overlapping"},{"line_number":10,"context_line":"erasure coding with other work (e.g. Swift\u0027s proxy on an eventlet hub) or"}],"source_content_type":"text/x-gerrit-commit-message","patch_set":1,"id":"7e884f51_6eb463df","line":7,"in_reply_to":"e8078103_26757a61","updated":"2026-08-15 21:18:48.000000000","message":"Oh, right, but it\u0027s just a *read* lock and we can take out as many of those as we like, provided we\u0027re done creating/deleting instances!\n\nDefinitely see the multithreading working out like we hope when I cherry-pick the threaded-bench-command patch:\n```\n% python -m pyeclib.cli.bench --ec-type liberasurecode_rs_vand\nUsing 10 data + 5 parity with 2 unavailable frags\nliberasurecode_rs_vand (encode): 425.7MB/s\nliberasurecode_rs_vand (decode): 392.4MB/s\n% python -m pyeclib.cli.bench --ec-type liberasurecode_rs_vand -t 2\nUsing 10 data + 5 parity with 2 unavailable frags\nliberasurecode_rs_vand (encode): 790.7MB/s\nliberasurecode_rs_vand (decode): 728.3MB/s\n% python -m pyeclib.cli.bench --ec-type liberasurecode_rs_vand -t 4\nUsing 10 data + 5 parity with 2 unavailable frags\nliberasurecode_rs_vand (encode): 1484.4MB/s\nliberasurecode_rs_vand (decode): 1443.4MB/s\n```","commit_id":"6db7a108ce64fdc21e86ad1fd53352321992d0d7"}],"src/pyeclib_c/pyeclib_c.c":[{"author":{"_account_id":15343,"name":"Tim Burke","email":"tburke@nvidia.com","username":"tburke"},"change_message_id":"e505205d0f000dbf61d6c731eb010c7de32ed3af","unresolved":true,"context_lines":[{"line_number":536,"context_line":""},{"line_number":537,"context_line":"  Py_BEGIN_ALLOW_THREADS"},{"line_number":538,"context_line":"  ret \u003d liberasurecode_encode(pyeclib_handle-\u003eec_desc, data, data_len, \u0026encoded_data, \u0026encoded_parity, \u0026fragment_len);"},{"line_number":539,"context_line":"  Py_END_ALLOW_THREADS"},{"line_number":540,"context_line":"  if (ret \u003c 0) {"},{"line_number":541,"context_line":"    pyeclib_c_seterr(ret, \"pyeclib_c_encode\");"},{"line_number":542,"context_line":"    return NULL;"}],"source_content_type":"text/x-csrc","patch_set":1,"id":"b27b4c74_b79b7386","line":539,"updated":"2026-08-15 21:18:48.000000000","message":"I think this is only safe with liberasurecode\u003e\u003d1.7.1, and we\u0027ll want version guards similar to what we have for [flagging the module as GIL-not-used for free-threaded builds](https://review.opendev.org/c/openstack/pyeclib/+/974263):\n\n```\nif (liberasurecode_get_version() \u003e 0x010701) {\n  Py_BEGIN_ALLOW_THREADS\n  ret \u003d liberasurecode_encode(pyeclib_handle-\u003eec_desc, data, data_len, \u0026encoded_data, \u0026encoded_parity, \u0026fragment_len);\n  Py_END_ALLOW_THREADS\n} else {\n  ret \u003d liberasurecode_encode(pyeclib_handle-\u003eec_desc, data, data_len, \u0026encoded_data, \u0026encoded_parity, \u0026fragment_len);\n}\n```\n\nSpeaking of free-threaded builds, looks like this is still a good idea there -- though I\u0027m still not entirely sure if it\u0027s strictly *necessary*. From https://docs.python.org/3/c-api/threads.html#detaching-the-thread-state-from-extension-code :\n\n\u003e On a free-threaded build, the GIL is usually out of the question, but **detaching the thread state is still required**, because the interpreter periodically needs to block all threads to get a consistent view of Python objects without the risk of race conditions. For example, CPython currently suspends all threads for a short period of time while running the garbage collector.","commit_id":"6db7a108ce64fdc21e86ad1fd53352321992d0d7"},{"author":{"_account_id":6968,"name":"Christian Schwede","email":"cschwede@nvidia.com","username":"cschwede"},"change_message_id":"aa22c154fe829604c6493917db06e4cc8e6925e4","unresolved":false,"context_lines":[{"line_number":536,"context_line":""},{"line_number":537,"context_line":"  Py_BEGIN_ALLOW_THREADS"},{"line_number":538,"context_line":"  ret \u003d liberasurecode_encode(pyeclib_handle-\u003eec_desc, data, data_len, \u0026encoded_data, \u0026encoded_parity, \u0026fragment_len);"},{"line_number":539,"context_line":"  Py_END_ALLOW_THREADS"},{"line_number":540,"context_line":"  if (ret \u003c 0) {"},{"line_number":541,"context_line":"    pyeclib_c_seterr(ret, \"pyeclib_c_encode\");"},{"line_number":542,"context_line":"    return NULL;"}],"source_content_type":"text/x-csrc","patch_set":1,"id":"2e6b0d2f_2e591af2","line":539,"in_reply_to":"b27b4c74_b79b7386","updated":"2026-08-18 09:14:26.000000000","message":"That\u0027s a nice idea, thx!","commit_id":"6db7a108ce64fdc21e86ad1fd53352321992d0d7"}]}