)]}'
{"specs/victoria/support-modern-compression-algorithms-in-cinder-backup.rst":[{"author":{"_account_id":4523,"name":"Eric Harney","email":"eharney@redhat.com","username":"eharney"},"change_message_id":"1ed8727c643d9e3c96b8f8df040be830581c5e97","unresolved":false,"context_lines":[{"line_number":41,"context_line":"Some general benchmark result is available on the website introduced above, or"},{"line_number":42,"context_line":"lzbench website."},{"line_number":43,"context_line":"https://github.com/inikep/lzbench/"},{"line_number":44,"context_line":""},{"line_number":45,"context_line":"Use Cases"},{"line_number":46,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"},{"line_number":47,"context_line":""}],"source_content_type":"text/x-rst","patch_set":3,"id":"ff570b3c_351a4ecb","line":44,"updated":"2020-05-18 13:12:27.000000000","message":"What\u0027s the benefit of adding both lz4 and zstd algorithms?\n\nIMO Cinder should be somewhat opinionated about which algorithms are best for cinder-backup, and only add more when there\u0027s a clear benefit.","commit_id":"84ac1d3ba34a905497d34ea6d197e2fbab5c67eb"},{"author":{"_account_id":25397,"name":"Kazufumi Noto","email":"noto.kazufumi@gmail.com","username":"notok"},"change_message_id":"f3cfcc790970c00c3886b0c9b960f4f4da4d8ef6","unresolved":false,"context_lines":[{"line_number":41,"context_line":"Some general benchmark result is available on the website introduced above, or"},{"line_number":42,"context_line":"lzbench website."},{"line_number":43,"context_line":"https://github.com/inikep/lzbench/"},{"line_number":44,"context_line":""},{"line_number":45,"context_line":"Use Cases"},{"line_number":46,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"},{"line_number":47,"context_line":""}],"source_content_type":"text/x-rst","patch_set":3,"id":"ff570b3c_ad7e1216","line":44,"in_reply_to":"ff570b3c_351a4ecb","updated":"2020-05-18 15:09:34.000000000","message":"As far as I checked, zstd is better in compression ratio with similar speed compared to lz4,\nbut lz4 is more generally available and supported.\n(e.g. Rpm package for lz4 python binding is available in epel repo, but not for zstd.)\n\nOf cource, zstd python binding is available from pypi.\nSo adding only zstd may be one choice. I want to hear more opinions from cinder members.","commit_id":"84ac1d3ba34a905497d34ea6d197e2fbab5c67eb"},{"author":{"_account_id":25397,"name":"Kazufumi Noto","email":"noto.kazufumi@gmail.com","username":"notok"},"change_message_id":"a446bf6184984d1fa2d54d3ae8a1bfc56cabbca0","unresolved":false,"context_lines":[{"line_number":41,"context_line":"Some general benchmark result is available on the website introduced above, or"},{"line_number":42,"context_line":"lzbench website."},{"line_number":43,"context_line":"https://github.com/inikep/lzbench/"},{"line_number":44,"context_line":""},{"line_number":45,"context_line":"Use Cases"},{"line_number":46,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"},{"line_number":47,"context_line":""}],"source_content_type":"text/x-rst","patch_set":3,"id":"bf51134e_74f26871","line":44,"in_reply_to":"ff570b3c_ad7e1216","updated":"2020-07-04 12:05:04.000000000","message":"Brian also says we should pick one, and zstd is better in performance.\nSo I\u0027ll change this to only add zstd algorithm.\n\nThe concern is that the package for python-zstd is not provided for ubuntu and fedora. I\u0027ll try to explain if global-requirement team point on this.\n(If they do not accept, I may switch to lz4.)","commit_id":"84ac1d3ba34a905497d34ea6d197e2fbab5c67eb"},{"author":{"_account_id":28948,"name":"Liang Fang","email":"liang.a.fang@intel.com","username":"liang"},"change_message_id":"7d27c8cac7b1c0139262ca0f273dcaecf22ddc1d","unresolved":false,"context_lines":[{"line_number":109,"context_line":"none                         203.8             112.4 17218167818         143.5"},{"line_number":110,"context_line":"gzip                         739.8            1440.0  6658990044          55.5"},{"line_number":111,"context_line":"bz2                         2076.4             735.4  6480248772          54.0"},{"line_number":112,"context_line":"lz4(to be added)             207.2             114.8  7903490364          65.9"},{"line_number":113,"context_line":"zstd(to be added)            251.0             102.6  6284156414          52.4"},{"line_number":114,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d \u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d \u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d \u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d \u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"},{"line_number":115,"context_line":""},{"line_number":116,"context_line":""}],"source_content_type":"text/x-rst","patch_set":3,"id":"bf51134e_60064d23","line":113,"range":{"start_line":112,"start_character":0,"end_line":113,"end_character":78},"updated":"2020-07-01 15:13:06.000000000","message":"Looks good. But it would be better if you can put CPU consumption data here. 8 cores are all 100% busy in all the tests, right? I remember some may only use one core.\n\nAnother point is *famous* algorithms (like gzip) is well supported by hardware accelerator, can be offload easily. It would be good to mention this here (gzip may be not so bad as listed here if offloaded to hardware^-^)\n\nI\u0027m not saying I don\u0027t support this spec, just saying this spec can add more info if you like to do.","commit_id":"84ac1d3ba34a905497d34ea6d197e2fbab5c67eb"},{"author":{"_account_id":25397,"name":"Kazufumi Noto","email":"noto.kazufumi@gmail.com","username":"notok"},"change_message_id":"a446bf6184984d1fa2d54d3ae8a1bfc56cabbca0","unresolved":false,"context_lines":[{"line_number":109,"context_line":"none                         203.8             112.4 17218167818         143.5"},{"line_number":110,"context_line":"gzip                         739.8            1440.0  6658990044          55.5"},{"line_number":111,"context_line":"bz2                         2076.4             735.4  6480248772          54.0"},{"line_number":112,"context_line":"lz4(to be added)             207.2             114.8  7903490364          65.9"},{"line_number":113,"context_line":"zstd(to be added)            251.0             102.6  6284156414          52.4"},{"line_number":114,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d \u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d \u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d \u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d \u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"},{"line_number":115,"context_line":""},{"line_number":116,"context_line":""}],"source_content_type":"text/x-rst","patch_set":3,"id":"bf51134e_34df50d7","line":113,"range":{"start_line":112,"start_character":0,"end_line":113,"end_character":78},"in_reply_to":"bf51134e_60064d23","updated":"2020-07-04 12:05:04.000000000","message":"On my test, CPU usage for all algorithm was about 100%~110% where max usage is 800% (\u003d8cores * 100%).\n(i.e. Using only one core for compression.)\n\nThe zlib, bzip2 and zstd (version 1.4.4.0) python libraries does not support multi-thread compression.\nBut I checked again now and noticed zstd added support for multi-thread compression at version 1.4.4.1.\n\nI will re-test with new zstd python library and add cpu consumption data here. (Maybe on monday.)\nAnd I\u0027ll add notes about hardware accelarator then.","commit_id":"84ac1d3ba34a905497d34ea6d197e2fbab5c67eb"},{"author":{"_account_id":28948,"name":"Liang Fang","email":"liang.a.fang@intel.com","username":"liang"},"change_message_id":"7d27c8cac7b1c0139262ca0f273dcaecf22ddc1d","unresolved":false,"context_lines":[{"line_number":117,"context_line":"Other deployer impact"},{"line_number":118,"context_line":"---------------------"},{"line_number":119,"context_line":""},{"line_number":120,"context_line":"None."},{"line_number":121,"context_line":""},{"line_number":122,"context_line":"Developer impact"},{"line_number":123,"context_line":"----------------"}],"source_content_type":"text/x-rst","patch_set":3,"id":"bf51134e_5b34dade","line":120,"range":{"start_line":120,"start_character":0,"end_line":120,"end_character":5},"updated":"2020-07-01 15:13:06.000000000","message":"Should mention package requirement here?","commit_id":"84ac1d3ba34a905497d34ea6d197e2fbab5c67eb"},{"author":{"_account_id":28948,"name":"Liang Fang","email":"liang.a.fang@intel.com","username":"liang"},"change_message_id":"47d15c523785f4fa2e4c8b2e4c7409026ff632f0","unresolved":false,"context_lines":[{"line_number":20,"context_line":"the volume data during backup and restore operation."},{"line_number":21,"context_line":""},{"line_number":22,"context_line":"It should support other compression algorithms as well because efficiency of"},{"line_number":23,"context_line":"compression entirely depends on cloud provider\u0027s environment and choice."},{"line_number":24,"context_line":"Some may want fast compression compromising the compression ratio,"},{"line_number":25,"context_line":"so to provide more choices to cloud provider,"},{"line_number":26,"context_line":"it will be good to add support of other algorithm like zstd."},{"line_number":27,"context_line":""},{"line_number":28,"context_line":"Zstandard(zstd) gives high compression ratio(comparable to zlib)"},{"line_number":29,"context_line":"with very much fast speed."}],"source_content_type":"text/x-rst","patch_set":4,"id":"bf51134e_102586c2","line":26,"range":{"start_line":23,"start_character":65,"end_line":26,"end_character":60},"updated":"2020-07-13 09:20:15.000000000","message":"every line should be as many as 80 characters. Please also check other paragraphs.","commit_id":"6b0ddae39b184ea0d6e5a4119bf0f09c587685df"},{"author":{"_account_id":5314,"name":"Brian Rosmaita","email":"rosmaita.fossdev@gmail.com","username":"brian-rosmaita"},"change_message_id":"77a4930af15b2286fd7dfbf0bebd1d40c3de1247","unresolved":false,"context_lines":[{"line_number":134,"context_line":""},{"line_number":135,"context_line":"Work Items"},{"line_number":136,"context_line":"----------"},{"line_number":137,"context_line":""},{"line_number":138,"context_line":"* Add the package details to cinder/requirements.txt."},{"line_number":139,"context_line":"* Implement the code to add support of zstd compression."},{"line_number":140,"context_line":"* Documents need to be updated to reflect the added support."}],"source_content_type":"text/x-rst","patch_set":4,"id":"bf51134e_591f786f","line":137,"updated":"2020-07-07 16:42:25.000000000","message":"* Propose python-zstd to the OpenStack global-requirements\n\nYou can use this as a sample of how to do it: https://opendev.org/openstack/requirements/commit/9b1ba719b75e1856ff6d897df847e401225babe2","commit_id":"6b0ddae39b184ea0d6e5a4119bf0f09c587685df"},{"author":{"_account_id":25397,"name":"Kazufumi Noto","email":"noto.kazufumi@gmail.com","username":"notok"},"change_message_id":"c67d98b4b2f3e0d4570ea1c4dacef7bf4f1e62a8","unresolved":false,"context_lines":[{"line_number":134,"context_line":""},{"line_number":135,"context_line":"Work Items"},{"line_number":136,"context_line":"----------"},{"line_number":137,"context_line":""},{"line_number":138,"context_line":"* Add the package details to cinder/requirements.txt."},{"line_number":139,"context_line":"* Implement the code to add support of zstd compression."},{"line_number":140,"context_line":"* Documents need to be updated to reflect the added support."}],"source_content_type":"text/x-rst","patch_set":4,"id":"bf51134e_fac73711","line":137,"in_reply_to":"bf51134e_591f786f","updated":"2020-07-09 12:30:11.000000000","message":"Thanks for your advice. I proposed as follows.\nhttps://review.opendev.org/#/c/740259/","commit_id":"6b0ddae39b184ea0d6e5a4119bf0f09c587685df"},{"author":{"_account_id":5314,"name":"Brian Rosmaita","email":"rosmaita.fossdev@gmail.com","username":"brian-rosmaita"},"change_message_id":"77a4930af15b2286fd7dfbf0bebd1d40c3de1247","unresolved":false,"context_lines":[{"line_number":155,"context_line":""},{"line_number":156,"context_line":"The following documents need to be updated to reflect the added support:"},{"line_number":157,"context_line":""},{"line_number":158,"context_line":"* cinder sample configuration file in configuration reference."},{"line_number":159,"context_line":""},{"line_number":160,"context_line":"References"},{"line_number":161,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"}],"source_content_type":"text/x-rst","patch_set":4,"id":"bf51134e_79da3ce9","line":158,"updated":"2020-07-07 16:42:25.000000000","message":"(The sample config will be updated automatically when you change the \u0027choices\u0027 on the config option definition)","commit_id":"6b0ddae39b184ea0d6e5a4119bf0f09c587685df"},{"author":{"_account_id":5314,"name":"Brian Rosmaita","email":"rosmaita.fossdev@gmail.com","username":"brian-rosmaita"},"change_message_id":"77a4930af15b2286fd7dfbf0bebd1d40c3de1247","unresolved":false,"context_lines":[{"line_number":156,"context_line":"The following documents need to be updated to reflect the added support:"},{"line_number":157,"context_line":""},{"line_number":158,"context_line":"* cinder sample configuration file in configuration reference."},{"line_number":159,"context_line":""},{"line_number":160,"context_line":"References"},{"line_number":161,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"},{"line_number":162,"context_line":""}],"source_content_type":"text/x-rst","patch_set":4,"id":"bf51134e_99df50d5","line":159,"updated":"2020-07-07 16:42:25.000000000","message":"You\u0027ll also want to update doc/source/admin/blockstorage-volume-backups.rst","commit_id":"6b0ddae39b184ea0d6e5a4119bf0f09c587685df"},{"author":{"_account_id":25397,"name":"Kazufumi Noto","email":"noto.kazufumi@gmail.com","username":"notok"},"change_message_id":"c67d98b4b2f3e0d4570ea1c4dacef7bf4f1e62a8","unresolved":false,"context_lines":[{"line_number":156,"context_line":"The following documents need to be updated to reflect the added support:"},{"line_number":157,"context_line":""},{"line_number":158,"context_line":"* cinder sample configuration file in configuration reference."},{"line_number":159,"context_line":""},{"line_number":160,"context_line":"References"},{"line_number":161,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"},{"line_number":162,"context_line":""}],"source_content_type":"text/x-rst","patch_set":4,"id":"bf51134e_7a77a7d5","line":159,"in_reply_to":"bf51134e_99df50d5","updated":"2020-07-09 12:30:11.000000000","message":"OK. I\u0027ve already fixed the file in https://review.opendev.org/#/c/726765/2 .\n\nAnd I\u0027ll fix the code to add only zstd (not with lz4).","commit_id":"6b0ddae39b184ea0d6e5a4119bf0f09c587685df"},{"author":{"_account_id":11904,"name":"Sean McGinnis","email":"sean.mcginnis@gmail.com","username":"SeanM"},"change_message_id":"de0fd69f192e8f45f9fcfc3a0af6173e0629e7fb","unresolved":false,"context_lines":[{"line_number":75,"context_line":"Other end user impact"},{"line_number":76,"context_line":"---------------------"},{"line_number":77,"context_line":""},{"line_number":78,"context_line":"None."},{"line_number":79,"context_line":""},{"line_number":80,"context_line":"Performance Impact"},{"line_number":81,"context_line":"------------------"}],"source_content_type":"text/x-rst","patch_set":5,"id":"bf51134e_95903a45","line":78,"range":{"start_line":78,"start_character":0,"end_line":78,"end_character":5},"updated":"2020-07-15 14:16:55.000000000","message":"One potential impact that should be called out here is that this could result in someone creating a backup in one cloud that has zstd enabled, then fail to restore that backup in a cloud that is running older code that does not have it.","commit_id":"035034fffc817c01eb57804567824e9ee4642d02"},{"author":{"_account_id":11904,"name":"Sean McGinnis","email":"sean.mcginnis@gmail.com","username":"SeanM"},"change_message_id":"de0fd69f192e8f45f9fcfc3a0af6173e0629e7fb","unresolved":false,"context_lines":[{"line_number":153,"context_line":""},{"line_number":154,"context_line":"The following documents need to be updated to reflect the added support:"},{"line_number":155,"context_line":""},{"line_number":156,"context_line":"* cinder sample configuration file in configuration reference."},{"line_number":157,"context_line":""},{"line_number":158,"context_line":"References"},{"line_number":159,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"}],"source_content_type":"text/x-rst","patch_set":5,"id":"bf51134e_d5e472e2","line":156,"updated":"2020-07-15 14:16:55.000000000","message":"Should probably at least add a note to https://docs.openstack.org/cinder/latest/admin/blockstorage-volume-backups-export-import.html noting this new compression option.","commit_id":"035034fffc817c01eb57804567824e9ee4642d02"}]}