)]}'
{"/PATCHSET_LEVEL":[{"author":{"_account_id":4523,"name":"Eric Harney","email":"eharney@redhat.com","username":"eharney"},"change_message_id":"3e45ff39297bf6a4893eadfaf1f6e34531e3359b","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":1,"id":"b53122d4_f05d9cca","updated":"2026-09-25 14:35:00.000000000","message":"IMO we should let the update to uutils resolve this rather than adding code to work around it.","commit_id":"4056a09b22c4022066aae08daa48e70b15b6ae0d"},{"author":{"_account_id":13425,"name":"Simon Dodsley","email":"simon@everpuredata.com","username":"sdodsley"},"change_message_id":"74ec2f27a03897b1adaac06fb90859aa07e4430b","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":1,"id":"18ee070a_953ca411","updated":"2026-09-25 01:41:02.000000000","message":"Tested on real hardware in the Everpure third-party CI, on an\nUbuntu 26.04 node whose dd is the distro one (uutils coreutils 0.10.0),\nwith two FlashArray iSCSI/multipath backends so that\nretype-with-migration goes through _copy_volume_data.  The CI logs are\npruned after a while, so the evidence is quoted here.\n\nWithout this change (same job, cinder master, 2026-09-24 21:14 UTC):\nboth tempest.api.volume.admin.test_volume_retype\n.VolumeRetypeWithMigrationTest tests failed.  cinder-volume ran\n\n  dd if\u003d/dev/dm-4 of\u003d/dev/dm-0 count\u003d1073741824 bs\u003d1M\n     iflag\u003dcount_bytes,direct oflag\u003ddirect\n\nwhich exited 1 after 0.15 s with\n\n  dd: IO error: Invalid input\n\nand the volume went to migration_status error.  The count\u003d0 probe had\npassed, so the direct flags were used.\n\nWith this change (same job, 2026-09-25 01:19 UTC): 361 tempest tests\npassed, 0 failed, including\n\n  test_available_volume_retype_with_migration      [20.1s] ... ok\n  test_volume_from_snapshot_retype_with_migration  [21.3s] ... ok\n\nThe cinder-volume log shows the new one-block input probe failing on\nevery source device, e.g.\n\n  dd count\u003d1 bs\u003d1M if\u003d/dev/dm-4 of\u003d/dev/null iflag\u003ddirect\n  ... returned: 1 in 0.160s\n\nafter which the copies ran as\n\n  dd if\u003d/dev/dm-4 of\u003d/dev/dm-0 count\u003d1073741824 bs\u003d1M\n     iflag\u003dcount_bytes oflag\u003ddirect\n\nat 295-658 MB/s for 1 GiB volumes, with a separate content checksum\nbefore and after the migration matching.  So the probe alone fixes the\nmigration case; the EINVAL retry in _copy_volume_with_path did not need\nto fire and is exercised by the unit tests only.  Direct writes\n(oflag\u003ddirect) work on this dd, so volume_clear from /dev/zero, which\nonly uses the output flag, is unaffected.\n\nThe same node with GNU dd (our other jobs divert /usr/bin/dd to gnudd)\nalso passes 361/0, so the change does not alter behaviour where direct\nreads work.  Ubuntu bug for the dd itself: LP #2168495.","commit_id":"4056a09b22c4022066aae08daa48e70b15b6ae0d"},{"author":{"_account_id":13425,"name":"Simon Dodsley","email":"simon@everpuredata.com","username":"sdodsley"},"change_message_id":"4bb1412f8e961533322bb1d7596ccba2aa7942e8","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":1,"id":"aee21e1a_fd9767f7","updated":"2026-09-25 00:41:30.000000000","message":"run Everpure CI","commit_id":"4056a09b22c4022066aae08daa48e70b15b6ae0d"},{"author":{"_account_id":13425,"name":"Simon Dodsley","email":"simon@everpuredata.com","username":"sdodsley"},"change_message_id":"95aba55ce1184a31d3e7aaf9d367db2cc0eeaf0f","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":1,"id":"0b9f9adc_150088aa","in_reply_to":"b53122d4_f05d9cca","updated":"2026-09-25 14:46:56.000000000","message":"Thanks Eric.  I\u0027d push back on the framing.  The uutils ``dd`` is how we\nfound this, but the probe is wrong independently of it:\n``\"dd count\u003d0 ... iflag\u003ddirect\"`` opens the source with ``O_DIRECT`` and never\nreads, so any ``dd`` whose direct reads fail after a successful open is\ninvisible to it.  GNU ``dd`` fails the same way with ``EINVAL`` when\n``volume_dd_blocksize`` is not a multiple of the device\u0027s logical sector\nsize, and some network filesystems accept the open and refuse the\nread.  Reading one block is what the probe should always have done;\nthe buffered retry is the belt to those braces.\n\nOn waiting for uutils: there is nothing to wait for yet.  26.04 has\n``rust-coreutils 0.10.0-1ubuntu2~26.04.1`` in both release and updates\n(the updates build landed on 21 September and is still 0.10.0); the\nfixed code, upstream 0.11.0, exists only in the 26.10 development\nseries.  Getting it into 26.04 means a maintainer backporting the\nalignment patch under SRU policy, and the bug asking for that\n(LP #2168495) is New, Undecided and unassigned with no response from\nCanonical.  We have no date.  2027.1 targets 26.04 per the PTI, so the\ngate meets this the day a multi-backend job moves to a resolute\nnodeset; today only a single-LVM smoke job runs there, and a single\nbackend never copies through the host, which is why the gate has not\nseen it.  Even once an SRU lands, nodes installed from release media\nrun the broken dd until they update, and 26.04 images without the\nupdate will be in use for the life of the LTS.\n\nMeanwhile every generic migration, retype with migration and LVM\nclone on a 26.04 node fails in under a second, with a volume in error\nand nothing in the log pointing at ``dd``.\n\nThe change alters nothing where direct reads work.  With GNU ``dd`` the\ncommands run are identical to today (361/0 on our CI either way); with\nthe uutils ``dd`` the copy switches to a buffered read at 295-658 MB/s\n(evidence in my earlier comment).\n\nIf the retry is the part that feels like a workaround, I\u0027m happy to\ndrop it and keep only the probe fix, which is what actually resolved\nthe CI failure.  But I don\u0027t think leaving the probe blind until an\nunscheduled downstream package update lands is the right call for an\nLTS.","commit_id":"4056a09b22c4022066aae08daa48e70b15b6ae0d"}],"cinder/volume/volume_utils.py":[{"author":{"_account_id":36171,"name":"jayaanand borra","display_name":"jayaanand borra","email":"jayaanand.borra@netapp.com","username":"jayaanan","status":"netapp"},"change_message_id":"7432e231ee640e81705562adafd22626a2583dd4","unresolved":true,"context_lines":[{"line_number":483,"context_line":"    \"Invalid input\". Either way the failure is on the first read or"},{"line_number":484,"context_line":"    write, before any data has moved."},{"line_number":485,"context_line":"    \"\"\""},{"line_number":486,"context_line":"    return \u0027invalid\u0027 in (exc.stderr or \u0027\u0027).lower()"},{"line_number":487,"context_line":""},{"line_number":488,"context_line":""},{"line_number":489,"context_line":"def _dd_flags(idirect: bool, odirect: bool, sync: bool,"}],"source_content_type":"text/x-python","patch_set":1,"id":"c5192ccd_6478669f","line":486,"updated":"2026-09-27 17:07:14.000000000","message":"This is true for \"Invalid argument\" and \"Invalid input\", and also for\n\"invalid option\", \"invalid number\", and \"invalid conversion\". The docstring\non lines 483-484 says the failure is the first read or write, before any\ndata has moved. Nothing here looks at records written or bytes copied.\n\nPlease match only those two phrases, and retry only when dd reports that\nnothing was written (GNU dd prints \"0 bytes\" / \"0+0 records out\" on a\nfailed transfer). If stderr shows data was written, re-raise. The uutils\n0.10.0 failure has no record line; that specific message can still be\nretried.","commit_id":"4056a09b22c4022066aae08daa48e70b15b6ae0d"},{"author":{"_account_id":13425,"name":"Simon Dodsley","email":"simon@everpuredata.com","username":"sdodsley"},"change_message_id":"96d42d4795d10dc3fcaaf77c677c8a1e68f07208","unresolved":false,"context_lines":[{"line_number":483,"context_line":"    \"Invalid input\". Either way the failure is on the first read or"},{"line_number":484,"context_line":"    write, before any data has moved."},{"line_number":485,"context_line":"    \"\"\""},{"line_number":486,"context_line":"    return \u0027invalid\u0027 in (exc.stderr or \u0027\u0027).lower()"},{"line_number":487,"context_line":""},{"line_number":488,"context_line":""},{"line_number":489,"context_line":"def _dd_flags(idirect: bool, odirect: bool, sync: bool,"}],"source_content_type":"text/x-python","patch_set":1,"id":"08775de3_9fd0960a","line":486,"in_reply_to":"c5192ccd_6478669f","updated":"2026-09-28 10:10:11.000000000","message":"Both points are right, and rather than tighten the retry I am\nremoving it in the next patch set.  On the hardware run the one-block\nprobe caught the uutils dd before any copy started and the retry never\nfired, and it is the probe that was wrong on its own terms: count\u003d0\nopens the source with O_DIRECT and never reads.  The change becomes\nthe probe fix alone, with the tests and release note trimmed to\nmatch.  If a dd ever turns up whose direct reads pass a one-block\nprobe and fail later, a retry can be argued for then, with the\nrecords-out check you describe.","commit_id":"4056a09b22c4022066aae08daa48e70b15b6ae0d"},{"author":{"_account_id":36171,"name":"jayaanand borra","display_name":"jayaanand borra","email":"jayaanand.borra@netapp.com","username":"jayaanan","status":"netapp"},"change_message_id":"7432e231ee640e81705562adafd22626a2583dd4","unresolved":true,"context_lines":[{"line_number":542,"context_line":"                    \"(%(err)s); retrying with buffered I/O.\","},{"line_number":543,"context_line":"                    {\u0027src\u0027: srcstr, \u0027dest\u0027: deststr,"},{"line_number":544,"context_line":"                     \u0027err\u0027: (exc.stderr or \u0027\u0027).strip()})"},{"line_number":545,"context_line":"        execute(*cmd, *_dd_flags(False, False, True, sparse),"},{"line_number":546,"context_line":"                run_as_root\u003dTrue)"},{"line_number":547,"context_line":"    duration \u003d timeutils.delta_seconds(start_time, timeutils.utcnow())"},{"line_number":548,"context_line":""}],"source_content_type":"text/x-python","patch_set":1,"id":"4d14deb2_5d54b030","line":545,"updated":"2026-09-27 17:07:14.000000000","message":"`True` is the sync argument, so this second dd is always\n`iflag\u003dcount_bytes` plus `conv\u003dfdatasync` (and `sparse` when the caller\nasked for it), even when the caller passed sync\u003dFalse. That part is right\nfor _copy_volume_data: the call at manager.py:2341 does not pass sync, and\nthe finally at 2352 force-detaches. Bug 1363016 is lost page cache on\ndetach. Do not drop the fdatasync if this retry stays.\n\nThe comment on lines 537-540 is not enforced. This execute always starts\nagain at byte 0 of deststr. For generic migration that second copy runs\nwhile both attachments from _copy_volume_data are still held. If this\nexecute fails, its exception replaces the direct-I/O exception; the first\nerror is only the warning above. Please log it as the cause, or re-raise\nfrom it.","commit_id":"4056a09b22c4022066aae08daa48e70b15b6ae0d"},{"author":{"_account_id":13425,"name":"Simon Dodsley","email":"simon@everpuredata.com","username":"sdodsley"},"change_message_id":"96d42d4795d10dc3fcaaf77c677c8a1e68f07208","unresolved":false,"context_lines":[{"line_number":542,"context_line":"                    \"(%(err)s); retrying with buffered I/O.\","},{"line_number":543,"context_line":"                    {\u0027src\u0027: srcstr, \u0027dest\u0027: deststr,"},{"line_number":544,"context_line":"                     \u0027err\u0027: (exc.stderr or \u0027\u0027).strip()})"},{"line_number":545,"context_line":"        execute(*cmd, *_dd_flags(False, False, True, sparse),"},{"line_number":546,"context_line":"                run_as_root\u003dTrue)"},{"line_number":547,"context_line":"    duration \u003d timeutils.delta_seconds(start_time, timeutils.utcnow())"},{"line_number":548,"context_line":""}],"source_content_type":"text/x-python","patch_set":1,"id":"20fbd198_3beb7d93","line":545,"in_reply_to":"4d14deb2_5d54b030","updated":"2026-09-28 10:10:11.000000000","message":"Acknowledged","commit_id":"4056a09b22c4022066aae08daa48e70b15b6ae0d"}]}