)]}'
{"/PATCHSET_LEVEL":[{"author":{"_account_id":16137,"name":"Tobias Urdin","email":"tobias.urdin@binero.com","username":"tobasco"},"change_message_id":"9fdf3402b97472edc6a0c8f1d6ca8b6d6baafd8f","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":3,"id":"6e96ea1f_e801fc91","updated":"2024-09-26 14:36:28.000000000","message":"As an operator this sounds great, the main thing I\u0027m worried about is evaluating existing images and change the ones applicable from `raw` to `gpt` but this clearly mentioned the potential plans for that.\n\nThat tooling would need to be carefully crafted because for clouds that have a lot of end-user owned images if there is no good way to do that discovery and changing disk_format I suspect a majority will just disable the safety if it interferes with users too much.","commit_id":"4b229658ce1b52a5dd7fb2b0c5590c40edad924d"},{"author":{"_account_id":10342,"name":"Jay Faulkner","display_name":"JayF","email":"jay@jvf.cc","username":"JayF","status":"youtube.com/@oss-gr / podcast.gr-oss.io"},"change_message_id":"3d539b667d80ca099a159c8c881a22769d0e3334","unresolved":true,"context_lines":[],"source_content_type":"","patch_set":3,"id":"93c2722b_0370eb6e","updated":"2024-09-25 16:40:09.000000000","message":"I think most of my disagreement about the contents of the spec mainly boil down to wishing we had the ability to distinguish \"garbage\" from \"valid data\", but that\u0027s pretty much not possible or at least  unreasonable to implement.\n\nWe do need to be certain the docs updates match what we do, and then make loud noises about the changes so operators know what to expect. I am happy to help with this (honestly we could probably have a whole OpenInfra live episode on images and the last 3 months) once we get there.","commit_id":"4b229658ce1b52a5dd7fb2b0c5590c40edad924d"},{"author":{"_account_id":10342,"name":"Jay Faulkner","display_name":"JayF","email":"jay@jvf.cc","username":"JayF","status":"youtube.com/@oss-gr / podcast.gr-oss.io"},"change_message_id":"69f9054961f77c5576e88675381379b25d8e52c4","unresolved":true,"context_lines":[],"source_content_type":"","patch_set":3,"id":"03578d51_9955f3ba","updated":"2024-09-24 22:30:00.000000000","message":"I think the story being told here is incomplete for Ironic use cases, and specifically around having proper image types for all images that are currently in use by Ironic (e.g. squashfs images for our kickstart driver, initramfs/squashfs/ramdisk images for IPA, kernel images for IPA).\n\nWhile it\u0027s laudable to try and chip away at raw being \"everything else\", I think there\u0027s value in mapping out the whole story to ensure we\u0027re covering all use cases -- including proposing new image formats for things like kernel and ramdisk -- since aki/ari are no longer being used for that (and likely never should have been).\n\nI\u0027m excited about what this could enable: Ironic specifically may want to support new image types in the future like tarball images which would be extracted into a filesystem, or kickstart files being treated as images. While obviously those ideas would need to be specified on their own, the idea of being able to be descriptive about what they are would significantly reduce the risk of implementing new ideas like that (since we wouldn\u0027t just be proposing to add those weird images types onto the pile).","commit_id":"4b229658ce1b52a5dd7fb2b0c5590c40edad924d"},{"author":{"_account_id":9303,"name":"Abhishek Kekane","email":"akekane@redhat.com","username":"abhishekkekane"},"change_message_id":"799d9764d3c24b1bc152a2a55d4809e95948fba7","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":3,"id":"6a43107e_0c4f9aa2","updated":"2024-07-31 14:24:35.000000000","message":"I think you need to add something to index.rst similar to https://review.opendev.org/c/openstack/glance-specs/+/917284","commit_id":"4b229658ce1b52a5dd7fb2b0c5590c40edad924d"},{"author":{"_account_id":10342,"name":"Jay Faulkner","display_name":"JayF","email":"jay@jvf.cc","username":"JayF","status":"youtube.com/@oss-gr / podcast.gr-oss.io"},"change_message_id":"dce75d7a1e94a79869f61a05a03dcd759911df94","unresolved":true,"context_lines":[],"source_content_type":"","patch_set":3,"id":"1c9869ad_ab9948d3","updated":"2024-09-25 14:37:49.000000000","message":"Just noting as a side note to this spec: Ironic is evaluating OCI-style tarball images as a supported thing in standalone mode during the PTG. Anyone interested in that is welcome to come by that PTG session.","commit_id":"4b229658ce1b52a5dd7fb2b0c5590c40edad924d"},{"author":{"_account_id":10342,"name":"Jay Faulkner","display_name":"JayF","email":"jay@jvf.cc","username":"JayF","status":"youtube.com/@oss-gr / podcast.gr-oss.io"},"change_message_id":"43f33b66d658cbf73a5b82534b83ea628349ba82","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":3,"id":"05d0f32a_78250710","updated":"2024-09-25 14:36:13.000000000","message":"The gist of all my feedback is: lets figure out a universal language around image types, including up-fronting the work to determine what \"raw\" will mean (and if it\u0027s different from some concept of \"unknown\") even if we aren\u0027t doing technical work to expose any other values. Documenting the vocabulary of types, and what they mean in OpenStack, will help operators and openstack developers communicate better about these things.\n\nSomething as simple as a document that says, in a global place:\n| Image Type | Description | Added in |\n| GPT | Image that contains a partition table for a whole disk | 2025.1 |\n| QCOW2 | Qemu Copy on Write 2 [link to spec] | Austin? |\n| raw | Image of raw bytes suitable for writing to disk. Usage is only if no other existing type matches. | Austin? | \n| filesystem | Image containing a full filesystem for writing to partition. | Future release |\n| unknown | Image of unknown type and style. Usage of these images is deprecated as they can\u0027t be security checked | Future release |\n\n\nI know having something like that, as an Ironic developer, would bring huge clarity and ensure we\u0027re thinking about and treating images in a similar way -- even if we know some of them may not be implemented on any specific timeline. (Honestly, I could even appreciate us adding image types we can\u0027t attest currently, but are more descriptive -- but I can understand not wanting to add a type to the API until we can safety check + detect it).","commit_id":"4b229658ce1b52a5dd7fb2b0c5590c40edad924d"},{"author":{"_account_id":10342,"name":"Jay Faulkner","display_name":"JayF","email":"jay@jvf.cc","username":"JayF","status":"youtube.com/@oss-gr / podcast.gr-oss.io"},"change_message_id":"b09f7b10e1175558e3b9ea23fad72839315fcab5","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":3,"id":"a790a342_614b0002","updated":"2024-09-25 15:02:00.000000000","message":"Yeah, that document being updated to reflect how we\u0027re thinking about those images now is exactly what I\u0027m wanting. My only concern with waiting is around \"raw\" no longer having both the meaning of \"unknown disk image\" and \"unstructured disk image\". I think there could be value in drawing that distinction even before we can detect it, but I think that\u0027s mainly because from an Ironic perspective, I really would like us to have a blanket term that means \"a disk image that doesn\u0027t need conversion before being written to a disk\". Maybe that\u0027s a different, new attribute than disk or container format, but it\u0027s really the heart of the distinction I\u0027d like to draw. We can make that distinction in Ironic with an indexed list of formats we should convert and should not, but it seems like a central piece of information that would be useful to multiple services.","commit_id":"4b229658ce1b52a5dd7fb2b0c5590c40edad924d"},{"author":{"_account_id":10342,"name":"Jay Faulkner","display_name":"JayF","email":"jay@jvf.cc","username":"JayF","status":"youtube.com/@oss-gr / podcast.gr-oss.io"},"change_message_id":"9becfa5a88f092e205133f7164438251a00523f2","unresolved":true,"context_lines":[],"source_content_type":"","patch_set":3,"id":"6b55646c_d5948374","in_reply_to":"03578d51_9955f3ba","updated":"2024-09-24 22:30:59.000000000","message":"**the pile of \u0027raw\u0027 images","commit_id":"4b229658ce1b52a5dd7fb2b0c5590c40edad924d"},{"author":{"_account_id":4393,"name":"Dan Smith","email":"dms@danplanet.com","username":"danms"},"change_message_id":"3581e142dbe4f4d3b0b7654ba002605ef1811c64","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":3,"id":"3fa9b9f6_3c10ba4e","in_reply_to":"05d0f32a_78250710","updated":"2024-09-25 14:50:44.000000000","message":"I think we shouldn\u0027t add types to the guideline that we can\u0027t detect (yet).\n\nWhat you\u0027re asking for is, I think, here:\n\nhttps://docs.openstack.org/glance/latest/user/formats.html\n\nalthough it gets not nearly enough attention and also misrepresents raw significantly (it explains it as what we idealize it as without mentioning the \"or anything else just choose this to make it always work\" aspect). Some of those have been added over the years of course and a \"first usable in $release\" declaration might be useful, although it will differ by project.\n\nSo if your ask is that when we add gpt (and again with the next one) we massively clarify that statement about raw (being only for data that is not assert-able as another format), then I\u0027m certainly on board with that.\n\nI *think* it makes sense for that to live in glance as the \"keeper of the images\". We pretty much reference that as the gospel everywhere else. But if you think it needs to go somewhere more neutral, then please suggest, propose, link, and I will contribute to that definition effort.","commit_id":"4b229658ce1b52a5dd7fb2b0c5590c40edad924d"},{"author":{"_account_id":4393,"name":"Dan Smith","email":"dms@danplanet.com","username":"danms"},"change_message_id":"86c6853b8bd700ce3de2b1af448732f653381405","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":3,"id":"5837c0f4_47dfcc3c","in_reply_to":"6a43107e_0c4f9aa2","updated":"2024-07-31 14:32:14.000000000","message":"Yeah, apparently. Not sure why that can\u0027t be in the setup patch (like it was for mine). If I do this, I\u0027ll just conflict with anyone else trying to do it. We don\u0027t have index files scattered throughout the tree in nova and thus don\u0027t have to do this, Might be worth thinking about a tweak to this structure in glance.\n\nAnyway, I\u0027ll get this updated in a bit.","commit_id":"4b229658ce1b52a5dd7fb2b0c5590c40edad924d"},{"author":{"_account_id":4393,"name":"Dan Smith","email":"dms@danplanet.com","username":"danms"},"change_message_id":"fe216b7c712f9b070200fb95fd03736edfea5e9f","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":3,"id":"a7443d1b_060ee0ab","in_reply_to":"6e96ea1f_e801fc91","updated":"2024-09-26 15:06:53.000000000","message":"\u003e As an operator this sounds great,\n\nThanks very much for confirming. Good to have operator voices saying this is important and welcome :)\n\n\u003e the main thing I\u0027m worried about is evaluating existing images and change the ones applicable from `raw` to `gpt` but this clearly mentioned the potential plans for that.\n\nYep, we\u0027ll need an either-or grace period for sure. I think the main compat thing will be nova allowing you to continue to boot raw images until you\u0027re satisfied that they\u0027re converted.\n\n\u003e That tooling would need to be carefully crafted because for clouds that have a lot of end-user owned images if there is no good way to do that discovery and changing disk_format I suspect a majority will just disable the safety if it interferes with users too much.\n\nYep, the inspector stuff is stream-based which means the conversion tool can stream just a few KiB of each image to detect if it should remain raw or be changed.","commit_id":"4b229658ce1b52a5dd7fb2b0c5590c40edad924d"},{"author":{"_account_id":4393,"name":"Dan Smith","email":"dms@danplanet.com","username":"danms"},"change_message_id":"190acbd0415a7bbd578ea348235371ba92b7f445","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":3,"id":"6e907647_b75fcb34","in_reply_to":"7393f988_67cb13a9","updated":"2024-09-25 17:10:13.000000000","message":"\u003e In Ironic world we usually document specifics like that kinda vocabulary in a spec (since it\u0027s API-facing) -- that\u0027s where the comments were coming from. If we want to ensure it gets documented as it gets done, that\u0027s fine.\n\nI think we\u0027ll all need that document in our projects, which references the updated glance definition (or some central one). I think nova probably has the \"luxury\" (read: curse) of sort of being the expected target for any type of thing you can store in glance. Thus, I think if nova stops being willing to boot images designated as \"raw\" we\u0027ll definitely have a lot of our own communication to do in order to pull that off.\n\n\u003e re: the gut feeling, I think the #1 sign you\u0027re correct is how difficult it is to describe the flag I want -- and the less ficticious example (although I\u0027d be \u0027interested\u0027 to see the fw implementation of qcow2 :P) is different CPU architectures. At a certain point, I think I want it to reflect intentions of the uploader, which is a different thing and not really anything we can validate (e.g. \"this random set of bits is actually a kernel\" might be a useful piece of information ... but Glance -- or OpenStack in general -- can\u0027t be counted on to understand it if it\u0027s, say, a kernel for a PowerPC-LE machine).\n\nYeah and I of course have the same gut feeling. It seems like a \"I know it when I see it\" sort of thing, but I think the counterexamples help to explain why it\u0027s just not that simple. The \"intent flag\" may be useful, I\u0027m not sure. Coming from the ashes of that mega-CVE I sort of don\u0027t want to add anything else that lets the user \"intend\" something that might conflict with a strict policy of \"no you can\u0027t boot that\". But, definitely worth discussing I think.","commit_id":"4b229658ce1b52a5dd7fb2b0c5590c40edad924d"},{"author":{"_account_id":10342,"name":"Jay Faulkner","display_name":"JayF","email":"jay@jvf.cc","username":"JayF","status":"youtube.com/@oss-gr / podcast.gr-oss.io"},"change_message_id":"3d539b667d80ca099a159c8c881a22769d0e3334","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":3,"id":"7393f988_67cb13a9","in_reply_to":"8c58cc74_d40022d1","updated":"2024-09-25 16:40:09.000000000","message":"\u003e But, we can surely propose and argue about that change to the text in a docs patch in parallel here if you want. Perhaps that would warrant a docs backport.\n\nIn Ironic world we usually document specifics like that kinda vocabulary in a spec (since it\u0027s API-facing) -- that\u0027s where the comments were coming from. If we want to ensure it gets documented as it gets done, that\u0027s fine.\n\n\nre: the gut feeling, I think the #1 sign you\u0027re correct is how difficult it is to describe the flag I want -- and the less ficticious example (although I\u0027d be \u0027interested\u0027 to see the fw implementation of qcow2 :P) is different CPU architectures. At a certain point, I think I want it to reflect intentions of the uploader, which is a different thing and not really anything we can validate (e.g. \"this random set of bits is actually a kernel\" might be a useful piece of information ... but Glance -- or OpenStack in general -- can\u0027t be counted on to understand it if it\u0027s, say, a kernel for a PowerPC-LE machine).\n\nThe idea of documented \"what we expect this to be useful for\" as an intent would be nice, but is massively outside of the scope of this spec (and maybe outside the scope of Glance in general)","commit_id":"4b229658ce1b52a5dd7fb2b0c5590c40edad924d"},{"author":{"_account_id":4393,"name":"Dan Smith","email":"dms@danplanet.com","username":"danms"},"change_message_id":"190acbd0415a7bbd578ea348235371ba92b7f445","unresolved":true,"context_lines":[],"source_content_type":"","patch_set":3,"id":"2f3e8dbf_3a3f523d","in_reply_to":"93c2722b_0370eb6e","updated":"2024-09-25 17:10:13.000000000","message":"Extracting GPT out of raw will definitely be something we have to choreograph. Inform by default for several cycles, then reject by default (with a knob that can be disabled) for many more cycles, followed by the removal of the escape hatch sometime far down the road someday.","commit_id":"4b229658ce1b52a5dd7fb2b0c5590c40edad924d"},{"author":{"_account_id":4393,"name":"Dan Smith","email":"dms@danplanet.com","username":"danms"},"change_message_id":"90c16947f1f990803c58061a999acb02cf443c38","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":3,"id":"8c58cc74_d40022d1","in_reply_to":"a790a342_614b0002","updated":"2024-09-25 15:30:07.000000000","message":"\u003e Yeah, that document being updated to reflect how we\u0027re thinking about those images now is exactly what I\u0027m wanting. My only concern with waiting is around \"raw\" no longer having both the meaning of \"unknown disk image\" and \"unstructured disk image\".\n\n...waiting around...until this spec is implemented? I wish we were having this conversation a few weeks ago so we could have updated the docs for 2024.2 to have some foreshadowing.\n\nBut, we can surely propose and argue about that change to the text in a docs patch in parallel here if you want. Perhaps that would warrant a docs backport.\n\n\u003e I think there could be value in drawing that distinction even before we can detect it, but I think that\u0027s mainly because from an Ironic perspective, I really would like us to have a blanket term that means \"a disk image that doesn\u0027t need conversion before being written to a disk\". Maybe that\u0027s a different, new attribute than disk or container format, but it\u0027s really the heart of the distinction I\u0027d like to draw. We can make that distinction in Ironic with an indexed list of formats we should convert and should not, but it seems like a central piece of information that would be useful to multiple services.\n\nAgain, I really think that your gut feeling that \"able to be written to a device\" is some intrinsic property that an image has or doesn\u0027t is wrong. A SPARC machine with openfirmware can read a BSD disk label and the UFS filesystem within a slice to find the kernel and start it booting (you can even manipulate the disk in some cases). A PC-based machine sees that same thing as random data, as does the SPARC machine about a totally bootable-by-PC MBR/GPT disk. Yet in both of those cases, those images can be dd\u0027d to a disk or volume and attached to a linux machine (physical or virtual, of either arch) and it can read/write it without any problem. The exact same arrangement but attached to a windows machine will see the MBR/GPT one as a known thing and the BSD one will look like garbage.\n\nI again refer back to my fictitious (but about to take the market by storm) custom-firmware totally-real machine that can read qcow2-formatted disks. qcow2 has a pre-allocated mode specifically for being written into a fixed-size container, which makes it entirely suitable for writing to a \"whole disk\", bootable by my machine, and fully read/write-able in place.\n\nMaybe another good example is Nova driving things like openvz or docker. What would normally be a bootable image that you can \"dd to a disk\" is no longer at all usable by these drivers. They can\u0027t boot/start from or even read a \"whole disk image\" any more than a PC machine can boot from a docker image. As I said before, I think that the context about when an image is suitable for a given scenario and writable to a given medium is entirely in the \"eye of the beholder\".\n\nIf you argue that we should take the \"bootable\" nature out of the above scenarios, I can\u0027t really think of any format that is *unsuitable* to be dd\u0027d to a disk for some reason. A tarball is a very high-level format consisting of file objects, not even block-based data, and yet it was designed *to* be written directly to tape. It\u0027s entirely possible to store a tarball directly on a disk (and in fact some people have used nova/cinder in this way for entirely legitimate reasons). The cpio format used by a lot of initrd scenarios is the same.\n\nI definitely understand the gut feeling, I just don\u0027t think that it makes sense as a global constant.","commit_id":"4b229658ce1b52a5dd7fb2b0c5590c40edad924d"},{"author":{"_account_id":19138,"name":"Pranali Deore","email":"pdeore@redhat.com","username":"PranaliD"},"change_message_id":"2e20bd4eea88b3ccde5332bcc99ba28af1154800","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":5,"id":"fa548bda_6fbe6fa0","updated":"2024-12-02 07:50:27.000000000","message":"Looks good to go! Thanks Dan and all reviewers !!","commit_id":"cc0b72089ef5a7d6dd467fd76ff7494e6335323e"},{"author":{"_account_id":8122,"name":"Cyril Roelandt","email":"cyril@redhat.com","username":"cyril.roelandt.enovance"},"change_message_id":"9adfdfbb915be94f47e801c57603ff6372582b47","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":5,"id":"7decf361_8d66c1f5","updated":"2024-11-18 18:50:08.000000000","message":"Looks perfect, thanks!","commit_id":"cc0b72089ef5a7d6dd467fd76ff7494e6335323e"},{"author":{"_account_id":4393,"name":"Dan Smith","email":"dms@danplanet.com","username":"danms"},"change_message_id":"04579fecbd6f2b2e2a98e7edd274ea9580c7b4f6","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":5,"id":"147b7a52_e370a9e2","updated":"2024-11-18 16:04:53.000000000","message":"This is the last working week of the year for me, so could we please get this merged this week. before I go? If there are feedback items, then I\u0027m happy to address. I\u0027ve basically got the full working code up across all the affected projects in bp/glance-as-defender.","commit_id":"cc0b72089ef5a7d6dd467fd76ff7494e6335323e"}],"specs/2025.1/approved/glance/glance-as-defender.rst":[{"author":{"_account_id":9303,"name":"Abhishek Kekane","email":"akekane@redhat.com","username":"abhishekkekane"},"change_message_id":"c179a7f2bbfc585055c38acb996430b9a605a451","unresolved":true,"context_lines":[{"line_number":29,"context_line":"declare its format to be any of the valid values we have for ``disk_format`` and"},{"line_number":30,"context_line":"``container_format``. This is certainly surprising to downstream services,"},{"line_number":31,"context_line":"external consumers, and humans which expect the format stated on the image to"},{"line_number":32,"context_line":"be in line with the actual content."},{"line_number":33,"context_line":""},{"line_number":34,"context_line":"A further problem is in the use of the ``raw`` value for ``disk_format``. In"},{"line_number":35,"context_line":"general, we use ``raw`` to mean \"a byte-for-byte image of a block device\","}],"source_content_type":"text/x-rst","patch_set":1,"id":"9ca05ac3_7ca7a80a","line":32,"updated":"2024-07-31 06:11:14.000000000","message":"+1","commit_id":"876588692f1b90b43324983f9368b0fde72453d7"},{"author":{"_account_id":9303,"name":"Abhishek Kekane","email":"akekane@redhat.com","username":"abhishekkekane"},"change_message_id":"fe7a931aae5b76eb013badfd5c4c5bc004d55e83","unresolved":false,"context_lines":[{"line_number":29,"context_line":"declare its format to be any of the valid values we have for ``disk_format`` and"},{"line_number":30,"context_line":"``container_format``. This is certainly surprising to downstream services,"},{"line_number":31,"context_line":"external consumers, and humans which expect the format stated on the image to"},{"line_number":32,"context_line":"be in line with the actual content."},{"line_number":33,"context_line":""},{"line_number":34,"context_line":"A further problem is in the use of the ``raw`` value for ``disk_format``. In"},{"line_number":35,"context_line":"general, we use ``raw`` to mean \"a byte-for-byte image of a block device\","}],"source_content_type":"text/x-rst","patch_set":1,"id":"9d0f197c_d896c096","line":32,"in_reply_to":"9ca05ac3_7ca7a80a","updated":"2024-10-24 06:40:54.000000000","message":"Done","commit_id":"876588692f1b90b43324983f9368b0fde72453d7"},{"author":{"_account_id":27615,"name":"Rajat Dhasmana","email":"rajatdhasmana@gmail.com","username":"whoami-rajat"},"change_message_id":"856729c46f356698612db55f30da73713b67b926","unresolved":true,"context_lines":[{"line_number":45,"context_line":""},{"line_number":46,"context_line":"This spec proposes two major changes to glance:"},{"line_number":47,"context_line":""},{"line_number":48,"context_line":"First, we will start enforcing that the format of the uploaded content matches"},{"line_number":49,"context_line":"the ``disk_format`` declared on the image. Since our ``format_inspector`` module"},{"line_number":50,"context_line":"is already in the data pipeline for both *upload* and *import* we simply need"},{"line_number":51,"context_line":"to remove the \"never fail\" behavior we currently have and abort the process"},{"line_number":52,"context_line":"if we determine that the format does not match what was claimed. Consider the"},{"line_number":53,"context_line":"following two examples:"},{"line_number":54,"context_line":""},{"line_number":55,"context_line":"1. An image is declared to be of format ``qcow2`` but the content uploaded is"},{"line_number":56,"context_line":"   something else (either another complex format such as ``vmdk`` or something"},{"line_number":57,"context_line":"   we do not recognize)."},{"line_number":58,"context_line":"2. An image is declared to be of format ``raw`` but the content uploaded is"},{"line_number":59,"context_line":"   detected as a ``qcow2``."},{"line_number":60,"context_line":""},{"line_number":61,"context_line":"In both of these cases, we should reject the upload/import and report a content"},{"line_number":62,"context_line":"type mismatch."},{"line_number":63,"context_line":""},{"line_number":64,"context_line":"The second major change is significant in terms of the model and behavior"},{"line_number":65,"context_line":"of existing users, but is small in absolute terms and impact to glance itself."}],"source_content_type":"text/x-rst","patch_set":1,"id":"b38b81f6_447894b5","line":62,"range":{"start_line":48,"start_character":0,"end_line":62,"end_character":14},"updated":"2024-07-29 18:08:34.000000000","message":"+1, this will help avoid duplicate checks on nova and cinder level so we can just rely on glance to provide correct images with accurate formats.","commit_id":"876588692f1b90b43324983f9368b0fde72453d7"},{"author":{"_account_id":27615,"name":"Rajat Dhasmana","email":"rajatdhasmana@gmail.com","username":"whoami-rajat"},"change_message_id":"856729c46f356698612db55f30da73713b67b926","unresolved":true,"context_lines":[{"line_number":61,"context_line":"In both of these cases, we should reject the upload/import and report a content"},{"line_number":62,"context_line":"type mismatch."},{"line_number":63,"context_line":""},{"line_number":64,"context_line":"The second major change is significant in terms of the model and behavior"},{"line_number":65,"context_line":"of existing users, but is small in absolute terms and impact to glance itself."},{"line_number":66,"context_line":"A new ``disk_format`` option of ``gpt`` will be added, which will henceforth"},{"line_number":67,"context_line":"serve to signify that \"this is an image of a raw block device with a partition"},{"line_number":68,"context_line":"table\" thus removing our need to overlap that definition with \"this is"},{"line_number":69,"context_line":"something we do not recognize\" for ``raw``. The definition of ``gpt`` is"},{"line_number":70,"context_line":"actually a superset of the legacy PC MBR (Master Boot Record) format, which"},{"line_number":71,"context_line":"means an inspector for this format should have no problem detecting and"},{"line_number":72,"context_line":"allowing images of very old (read: Windows XP/2003 vintage) disk images. Thus,"},{"line_number":73,"context_line":"many of the images we now legitimately use ``raw`` for will be (and need to be"},{"line_number":74,"context_line":"converted to be) a ``disk_format`` of ``gpt`` going forward."},{"line_number":75,"context_line":""},{"line_number":76,"context_line":"We should make all of this new behavior disable-able so as to provide a"},{"line_number":77,"context_line":"relief valve for bugs and identification issues."}],"source_content_type":"text/x-rst","patch_set":1,"id":"207dcba8_91bfda05","line":74,"range":{"start_line":64,"start_character":0,"end_line":74,"end_character":60},"updated":"2024-07-29 18:08:34.000000000","message":"this will require changes on cinder side as well, since, currently we detect the raw format and do a raw-\u003eraw conversion. It doesn\u0027t make much sense that\u0027s why Brian is working on replacing the conversion with a dd command instead.\nNeed to verify if the dd is sufficient after this new format introduction or additional processing is needed.","commit_id":"876588692f1b90b43324983f9368b0fde72453d7"},{"author":{"_account_id":4393,"name":"Dan Smith","email":"dms@danplanet.com","username":"danms"},"change_message_id":"d6d23283496d50d1e72bc44c557d2adb259c32d0","unresolved":true,"context_lines":[{"line_number":61,"context_line":"In both of these cases, we should reject the upload/import and report a content"},{"line_number":62,"context_line":"type mismatch."},{"line_number":63,"context_line":""},{"line_number":64,"context_line":"The second major change is significant in terms of the model and behavior"},{"line_number":65,"context_line":"of existing users, but is small in absolute terms and impact to glance itself."},{"line_number":66,"context_line":"A new ``disk_format`` option of ``gpt`` will be added, which will henceforth"},{"line_number":67,"context_line":"serve to signify that \"this is an image of a raw block device with a partition"},{"line_number":68,"context_line":"table\" thus removing our need to overlap that definition with \"this is"},{"line_number":69,"context_line":"something we do not recognize\" for ``raw``. The definition of ``gpt`` is"},{"line_number":70,"context_line":"actually a superset of the legacy PC MBR (Master Boot Record) format, which"},{"line_number":71,"context_line":"means an inspector for this format should have no problem detecting and"},{"line_number":72,"context_line":"allowing images of very old (read: Windows XP/2003 vintage) disk images. Thus,"},{"line_number":73,"context_line":"many of the images we now legitimately use ``raw`` for will be (and need to be"},{"line_number":74,"context_line":"converted to be) a ``disk_format`` of ``gpt`` going forward."},{"line_number":75,"context_line":""},{"line_number":76,"context_line":"We should make all of this new behavior disable-able so as to provide a"},{"line_number":77,"context_line":"relief valve for bugs and identification issues."}],"source_content_type":"text/x-rst","patch_set":1,"id":"cf728d5d_22b7d40c","line":74,"range":{"start_line":64,"start_character":0,"end_line":74,"end_character":60},"in_reply_to":"207dcba8_91bfda05","updated":"2024-07-29 18:16:15.000000000","message":"Yeah, right now your \"whole disk\" files are the same as they will be in the future, we just won\u0027t call them raw, we\u0027ll call them gpt. It\u0027s the same bits in the file, we will just have verified that they\u0027re in the right arrangement ahead of time and not call them \"raw\". The `dd` will be unchanged, you\u0027ll just (ideally) only do that if the type is gpt and (hopefully) refuse to use an image that claims to be \"raw\" (like nova will refuse to boot it). Since \u0027raw\u0027 could be anything from a text file to a JPEG, it makes sense for nova and cinder to refuse to blindly transform a file with no structure into one that has one.","commit_id":"876588692f1b90b43324983f9368b0fde72453d7"},{"author":{"_account_id":4393,"name":"Dan Smith","email":"dms@danplanet.com","username":"danms"},"change_message_id":"6d477736ba105662593ed2aa9e34c1a6a1d5c843","unresolved":true,"context_lines":[{"line_number":61,"context_line":"In both of these cases, we should reject the upload/import and report a content"},{"line_number":62,"context_line":"type mismatch."},{"line_number":63,"context_line":""},{"line_number":64,"context_line":"The second major change is significant in terms of the model and behavior"},{"line_number":65,"context_line":"of existing users, but is small in absolute terms and impact to glance itself."},{"line_number":66,"context_line":"A new ``disk_format`` option of ``gpt`` will be added, which will henceforth"},{"line_number":67,"context_line":"serve to signify that \"this is an image of a raw block device with a partition"},{"line_number":68,"context_line":"table\" thus removing our need to overlap that definition with \"this is"},{"line_number":69,"context_line":"something we do not recognize\" for ``raw``. The definition of ``gpt`` is"},{"line_number":70,"context_line":"actually a superset of the legacy PC MBR (Master Boot Record) format, which"},{"line_number":71,"context_line":"means an inspector for this format should have no problem detecting and"},{"line_number":72,"context_line":"allowing images of very old (read: Windows XP/2003 vintage) disk images. Thus,"},{"line_number":73,"context_line":"many of the images we now legitimately use ``raw`` for will be (and need to be"},{"line_number":74,"context_line":"converted to be) a ``disk_format`` of ``gpt`` going forward."},{"line_number":75,"context_line":""},{"line_number":76,"context_line":"We should make all of this new behavior disable-able so as to provide a"},{"line_number":77,"context_line":"relief valve for bugs and identification issues."}],"source_content_type":"text/x-rst","patch_set":1,"id":"57c18806_b856ac4b","line":74,"range":{"start_line":64,"start_character":0,"end_line":74,"end_character":60},"in_reply_to":"2204c0d2_bcc05fd1","updated":"2024-09-25 14:24:18.000000000","message":"FWIW, the qemu team has communicated to me that they regret using \"raw\" to mean \"unknown\" as well as \"a raw byte image of a disk\" in the same way we do. They have also strenuously recommended that we stop doing that.\n\nAs I said on IRC, I\u0027m also happy if people want to take the approach of redefining \"raw\" to mean \"a raw disk that we can assert things about\" and have another image type called \"unknown\" that we use for things about which we can\u0027t. However, I think that\u0027s likely to be *more* disruptive.\n\nI\u0027ll also say that until/unless we add a type for a filesystem and a checker, all those things (kernel images, raw copies of an ext2 filesystem, etc) fall into the bucket of \"no more specific definition\").","commit_id":"876588692f1b90b43324983f9368b0fde72453d7"},{"author":{"_account_id":10342,"name":"Jay Faulkner","display_name":"JayF","email":"jay@jvf.cc","username":"JayF","status":"youtube.com/@oss-gr / podcast.gr-oss.io"},"change_message_id":"43f33b66d658cbf73a5b82534b83ea628349ba82","unresolved":true,"context_lines":[{"line_number":61,"context_line":"In both of these cases, we should reject the upload/import and report a content"},{"line_number":62,"context_line":"type mismatch."},{"line_number":63,"context_line":""},{"line_number":64,"context_line":"The second major change is significant in terms of the model and behavior"},{"line_number":65,"context_line":"of existing users, but is small in absolute terms and impact to glance itself."},{"line_number":66,"context_line":"A new ``disk_format`` option of ``gpt`` will be added, which will henceforth"},{"line_number":67,"context_line":"serve to signify that \"this is an image of a raw block device with a partition"},{"line_number":68,"context_line":"table\" thus removing our need to overlap that definition with \"this is"},{"line_number":69,"context_line":"something we do not recognize\" for ``raw``. The definition of ``gpt`` is"},{"line_number":70,"context_line":"actually a superset of the legacy PC MBR (Master Boot Record) format, which"},{"line_number":71,"context_line":"means an inspector for this format should have no problem detecting and"},{"line_number":72,"context_line":"allowing images of very old (read: Windows XP/2003 vintage) disk images. Thus,"},{"line_number":73,"context_line":"many of the images we now legitimately use ``raw`` for will be (and need to be"},{"line_number":74,"context_line":"converted to be) a ``disk_format`` of ``gpt`` going forward."},{"line_number":75,"context_line":""},{"line_number":76,"context_line":"We should make all of this new behavior disable-able so as to provide a"},{"line_number":77,"context_line":"relief valve for bugs and identification issues."}],"source_content_type":"text/x-rst","patch_set":1,"id":"3b9069c0_07999c01","line":74,"range":{"start_line":64,"start_character":0,"end_line":74,"end_character":60},"in_reply_to":"57c18806_b856ac4b","updated":"2024-09-25 14:36:13.000000000","message":"I think us coming up with those words/glossary is /exactly/ the kind of thing I mean when talking about wanting to have the full story. That way we can start clearly communicating it to users and projects. Honestly we could probably even do something like that as an openstack-wide goal/spec ala the original logging guidelines.","commit_id":"876588692f1b90b43324983f9368b0fde72453d7"},{"author":{"_account_id":10342,"name":"Jay Faulkner","display_name":"JayF","email":"jay@jvf.cc","username":"JayF","status":"youtube.com/@oss-gr / podcast.gr-oss.io"},"change_message_id":"69f9054961f77c5576e88675381379b25d8e52c4","unresolved":true,"context_lines":[{"line_number":61,"context_line":"In both of these cases, we should reject the upload/import and report a content"},{"line_number":62,"context_line":"type mismatch."},{"line_number":63,"context_line":""},{"line_number":64,"context_line":"The second major change is significant in terms of the model and behavior"},{"line_number":65,"context_line":"of existing users, but is small in absolute terms and impact to glance itself."},{"line_number":66,"context_line":"A new ``disk_format`` option of ``gpt`` will be added, which will henceforth"},{"line_number":67,"context_line":"serve to signify that \"this is an image of a raw block device with a partition"},{"line_number":68,"context_line":"table\" thus removing our need to overlap that definition with \"this is"},{"line_number":69,"context_line":"something we do not recognize\" for ``raw``. The definition of ``gpt`` is"},{"line_number":70,"context_line":"actually a superset of the legacy PC MBR (Master Boot Record) format, which"},{"line_number":71,"context_line":"means an inspector for this format should have no problem detecting and"},{"line_number":72,"context_line":"allowing images of very old (read: Windows XP/2003 vintage) disk images. Thus,"},{"line_number":73,"context_line":"many of the images we now legitimately use ``raw`` for will be (and need to be"},{"line_number":74,"context_line":"converted to be) a ``disk_format`` of ``gpt`` going forward."},{"line_number":75,"context_line":""},{"line_number":76,"context_line":"We should make all of this new behavior disable-able so as to provide a"},{"line_number":77,"context_line":"relief valve for bugs and identification issues."}],"source_content_type":"text/x-rst","patch_set":1,"id":"2204c0d2_bcc05fd1","line":74,"range":{"start_line":64,"start_character":0,"end_line":74,"end_character":60},"in_reply_to":"cf728d5d_22b7d40c","updated":"2024-09-24 22:30:00.000000000","message":"I like this, but it makes me wonder if we need a type of \u0027filesystem image\u0027 or similar to cover the Ironic partition image case as mentioned above. I also think users not being able to upload `dd if\u003d/dev/disk of\u003dimage.raw` as a raw image is going to be expectation-breaking for users, where the term \"raw disk image\" is used often -- even outside our community -- to describe these kinds of images.\n\nThis is more than a technical problem, it\u0027s a marketing/documentation/mindshare problem and I have concerns about our ability to fix operator expectations.","commit_id":"876588692f1b90b43324983f9368b0fde72453d7"},{"author":{"_account_id":9303,"name":"Abhishek Kekane","email":"akekane@redhat.com","username":"abhishekkekane"},"change_message_id":"c179a7f2bbfc585055c38acb996430b9a605a451","unresolved":true,"context_lines":[{"line_number":80,"context_line":"  to avoid aborting the upload/import if the format does not match the content."},{"line_number":81,"context_line":"* ``require_gpt_not_raw```: Default to true, but allow setting to false to"},{"line_number":82,"context_line":"  avoid detecting full disk images as ``gpt`` instead of legacy ``raw``."},{"line_number":83,"context_line":"* ``allowed_disk_formats``: A list of ``disk_format`` values that will be"},{"line_number":84,"context_line":"  allowed, rejecting all others. This is a feature that would have provided a"},{"line_number":85,"context_line":"  strong mitigation for previous CVEs."},{"line_number":86,"context_line":""}],"source_content_type":"text/x-rst","patch_set":1,"id":"49482cf8_c2c41e0f","line":83,"range":{"start_line":83,"start_character":3,"end_line":83,"end_character":26},"updated":"2024-07-31 06:11:14.000000000","message":"How it will be different from disk_formats config option we already have under [image_format] section?","commit_id":"876588692f1b90b43324983f9368b0fde72453d7"},{"author":{"_account_id":9303,"name":"Abhishek Kekane","email":"akekane@redhat.com","username":"abhishekkekane"},"change_message_id":"fe7a931aae5b76eb013badfd5c4c5bc004d55e83","unresolved":false,"context_lines":[{"line_number":80,"context_line":"  to avoid aborting the upload/import if the format does not match the content."},{"line_number":81,"context_line":"* ``require_gpt_not_raw```: Default to true, but allow setting to false to"},{"line_number":82,"context_line":"  avoid detecting full disk images as ``gpt`` instead of legacy ``raw``."},{"line_number":83,"context_line":"* ``allowed_disk_formats``: A list of ``disk_format`` values that will be"},{"line_number":84,"context_line":"  allowed, rejecting all others. This is a feature that would have provided a"},{"line_number":85,"context_line":"  strong mitigation for previous CVEs."},{"line_number":86,"context_line":""}],"source_content_type":"text/x-rst","patch_set":1,"id":"3c975821_d90c4935","line":83,"range":{"start_line":83,"start_character":3,"end_line":83,"end_character":26},"in_reply_to":"3fe60326_bb8c4e01","updated":"2024-10-24 06:40:54.000000000","message":"Acknowledged","commit_id":"876588692f1b90b43324983f9368b0fde72453d7"},{"author":{"_account_id":4393,"name":"Dan Smith","email":"dms@danplanet.com","username":"danms"},"change_message_id":"7b513af1ace85e07feae55de55fd4d3b2096ac40","unresolved":true,"context_lines":[{"line_number":80,"context_line":"  to avoid aborting the upload/import if the format does not match the content."},{"line_number":81,"context_line":"* ``require_gpt_not_raw```: Default to true, but allow setting to false to"},{"line_number":82,"context_line":"  avoid detecting full disk images as ``gpt`` instead of legacy ``raw``."},{"line_number":83,"context_line":"* ``allowed_disk_formats``: A list of ``disk_format`` values that will be"},{"line_number":84,"context_line":"  allowed, rejecting all others. This is a feature that would have provided a"},{"line_number":85,"context_line":"  strong mitigation for previous CVEs."},{"line_number":86,"context_line":""}],"source_content_type":"text/x-rst","patch_set":1,"id":"3fe60326_bb8c4e01","line":83,"range":{"start_line":83,"start_character":3,"end_line":83,"end_character":26},"in_reply_to":"49482cf8_c2c41e0f","updated":"2024-07-31 13:39:29.000000000","message":"For some reason I thought that was more schema related than enforcement (I guess because you can stuff any *content* in regardless of what you claim it to be).\n\nBut if that currently limits the disk_formats you\u0027re allowed to specify in an image create, then that\u0027s cool (and I can remove this).","commit_id":"876588692f1b90b43324983f9368b0fde72453d7"},{"author":{"_account_id":10342,"name":"Jay Faulkner","display_name":"JayF","email":"jay@jvf.cc","username":"JayF","status":"youtube.com/@oss-gr / podcast.gr-oss.io"},"change_message_id":"69f9054961f77c5576e88675381379b25d8e52c4","unresolved":true,"context_lines":[{"line_number":37,"context_line":"disk). However, in reality ``raw`` has come to mean \"anything in a format for"},{"line_number":38,"context_line":"which we do not have another name\". This catch-all behavior means that if we"},{"line_number":39,"context_line":"want to support images of full-disk formats, we also need to support"},{"line_number":40,"context_line":"\"anything else we don\u0027t know about\" to some degree."},{"line_number":41,"context_line":""},{"line_number":42,"context_line":""},{"line_number":43,"context_line":"Proposed change"}],"source_content_type":"text/x-rst","patch_set":3,"id":"6eb9465a_01d9dccc","line":40,"updated":"2024-09-24 22:30:00.000000000","message":"We treat raw images as more than just full-disk formats; they can be filesystem images which are written into a preexisting partition. This use case is supported by Ironic today.","commit_id":"4b229658ce1b52a5dd7fb2b0c5590c40edad924d"},{"author":{"_account_id":4393,"name":"Dan Smith","email":"dms@danplanet.com","username":"danms"},"change_message_id":"6d477736ba105662593ed2aa9e34c1a6a1d5c843","unresolved":true,"context_lines":[{"line_number":37,"context_line":"disk). However, in reality ``raw`` has come to mean \"anything in a format for"},{"line_number":38,"context_line":"which we do not have another name\". This catch-all behavior means that if we"},{"line_number":39,"context_line":"want to support images of full-disk formats, we also need to support"},{"line_number":40,"context_line":"\"anything else we don\u0027t know about\" to some degree."},{"line_number":41,"context_line":""},{"line_number":42,"context_line":""},{"line_number":43,"context_line":"Proposed change"}],"source_content_type":"text/x-rst","patch_set":3,"id":"09c9de38_a8b08a26","line":40,"in_reply_to":"6eb9465a_01d9dccc","updated":"2024-09-25 14:24:18.000000000","message":"Sure, but I think that\u0027s covered in the \"anything in a format for which we do not have another name\" part of the clause.","commit_id":"4b229658ce1b52a5dd7fb2b0c5590c40edad924d"},{"author":{"_account_id":9303,"name":"Abhishek Kekane","email":"akekane@redhat.com","username":"abhishekkekane"},"change_message_id":"fe7a931aae5b76eb013badfd5c4c5bc004d55e83","unresolved":true,"context_lines":[{"line_number":79,"context_line":"We should make all of this new behavior disable-able so as to provide a"},{"line_number":80,"context_line":"relief valve for bugs and identification issues."},{"line_number":81,"context_line":""},{"line_number":82,"context_line":"* ``require_image_format_match``: Default to true, but allow setting to false"},{"line_number":83,"context_line":"  to avoid aborting the upload/import if the format does not match the content."},{"line_number":84,"context_line":"* ``require_gpt_not_raw```: Default to true, but allow setting to false to"},{"line_number":85,"context_line":"  avoid detecting full disk images as ``gpt`` instead of legacy ``raw``."},{"line_number":86,"context_line":""}],"source_content_type":"text/x-rst","patch_set":3,"id":"0d8e1d57_7e0fde08","line":83,"range":{"start_line":82,"start_character":2,"end_line":83,"end_character":79},"updated":"2024-10-24 06:40:54.000000000","message":"assuming setting it to false and image has format mismatch, it will be anyway rejected by consumer while they try to boot from it?","commit_id":"4b229658ce1b52a5dd7fb2b0c5590c40edad924d"},{"author":{"_account_id":4393,"name":"Dan Smith","email":"dms@danplanet.com","username":"danms"},"change_message_id":"6408aa975d8cf78bb823c4866fa34c283af074b3","unresolved":true,"context_lines":[{"line_number":79,"context_line":"We should make all of this new behavior disable-able so as to provide a"},{"line_number":80,"context_line":"relief valve for bugs and identification issues."},{"line_number":81,"context_line":""},{"line_number":82,"context_line":"* ``require_image_format_match``: Default to true, but allow setting to false"},{"line_number":83,"context_line":"  to avoid aborting the upload/import if the format does not match the content."},{"line_number":84,"context_line":"* ``require_gpt_not_raw```: Default to true, but allow setting to false to"},{"line_number":85,"context_line":"  avoid detecting full disk images as ``gpt`` instead of legacy ``raw``."},{"line_number":86,"context_line":""}],"source_content_type":"text/x-rst","patch_set":3,"id":"79a3670a_48ee28d4","line":83,"range":{"start_line":82,"start_character":2,"end_line":83,"end_character":79},"in_reply_to":"0d8e1d57_7e0fde08","updated":"2024-10-24 13:33:09.000000000","message":"Right, this doesn\u0027t change/affect nova and cinder\u0027s own coverage. I want glance to be a defender, but not the *only* defender. I think the other projects need to continue to guard against this sort of thing. Nova could be upgraded without/before glance, and have newer checks. Cinder and Ironic could be used standalone, without glance there to defend.","commit_id":"4b229658ce1b52a5dd7fb2b0c5590c40edad924d"},{"author":{"_account_id":10342,"name":"Jay Faulkner","display_name":"JayF","email":"jay@jvf.cc","username":"JayF","status":"youtube.com/@oss-gr / podcast.gr-oss.io"},"change_message_id":"69f9054961f77c5576e88675381379b25d8e52c4","unresolved":true,"context_lines":[{"line_number":90,"context_line":"Glance could continue to be ambivalent about the content uploaded and the"},{"line_number":91,"context_line":"mismatch between that and the metadata it stores. Services like nova and"},{"line_number":92,"context_line":"cinder will have to continue treating glance as untrustworthy and remain"},{"line_number":93,"context_line":"highly suspicious of its metadata."},{"line_number":94,"context_line":""},{"line_number":95,"context_line":"Data model impact"},{"line_number":96,"context_line":"-----------------"}],"source_content_type":"text/x-rst","patch_set":3,"id":"93bf73a3_a27c1648","line":93,"updated":"2024-09-24 22:30:00.000000000","message":"I\u0027ll note that regardless of the results of this spec, Ironic will have to treat all images as highly suspicious due to our support for deploying images directly from an http URL (for standalone use cases, but we don\u0027t block it in integrated clouds outside of RBAC settings)","commit_id":"4b229658ce1b52a5dd7fb2b0c5590c40edad924d"},{"author":{"_account_id":4393,"name":"Dan Smith","email":"dms@danplanet.com","username":"danms"},"change_message_id":"6d477736ba105662593ed2aa9e34c1a6a1d5c843","unresolved":true,"context_lines":[{"line_number":90,"context_line":"Glance could continue to be ambivalent about the content uploaded and the"},{"line_number":91,"context_line":"mismatch between that and the metadata it stores. Services like nova and"},{"line_number":92,"context_line":"cinder will have to continue treating glance as untrustworthy and remain"},{"line_number":93,"context_line":"highly suspicious of its metadata."},{"line_number":94,"context_line":""},{"line_number":95,"context_line":"Data model impact"},{"line_number":96,"context_line":"-----------------"}],"source_content_type":"text/x-rst","patch_set":3,"id":"07193763_80ce4629","line":93,"in_reply_to":"93bf73a3_a27c1648","updated":"2024-09-25 14:24:18.000000000","message":"Nova and cinder will as well, as it says there. I want glance to reject this stuff outright as early as possible but I do *not* want nova and cinder to start just trusting that glance will pre-check everything so they can drop their guard.","commit_id":"4b229658ce1b52a5dd7fb2b0c5590c40edad924d"},{"author":{"_account_id":10342,"name":"Jay Faulkner","display_name":"JayF","email":"jay@jvf.cc","username":"JayF","status":"youtube.com/@oss-gr / podcast.gr-oss.io"},"change_message_id":"69f9054961f77c5576e88675381379b25d8e52c4","unresolved":true,"context_lines":[{"line_number":104,"context_line":"could be a ``glance-manage`` command to automatically (or manually) do this,"},{"line_number":105,"context_line":"or allow it to be done through the API. Alternately, we could also annotate"},{"line_number":106,"context_line":"existing images and have the API report them as ``gpt`` if the client is"},{"line_number":107,"context_line":"determined to be new enough."},{"line_number":108,"context_line":""},{"line_number":109,"context_line":"REST API impact"},{"line_number":110,"context_line":"---------------"}],"source_content_type":"text/x-rst","patch_set":3,"id":"f688e5c7_0e20afef","line":107,"updated":"2024-09-24 22:30:00.000000000","message":"This assumes all raw images currently uploaded are GPT.\n\nWhat about IPA kernel images / ramdisk images? We know these are two specific image types that\u0027ll need to remain \"raw\" unless we define types for those. I\u0027m curious if there\u0027s value in just doing this for GPT, when we likely need other types to have even a theoretical path to a \"raw-less\" openstack deployment experience (with Ironic, at least).\n\nI have a strong preference for developing this whole story now, so that Ironic at least knows the design we\u0027re working towards across all those image types, even if we\u0027re going to implement it piecemeal.","commit_id":"4b229658ce1b52a5dd7fb2b0c5590c40edad924d"},{"author":{"_account_id":10342,"name":"Jay Faulkner","display_name":"JayF","email":"jay@jvf.cc","username":"JayF","status":"youtube.com/@oss-gr / podcast.gr-oss.io"},"change_message_id":"43f33b66d658cbf73a5b82534b83ea628349ba82","unresolved":true,"context_lines":[{"line_number":104,"context_line":"could be a ``glance-manage`` command to automatically (or manually) do this,"},{"line_number":105,"context_line":"or allow it to be done through the API. Alternately, we could also annotate"},{"line_number":106,"context_line":"existing images and have the API report them as ``gpt`` if the client is"},{"line_number":107,"context_line":"determined to be new enough."},{"line_number":108,"context_line":""},{"line_number":109,"context_line":"REST API impact"},{"line_number":110,"context_line":"---------------"}],"source_content_type":"text/x-rst","patch_set":3,"id":"227d2d2b_e55ed387","line":107,"in_reply_to":"0b8a37ac_37cbd0bd","updated":"2024-09-25 14:36:13.000000000","message":"As mentioned above, my concern is around -- as much as anything else -- ensuring we are using the same language to describe things. If we\u0027re going to try to differentiate \"image intended to be dd\u0027d onto a disk\" and \"unknown\", as you referenced above, I want us to have that terminology understood and documented -- even if some of it remains undocumented.\n\nIronic exposes a lot of configs / node attributes that contain the word \"raw\" to mean \"image ready to be dd\u0027d to a disk\" -- part of why I want to figure out the distinction now is that I think, for many cases, those configs would now refer to GPT images instead of raw. Having the nomenclature determined up front allows us to make better decisions about when/how to transition Ironic configs to this new more-specific-image-type world you\u0027re laying out.","commit_id":"4b229658ce1b52a5dd7fb2b0c5590c40edad924d"},{"author":{"_account_id":4393,"name":"Dan Smith","email":"dms@danplanet.com","username":"danms"},"change_message_id":"6d477736ba105662593ed2aa9e34c1a6a1d5c843","unresolved":true,"context_lines":[{"line_number":104,"context_line":"could be a ``glance-manage`` command to automatically (or manually) do this,"},{"line_number":105,"context_line":"or allow it to be done through the API. Alternately, we could also annotate"},{"line_number":106,"context_line":"existing images and have the API report them as ``gpt`` if the client is"},{"line_number":107,"context_line":"determined to be new enough."},{"line_number":108,"context_line":""},{"line_number":109,"context_line":"REST API impact"},{"line_number":110,"context_line":"---------------"}],"source_content_type":"text/x-rst","patch_set":3,"id":"0b8a37ac_37cbd0bd","line":107,"in_reply_to":"f688e5c7_0e20afef","updated":"2024-09-25 14:24:18.000000000","message":"\u003e This assumes all raw images currently uploaded are GPT.\n\nIt most certainly does not. Anything that is marked as a `raw` but that is not a `gpt` should remain as such, of course. Remember, nova supports boot from kernel and ramdisk as well, and those will remain as raw (or misnomer\u0027d as aki/ari if they are, but most common uses of them choose raw for these, AFAIK). Nova can and should refuse to boot a raw as a root disk, but it can (and will for some time) continue to allow raw images for kernels and ramdisks.\n\n\u003e What about IPA kernel images / ramdisk images? We know these are two specific image types that\u0027ll need to remain \"raw\" unless we define types for those. I\u0027m curious if there\u0027s value in just doing this for GPT, when we likely need other types to have even a theoretical path to a \"raw-less\" openstack deployment experience (with Ironic, at least).\n\nAlmost all of the formats we support can be detected by reading the first few blocks of the content, which is one reason format_inspector is designed the way that it is. The `glance_manage` (or whatever) tool we choose to do these conversions can and should \"sniff\" the image to determine if it should _remain_ as `raw` or be converted to `gpt`. The same can and should be applied for any other format metadata conversions we do in the future.\n\nThere\u0027s a substantial benefit to \"just doing this for GPT\" because nova and cinder currently will allow you to try to boot from something that isn\u0027t at all bootable, because a bootable disk and a tarball have the same type in glance (i.e. a raw).\n\nNova also allows a bare-filesystem type behavior when combined with a kernel/ramdisk, which we also need to solve as you point out, but it\u0027s a massive slice out of the pie to be able to close that hole for the 90% of users which are booting an actual disk image. I could be wrong, but I suspect cinder never wants to be handed a thing claiming to be raw but is actually a filesystem or tarball and that they\u0027ll never do the right thing if so.\n\n\u003e I have a strong preference for developing this whole story now, so that Ironic at least knows the design we\u0027re working towards across all those image types, even if we\u0027re going to implement it piecemeal.\n\nI have a strong preference for actually getting this done, and the raw-\u003egpt transition seems like a very valuable and straightforward one to cleave off the front. It won\u0027t affect any non-GPT images in glance, and in fact, glanceclient can effectively hide GPT images from services that haven\u0027t been taught what it means, if desired.\n\nI think the filesystem type you suggest is probably a good idea, but it\u0027s a significantly more complex beast to tackle than this one (do we have one thing that is \"filesystem\" or do we have multiples like \"ext2\", \"ntfs\", etc?). I\u0027m happy to contribute to that discussion as a follow-on spec, and if you feel it\u0027s only fair, I can split the gpt part of this (away from the assert-the-format-matches-the-content bit) so the new format proposals are separate from the base part.\n\nHowever, glance does not have much bandwidth to get stuff like this done in a cycle and I think doing something straightforward like this would make a lot of sense, knowing that the filesystem (and other) formats are coming after. The benefits that come from being able to assert a structure on the 99% of images (not counting the other formats) in glance and being able to block blind booting of the 1% seems like a significant benefit to me.","commit_id":"4b229658ce1b52a5dd7fb2b0c5590c40edad924d"},{"author":{"_account_id":10342,"name":"Jay Faulkner","display_name":"JayF","email":"jay@jvf.cc","username":"JayF","status":"youtube.com/@oss-gr / podcast.gr-oss.io"},"change_message_id":"69f9054961f77c5576e88675381379b25d8e52c4","unresolved":true,"context_lines":[{"line_number":134,"context_line":""},{"line_number":135,"context_line":"Users will defintitely be impacted as the muscle memory of (over) using raw as"},{"line_number":136,"context_line":"both a catch-all and as meaning \"an image of a whole disk\" will take some"},{"line_number":137,"context_line":"time to un-learn."},{"line_number":138,"context_line":""},{"line_number":139,"context_line":"Performance Impact"},{"line_number":140,"context_line":"------------------"}],"source_content_type":"text/x-rst","patch_set":3,"id":"82df04cb_6abab4a6","line":137,"updated":"2024-09-24 22:30:00.000000000","message":"This is a belief/assumption that exists outside of the openstack community, too. I think we should have deliberate plans for how to mitigate this impact.","commit_id":"4b229658ce1b52a5dd7fb2b0c5590c40edad924d"},{"author":{"_account_id":10342,"name":"Jay Faulkner","display_name":"JayF","email":"jay@jvf.cc","username":"JayF","status":"youtube.com/@oss-gr / podcast.gr-oss.io"},"change_message_id":"43f33b66d658cbf73a5b82534b83ea628349ba82","unresolved":true,"context_lines":[{"line_number":134,"context_line":""},{"line_number":135,"context_line":"Users will defintitely be impacted as the muscle memory of (over) using raw as"},{"line_number":136,"context_line":"both a catch-all and as meaning \"an image of a whole disk\" will take some"},{"line_number":137,"context_line":"time to un-learn."},{"line_number":138,"context_line":""},{"line_number":139,"context_line":"Performance Impact"},{"line_number":140,"context_line":"------------------"}],"source_content_type":"text/x-rst","patch_set":3,"id":"57f7c985_fd910f8b","line":137,"in_reply_to":"32634664_c8d17b72","updated":"2024-09-25 14:36:13.000000000","message":"\u003e what it really means and what we want it to mean is pretty dang fuzzy\n\nExactly the reason I want the language de-fuzzed and documented before code is written. Having exact terms to refer to these things will help us talk more specifically about it while transitioning.","commit_id":"4b229658ce1b52a5dd7fb2b0c5590c40edad924d"},{"author":{"_account_id":4393,"name":"Dan Smith","email":"dms@danplanet.com","username":"danms"},"change_message_id":"6d477736ba105662593ed2aa9e34c1a6a1d5c843","unresolved":true,"context_lines":[{"line_number":134,"context_line":""},{"line_number":135,"context_line":"Users will defintitely be impacted as the muscle memory of (over) using raw as"},{"line_number":136,"context_line":"both a catch-all and as meaning \"an image of a whole disk\" will take some"},{"line_number":137,"context_line":"time to un-learn."},{"line_number":138,"context_line":""},{"line_number":139,"context_line":"Performance Impact"},{"line_number":140,"context_line":"------------------"}],"source_content_type":"text/x-rst","patch_set":3,"id":"32634664_c8d17b72","line":137,"in_reply_to":"82df04cb_6abab4a6","updated":"2024-09-25 14:24:18.000000000","message":"If glance is able to, after only a few blocks of data, return an error message that says \"You said this was raw, but I detected it as gpt\" then I think it will be fairly user-friendly.\n\nThere are stackoverflow answers out there that tell people to just use \"raw\" even when you\u0027re uploading a qcow2 file because \"it doesn\u0027t matter\" and \"it\u0027s easier to just always say raw\". So that, to me, also indicates that the shared understanding of what it really means and what we want it to mean is pretty dang fuzzy.","commit_id":"4b229658ce1b52a5dd7fb2b0c5590c40edad924d"}]}