)]}'
{"specs/mitaka/approved/image-import/image-import-refactor.rst":[{"author":{"_account_id":4146,"name":"Clark Boylan","email":"cboylan@sapwetik.org","username":"cboylan"},"change_message_id":"5b7b15c7e21a2b4ac5e20216c4d55a37afa6ee91","unresolved":false,"context_lines":[{"line_number":242,"context_line":"      Glance must expose at least one import method that does not rely upon the"},{"line_number":243,"context_line":"      presence of an end-user-accessible object store."},{"line_number":244,"context_line":""},{"line_number":245,"context_line":"#. The \"three step dance\" import style was deemed acceptable"},{"line_number":246,"context_line":""},{"line_number":247,"context_line":"   #. one: create image record, two: upload data, three: import call"},{"line_number":248,"context_line":""}],"source_content_type":"text/x-rst","patch_set":1,"id":"bab6814e_bc21c3b6","line":245,"updated":"2016-05-19 18:04:32.000000000","message":"Specific feedback for this item after having done 6 images * ~9 cloud region uploads per day for a few months.\n\nThis is leaky. In the case of the \"normal\" PUT process we must first create the image record then PUT to it. If the PUT fails then you must also clean up the image record that you created in the first step.\n\nIn the case of the Rax\u0027s image import tasks we leak the swift objects. In that case you have to create the swift object first then run the import task. This means that if the task fails you have leaked the swift object. If the task succeeds there is no communication on when it is safe to delete the swift object and you leak there too.\n\nThere are many ways of solving this. The clients and SDKs that perform these actions could be smarter and handle these cases to avoid leaks. I don\u0027t know a ton about glance\u0027s internals but possibly it could handle the leaks for the users too. Another option is more \"atomic\" image creation so that if a single operation fails the entire process is considered a failure and garbage collected.\n\nAt the very least we need to very clearly communicate what the semantics are of this process to users and their tools so that they can make smart decisions on how to properly upload/import their images.","commit_id":"c29d9bb3034188f34a13667aad3c6a36c4b022c7"},{"author":{"_account_id":2,"name":"Monty Taylor","email":"mordred@inaugust.com","username":"mordred"},"change_message_id":"39a7ffaa0718d1e7e5041c9ea74629aa52cb567f","unresolved":false,"context_lines":[{"line_number":1084,"context_line":"    the scenes, after the user has made the data available to Glance but before"},{"line_number":1085,"context_line":"    the image status goes to \u0027active\u0027."},{"line_number":1086,"context_line":""},{"line_number":1087,"context_line":"#.  You must be bullshitting me, because the spec talks about more than one"},{"line_number":1088,"context_line":"    import method."},{"line_number":1089,"context_line":""},{"line_number":1090,"context_line":"    I bullshit you not.  While there are multiple import methods, each method"}],"source_content_type":"text/x-rst","patch_set":1,"id":"bab6814e_dcde476c","line":1087,"updated":"2016-05-19 17:43:10.000000000","message":"maybe not so much with \u0027bullshitting\u0027 in the spec. I know of some members of our community who would be rather offended","commit_id":"c29d9bb3034188f34a13667aad3c6a36c4b022c7"},{"author":{"_account_id":5314,"name":"Brian Rosmaita","email":"rosmaita.fossdev@gmail.com","username":"brian-rosmaita"},"change_message_id":"ef3e60b726c001531363bcb5d1c37a0f39cb1a40","unresolved":false,"context_lines":[{"line_number":1084,"context_line":"    the scenes, after the user has made the data available to Glance but before"},{"line_number":1085,"context_line":"    the image status goes to \u0027active\u0027."},{"line_number":1086,"context_line":""},{"line_number":1087,"context_line":"#.  You must be bullshitting me, because the spec talks about more than one"},{"line_number":1088,"context_line":"    import method."},{"line_number":1089,"context_line":""},{"line_number":1090,"context_line":"    I bullshit you not.  While there are multiple import methods, each method"}],"source_content_type":"text/x-rst","patch_set":1,"id":"bab6814e_7cebdbb0","line":1087,"in_reply_to":"bab6814e_dcde476c","updated":"2016-05-19 17:57:30.000000000","message":"Good point, I will make this disappear on the next patch.","commit_id":"c29d9bb3034188f34a13667aad3c6a36c4b022c7"},{"author":{"_account_id":2,"name":"Monty Taylor","email":"mordred@inaugust.com","username":"mordred"},"change_message_id":"39a7ffaa0718d1e7e5041c9ea74629aa52cb567f","unresolved":false,"context_lines":[{"line_number":1101,"context_line":"    whereby an end-user directly streams the object into Glance.  This will"},{"line_number":1102,"context_line":"    provide a non-optimal user experience, but it will be standard.  Since"},{"line_number":1103,"context_line":"    OpenStack provides free software for an object store solution, and since"},{"line_number":1104,"context_line":"    Swift is in fact widely deployed in OpenStack clouds, we propose"},{"line_number":1105,"context_line":"    implementing a \u0027swift-local\u0027 import method in which the image data would be"},{"line_number":1106,"context_line":"    uploaded to Swift out of band."},{"line_number":1107,"context_line":""}],"source_content_type":"text/x-rst","patch_set":1,"id":"bab6814e_bcd9c385","line":1104,"updated":"2016-05-19 17:43:10.000000000","message":"It\u0027s not widely deployed. It\u0027s less widely deployed than ceilometer, actually, per the most recent stats. (sits at about 50%)","commit_id":"c29d9bb3034188f34a13667aad3c6a36c4b022c7"},{"author":{"_account_id":5314,"name":"Brian Rosmaita","email":"rosmaita.fossdev@gmail.com","username":"brian-rosmaita"},"change_message_id":"ef3e60b726c001531363bcb5d1c37a0f39cb1a40","unresolved":false,"context_lines":[{"line_number":1101,"context_line":"    whereby an end-user directly streams the object into Glance.  This will"},{"line_number":1102,"context_line":"    provide a non-optimal user experience, but it will be standard.  Since"},{"line_number":1103,"context_line":"    OpenStack provides free software for an object store solution, and since"},{"line_number":1104,"context_line":"    Swift is in fact widely deployed in OpenStack clouds, we propose"},{"line_number":1105,"context_line":"    implementing a \u0027swift-local\u0027 import method in which the image data would be"},{"line_number":1106,"context_line":"    uploaded to Swift out of band."},{"line_number":1107,"context_line":""}],"source_content_type":"text/x-rst","patch_set":1,"id":"bab6814e_5c2c1702","line":1104,"in_reply_to":"bab6814e_bcd9c385","updated":"2016-05-19 17:57:30.000000000","message":"Thanks, will update to be factually correct.","commit_id":"c29d9bb3034188f34a13667aad3c6a36c4b022c7"},{"author":{"_account_id":2,"name":"Monty Taylor","email":"mordred@inaugust.com","username":"mordred"},"change_message_id":"39a7ffaa0718d1e7e5041c9ea74629aa52cb567f","unresolved":false,"context_lines":[{"line_number":1108,"context_line":"    Additionally, in anticipation of the deprecation of the Images v1 API, some"},{"line_number":1109,"context_line":"    operators have pointed out that the \"copy-from\" capability in the older API"},{"line_number":1110,"context_line":"    is missing from the Images v2 API.  While it\u0027s not a driver for this work,"},{"line_number":1111,"context_line":"    a well-defined copy-from import method can be accommodated by this design."},{"line_number":1112,"context_line":""},{"line_number":1113,"context_line":"#.  What if I don\u0027t like there being multiple import methods?"},{"line_number":1114,"context_line":""}],"source_content_type":"text/x-rst","patch_set":1,"id":"bab6814e_1ce9ef14","line":1111,"updated":"2016-05-19 17:43:10.000000000","message":"I think this whole answer is good, but is clearly written from the POV of someone who believes that the object-storage upload options is inherently superior for end users. As an end user, I can attest that it\u0027s actually a significantly worse experience and that the direct upload is SO much better. However, we disagree on that, and that\u0027s totally cool. I think it would be friendly if this answer did not present one thing as \"right\" and one thing as \"well, if you have wrongthink, we\u0027ll also give you this option\" - but instead that there are two completely valid view points on this and that allowing consistent api across clouds iwth defined meanings that involved capabilities discoverable by our users is a way for us to empower our end users to have opinions.","commit_id":"c29d9bb3034188f34a13667aad3c6a36c4b022c7"},{"author":{"_account_id":5314,"name":"Brian Rosmaita","email":"rosmaita.fossdev@gmail.com","username":"brian-rosmaita"},"change_message_id":"ef3e60b726c001531363bcb5d1c37a0f39cb1a40","unresolved":false,"context_lines":[{"line_number":1108,"context_line":"    Additionally, in anticipation of the deprecation of the Images v1 API, some"},{"line_number":1109,"context_line":"    operators have pointed out that the \"copy-from\" capability in the older API"},{"line_number":1110,"context_line":"    is missing from the Images v2 API.  While it\u0027s not a driver for this work,"},{"line_number":1111,"context_line":"    a well-defined copy-from import method can be accommodated by this design."},{"line_number":1112,"context_line":""},{"line_number":1113,"context_line":"#.  What if I don\u0027t like there being multiple import methods?"},{"line_number":1114,"context_line":""}],"source_content_type":"text/x-rst","patch_set":1,"id":"bab6814e_7c875bd7","line":1111,"in_reply_to":"bab6814e_1ce9ef14","updated":"2016-05-19 17:57:30.000000000","message":"Thanks, your suggestion really captures the whole point of why we\u0027er doing it this way!","commit_id":"c29d9bb3034188f34a13667aad3c6a36c4b022c7"},{"author":{"_account_id":2472,"name":"Doug Hellmann","email":"dhellmann@redhat.com","username":"doug-hellmann"},"change_message_id":"7a593378be91912561f64a6538ca3717993c9ded","unresolved":false,"context_lines":[{"line_number":1117,"context_line":"    That, however, would mitigate against the goal of having stable API calls"},{"line_number":1118,"context_line":"    supported in all OpenStack-branded clouds, since not every cloud operator"},{"line_number":1119,"context_line":"    will want to expose all import methods.  That would make it difficult for"},{"line_number":1120,"context_line":"    image import to be in DefCore."},{"line_number":1121,"context_line":""},{"line_number":1122,"context_line":"    Let\u0027s be really careful in describing what we\u0027re talking about here.  The"},{"line_number":1123,"context_line":"    approved design for image import allows operator choice, but in a very"}],"source_content_type":"text/x-rst","patch_set":1,"id":"bab6814e_7913ed0d","line":1120,"updated":"2016-05-19 17:30:46.000000000","message":"Is that really why we said a single API call to start the process? Because with a discovery API we could still ask \"how do I upload images to you?\" and get an answer that indicated what next step should be taken, even if that next step was not always the same. IIUC, there is some variability in which calls need to be made based on whether an image is being uploaded directly or whether it exists in swift, so I\u0027m not sure the results would really be that different, would they?","commit_id":"c29d9bb3034188f34a13667aad3c6a36c4b022c7"},{"author":{"_account_id":6754,"name":"Mark T. Voelker","email":"markvoelker@gmail.com","username":"mvoelker"},"change_message_id":"7f303855b018a0efc7e75c549d451f02bfc95228","unresolved":false,"context_lines":[{"line_number":1117,"context_line":"    That, however, would mitigate against the goal of having stable API calls"},{"line_number":1118,"context_line":"    supported in all OpenStack-branded clouds, since not every cloud operator"},{"line_number":1119,"context_line":"    will want to expose all import methods.  That would make it difficult for"},{"line_number":1120,"context_line":"    image import to be in DefCore."},{"line_number":1121,"context_line":""},{"line_number":1122,"context_line":"    Let\u0027s be really careful in describing what we\u0027re talking about here.  The"},{"line_number":1123,"context_line":"    approved design for image import allows operator choice, but in a very"}],"source_content_type":"text/x-rst","patch_set":1,"id":"bab6814e_02502831","line":1120,"in_reply_to":"bab6814e_7913ed0d","updated":"2016-05-26 16:40:41.000000000","message":"So, recapping what I think I heard at the meeting today:\n\n\nThe state of the world right now is that:\n\n\n1.) There\u0027s no way to discover if it\u0027s even possible to get images into the cloud at all or if I can only choose from ones that have already been provided for me (other than trail and error, which stinks).\n\n\n2.)  Assuming it is possible to bring my own images, there\u0027s no way to discover what methods one might use to get images into a given cloud (other than trial and error, which stinks).\n\n\n3.)  Assuming it is possible to bring my own images, there\u0027s no interoperable method of actually pushing the bits into clouds.  Some clouds use only tasks, some use only direct upload, some want copy-from, etc.\n\n\nFrom a DefCore perspective, the ask is to think of those three things as separate Capabilities (which also implies there should be a test for each one).\n\n\n#1 is a thing that could probably make it into DefCore (since *most* clouds today probably do allow BYOI via some means or another) if we consider it to be it\u0027s own Capability.  We\u0027d need to write a test that proves that a cloud has some way for end users to provide their own images (maybe that\u0027s a test that calls a discovery API and verifies a non-empty list of methods returned, for example, though I won\u0027t dwell on the mechanics of the proposed test here).  It makes application writers\u0027 lives somewhat easier in that they would conceivably have a way to discover whether or not they can bring their own images at all.\n\n\n#2 is a thing we can probably solve with a new discovery API that allows clients to see what methods are available to push bits when talking to a given cloud.  It seems quite feasible that most clouds would be willing to implement that, so this Capability feels like it has a high chance of becoming market-accepted and therefore DefCore-able.  It makes application writers\u0027 lives somewhat easier in that they would conceivably have a way to discover what methods are available to them.\n\n\n#3 *may* not be a thing we can ever get to a truly interoperable point with because the use cases and market preferences are currently so different (as hogepodge mentioned, public clouds may want only the task method so they can do security scans on tenant-created images, private clouds may often not want that at all).  DefCore is mostly a trailing indicator of market acceptance [1], so we can only mandate that one or method becomes required if it actually gets traction in the real world.  We can write individual tests for the various methods of getting image data into the cloud, and some or none of them may ever make into a DefCore Guideline depending on how many clouds actually provide them and how many clients/SDK\u0027s/etc support them.  If we assume that not all methods are going to be widely deployed and widely client/SDK/tool-supported, we may not actually improve developers\u0027 lives much here.  If one or methods *do* stack up to Criteria [2], then those can go in and that\u0027ll be a win, but that\u0027s not certain to happen given how opinionated deployers are today.\n\n\nSo all in all, this plan makes developers lives better (though potentially not perfect), which is still progress.  Personally I think it sounds feasible for DefCore to consider these as three Capabilities, and I\u0027m happy to hash that out with the rest of the DefCore Committee.\n\n\n[1] http://markvoelker.github.io/blog/defcore-misconceptions-1/\n\n[2] http://git.openstack.org/cgit/openstack/defcore/tree/doc/source/process/CoreCriteria.rst","commit_id":"c29d9bb3034188f34a13667aad3c6a36c4b022c7"},{"author":{"_account_id":2472,"name":"Doug Hellmann","email":"dhellmann@redhat.com","username":"doug-hellmann"},"change_message_id":"7a593378be91912561f64a6538ca3717993c9ded","unresolved":false,"context_lines":[{"line_number":1133,"context_line":"    This is ultimately up to the DefCore project.  Our idea is that any"},{"line_number":1134,"context_line":"    OpenStack cloud would support at least one of the import methods.  Even so,"},{"line_number":1135,"context_line":"    note that at the same time, *all* OpenStack clouds would support Images v2"},{"line_number":1136,"context_line":"    API image import."},{"line_number":1137,"context_line":""},{"line_number":1138,"context_line":"#.  If there are multiple methods, how can I determine what\u0027s available in a"},{"line_number":1139,"context_line":"    particular cloud?"}],"source_content_type":"text/x-rst","patch_set":1,"id":"bab6814e_59d4291f","line":1136,"updated":"2016-05-19 17:30:46.000000000","message":"Is this instance of \"image import\" referring to the \"make an image from a snapshot via nova\" feature?","commit_id":"c29d9bb3034188f34a13667aad3c6a36c4b022c7"},{"author":{"_account_id":2,"name":"Monty Taylor","email":"mordred@inaugust.com","username":"mordred"},"change_message_id":"39a7ffaa0718d1e7e5041c9ea74629aa52cb567f","unresolved":false,"context_lines":[{"line_number":1133,"context_line":"    This is ultimately up to the DefCore project.  Our idea is that any"},{"line_number":1134,"context_line":"    OpenStack cloud would support at least one of the import methods.  Even so,"},{"line_number":1135,"context_line":"    note that at the same time, *all* OpenStack clouds would support Images v2"},{"line_number":1136,"context_line":"    API image import."},{"line_number":1137,"context_line":""},{"line_number":1138,"context_line":"#.  If there are multiple methods, how can I determine what\u0027s available in a"},{"line_number":1139,"context_line":"    particular cloud?"}],"source_content_type":"text/x-rst","patch_set":1,"id":"bab6814e_fce34b31","line":1136,"updated":"2016-05-19 17:43:10.000000000","message":"Or is it instead \"since all OpenStack clouds would support at least one of the import methods, all OpenStack clouds would have a working Image v2 API import\"","commit_id":"c29d9bb3034188f34a13667aad3c6a36c4b022c7"},{"author":{"_account_id":5314,"name":"Brian Rosmaita","email":"rosmaita.fossdev@gmail.com","username":"brian-rosmaita"},"change_message_id":"92a49ab70416dea69a3aa340653a3d8326b0a0ce","unresolved":false,"context_lines":[{"line_number":1133,"context_line":"    This is ultimately up to the DefCore project.  Our idea is that any"},{"line_number":1134,"context_line":"    OpenStack cloud would support at least one of the import methods.  Even so,"},{"line_number":1135,"context_line":"    note that at the same time, *all* OpenStack clouds would support Images v2"},{"line_number":1136,"context_line":"    API image import."},{"line_number":1137,"context_line":""},{"line_number":1138,"context_line":"#.  If there are multiple methods, how can I determine what\u0027s available in a"},{"line_number":1139,"context_line":"    particular cloud?"}],"source_content_type":"text/x-rst","patch_set":1,"id":"bab6814e_793f6dd6","line":1136,"in_reply_to":"bab6814e_59d4291f","updated":"2016-05-19 17:38:28.000000000","message":"No, this is meant as a reference to \"bring your own image\" to satisfy the TC resolution (see [NEW1] in the references section, below).","commit_id":"c29d9bb3034188f34a13667aad3c6a36c4b022c7"},{"author":{"_account_id":2472,"name":"Doug Hellmann","email":"dhellmann@redhat.com","username":"doug-hellmann"},"change_message_id":"1e02c1dc87f183a39783b6e284d6432f915b81ac","unresolved":false,"context_lines":[{"line_number":1133,"context_line":"    This is ultimately up to the DefCore project.  Our idea is that any"},{"line_number":1134,"context_line":"    OpenStack cloud would support at least one of the import methods.  Even so,"},{"line_number":1135,"context_line":"    note that at the same time, *all* OpenStack clouds would support Images v2"},{"line_number":1136,"context_line":"    API image import."},{"line_number":1137,"context_line":""},{"line_number":1138,"context_line":"#.  If there are multiple methods, how can I determine what\u0027s available in a"},{"line_number":1139,"context_line":"    particular cloud?"}],"source_content_type":"text/x-rst","patch_set":1,"id":"bab6814e_bcca636c","line":1136,"in_reply_to":"bab6814e_793f6dd6","updated":"2016-05-19 17:48:47.000000000","message":"Ah, so you\u0027re saying that all clouds would eventually be able to pass a DefCore rule saying that image import needs to be provided, because they would be able to choose at least one of the mechanisms backing this new API?","commit_id":"c29d9bb3034188f34a13667aad3c6a36c4b022c7"},{"author":{"_account_id":5314,"name":"Brian Rosmaita","email":"rosmaita.fossdev@gmail.com","username":"brian-rosmaita"},"change_message_id":"ef3e60b726c001531363bcb5d1c37a0f39cb1a40","unresolved":false,"context_lines":[{"line_number":1133,"context_line":"    This is ultimately up to the DefCore project.  Our idea is that any"},{"line_number":1134,"context_line":"    OpenStack cloud would support at least one of the import methods.  Even so,"},{"line_number":1135,"context_line":"    note that at the same time, *all* OpenStack clouds would support Images v2"},{"line_number":1136,"context_line":"    API image import."},{"line_number":1137,"context_line":""},{"line_number":1138,"context_line":"#.  If there are multiple methods, how can I determine what\u0027s available in a"},{"line_number":1139,"context_line":"    particular cloud?"}],"source_content_type":"text/x-rst","patch_set":1,"id":"bab6814e_dc55a750","line":1136,"in_reply_to":"bab6814e_bcca636c","updated":"2016-05-19 17:57:30.000000000","message":"You\u0027ve got it!  I\u0027ll try to make the point more clearly on the next patch.","commit_id":"c29d9bb3034188f34a13667aad3c6a36c4b022c7"},{"author":{"_account_id":6159,"name":"Flavio Percoco Premoli","display_name":"flaper87","email":"flavio.percoco@flyrlabs.com","username":"flaper87"},"change_message_id":"585d4567beabef328bf2210969c08cf5ccddda65","unresolved":false,"context_lines":[{"line_number":1161,"context_line":"    implementing for image import.  So it\u0027s not the kind of thing that can be"},{"line_number":1162,"context_line":"    done lightly."},{"line_number":1163,"context_line":""},{"line_number":1164,"context_line":"#.  What about the old image upload call?"},{"line_number":1165,"context_line":""},{"line_number":1166,"context_line":"    The PUT to v2/images/{image_id}/file makes sense as a call used by"},{"line_number":1167,"context_line":"    operators and trusted services, so we are not proposing that it be removed."}],"source_content_type":"text/x-rst","patch_set":2,"id":"9abb7d3a_9d209ef9","line":1164,"updated":"2016-05-27 11:56:24.000000000","message":"This requires https://review.openstack.org/#/c/313947/ to land, doesn\u0027t it?","commit_id":"29899a5a4181f605dc5c04e6efd169fa19b99061"},{"author":{"_account_id":5314,"name":"Brian Rosmaita","email":"rosmaita.fossdev@gmail.com","username":"brian-rosmaita"},"change_message_id":"a1888113080795debdbf767d10a92731eab04a9c","unresolved":false,"context_lines":[{"line_number":1161,"context_line":"    implementing for image import.  So it\u0027s not the kind of thing that can be"},{"line_number":1162,"context_line":"    done lightly."},{"line_number":1163,"context_line":""},{"line_number":1164,"context_line":"#.  What about the old image upload call?"},{"line_number":1165,"context_line":""},{"line_number":1166,"context_line":"    The PUT to v2/images/{image_id}/file makes sense as a call used by"},{"line_number":1167,"context_line":"    operators and trusted services, so we are not proposing that it be removed."}],"source_content_type":"text/x-rst","patch_set":2,"id":"9abb7d3a_d53771e1","line":1164,"in_reply_to":"9abb7d3a_9d209ef9","updated":"2016-05-29 13:02:21.000000000","message":"I don\u0027t think so.  The current text just says \"Mark as deprecated for non-admin use\", nothing about removal or changing policies, or anything drastic ... and we do want to encourage operators to encourage their users to adopt the new import calls.","commit_id":"29899a5a4181f605dc5c04e6efd169fa19b99061"},{"author":{"_account_id":6159,"name":"Flavio Percoco Premoli","display_name":"flaper87","email":"flavio.percoco@flyrlabs.com","username":"flaper87"},"change_message_id":"585d4567beabef328bf2210969c08cf5ccddda65","unresolved":false,"context_lines":[{"line_number":1197,"context_line":"    There are also some recent etherpads:"},{"line_number":1198,"context_line":""},{"line_number":1199,"context_line":"    - `Newton contributors\u0027 meeting: image import \u003chttps://etherpad.openstack.org/p/newton-glance-import-refactor\u003e`_"},{"line_number":1200,"context_line":"    - `Image Import MVP \u003chttps://etherpad.openstack.org/p/glance-image-import-implementation-tactics\u003e`_"},{"line_number":1201,"context_line":""},{"line_number":1202,"context_line":"    Additionally, you may find the discussion on the following patches"},{"line_number":1203,"context_line":"    entertaining:"}],"source_content_type":"text/x-rst","patch_set":2,"id":"9abb7d3a_7d469ac0","line":1200,"updated":"2016-05-27 11:56:24.000000000","message":"We could also add the link to our last hangout recording.","commit_id":"29899a5a4181f605dc5c04e6efd169fa19b99061"}]}
