)]}'
{"/PATCHSET_LEVEL":[{"author":{"_account_id":2,"name":"Monty Taylor","email":"mordred@inaugust.com","username":"mordred"},"change_message_id":"5712c5043817a2096556f0f4aab3e9b30d284bc4","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":1,"id":"b30f066f_1bbe8239","updated":"2022-07-14 18:27:21.000000000","message":"Love.","commit_id":"032f625fd751f08ee572babe681088aa418c0447"},{"author":{"_account_id":1,"name":"James E. Blair","email":"jim@acmegating.com","username":"corvus"},"change_message_id":"761c9d23340ad66cab3e024af7c4ee8f27f5fe3e","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":1,"id":"526cd215_c58986db","updated":"2022-07-15 00:27:37.000000000","message":"PS2 moves the configuration from the tenant config to in-repo config.","commit_id":"032f625fd751f08ee572babe681088aa418c0447"},{"author":{"_account_id":7118,"name":"Ian Wienand","email":"iwienand@redhat.com","username":"iwienand"},"change_message_id":"3d7a07209d50d4b52dd83fa11fc6ae5db9dce733","unresolved":true,"context_lines":[],"source_content_type":"","patch_set":2,"id":"8a925c6e_40b755a1","updated":"2022-07-15 01:11:21.000000000","message":"Broadly this seems like the logical thing to do; -1 just for visibility of comments on dib testing here:\n\nOne thing I\u0027d like to see covered as a work-item is decoupling dib testing from nodepool too.  Currently dib image build+boot testing is based on having nodepool-builder run the build and nodepool-launcher upload it to a devstack-based test environment.\n\nThe broadness of this testing has been of mutual benefit to both sides of the dib/nodepool equation; although the trade-off is that it is slow, and there is some tension between the projects as to \"why is my \u003cdib|nodepool\u003e change blocked on \u003cnodepool|dib\u003e\" (usually this is a real problem, but not always).\n\nWith the validation steps covered in here, we can set up something that we test the zuul built images before making them live; but I don\u0027t think this is appropriate for speculative testing of the more generic image changes that come into dib.  \n\nWhat I think dib wants to keep is the end-to-end build+boot cloud test.  Because this has also traditionally been a openstacksdk and glean (i.e. metadata) proxy test, it seems worth nothing that we think keeping the \"push to a devstack\" part of the testing is useful (rather than say, just booting in a qemu instance by hand, or using virt-manager or something)?\n\nThe easiest path is thus, I guess, to keep the extant environment, but to excise nodepool from the dib boot tests; call dib directly in the tests to produce the images (remove nodepool-builder) and upload/boot the images directly (remove nodepool-launcher) rather than having nodepool do it.  I\u0027d call this a moderate amount of work, as we\u0027re replicating a lot of things nodepool handles for us like networking setup etc.  That could happen more or less asynchronously, but would need to be done before retirement of nodepool.\n\nThe other option would be to integrate zuul\u0027s new image handling into this testing, in the same way nodepool does now; basically converting dib testing to brings up a zuul to talk to devstack and use that to build, push and boot the test images.  This does have the benefit of being a very good test, but the downside of sounding very complicated.\n\nso; tldr\n\n- dibs boot testing of common platforms builds and pushes into a devstack cloud.\n- propose we agree that this is still quite useful given what it\u0027s trying to verify (that changes boot and communicate in a openstack cloud).\n- note that nodepool needs to be excised from this testing\n- decide if we replace that in dib with a bespoke build/upload step, or try to setup a zuul environment to mostly replicate the complete end-to-end testing that happens with nodepool.\n","commit_id":"bf0137b6b528b0683f92e3117c76cb2395ff3146"},{"author":{"_account_id":9311,"name":"Tristan Cacqueray","email":"tdecacqu@redhat.com","username":"tristanC"},"change_message_id":"e3e145f8ca9f6792ac4f2d0ee08d3c4f7314d523","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":2,"id":"4ef3b4a6_b517f3da","updated":"2022-07-15 00:40:17.000000000","message":"Thanks, the proposal looks really good to me.","commit_id":"bf0137b6b528b0683f92e3117c76cb2395ff3146"},{"author":{"_account_id":1,"name":"James E. Blair","email":"jim@acmegating.com","username":"corvus"},"change_message_id":"841e316216aabffeebefd63e0c26a3b6aa6f578b","unresolved":true,"context_lines":[],"source_content_type":"","patch_set":2,"id":"070e45a2_b3063589","in_reply_to":"8a925c6e_40b755a1","updated":"2022-07-15 14:41:50.000000000","message":"Agree -- my inclination is toward the \"bespoke build/upload\" option.  Mostly because a lot of the \"build an image with dib\" parts of this will be self-testing in jobs, so a full Zuul system with the jobs to do that would be redundant (and as you say complicated).  Whereas extracting just the steps to build and upload an image to devstack into a python script localizes what we want to test better.  I\u0027ll update the spec to mention that.","commit_id":"bf0137b6b528b0683f92e3117c76cb2395ff3146"},{"author":{"_account_id":1,"name":"James E. Blair","email":"jim@acmegating.com","username":"corvus"},"change_message_id":"8deab3d58bfc954822fa18014f75b3d1ff0972bd","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":2,"id":"4787b6eb_200ccbf1","in_reply_to":"8a925c6e_40b755a1","updated":"2022-07-16 17:05:25.000000000","message":"Done","commit_id":"bf0137b6b528b0683f92e3117c76cb2395ff3146"},{"author":{"_account_id":4146,"name":"Clark Boylan","email":"cboylan@sapwetik.org","username":"cboylan"},"change_message_id":"7be0b064a7c79187fa42b085a39538393c044538","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":8,"id":"6fbc8677_15325673","updated":"2022-09-01 19:55:00.000000000","message":"Left some thoughts inline. I also notice that this seems to skip over leaked resource pruning. We\u0027ve seen that this is an important part of the existing nodepool services and it may be worth touching on how the new zuul-launcher (or something else?) is going to be able to deal with leaked resources.","commit_id":"781a8470d3790e6257fa5e813e9d8750c41b72bc"},{"author":{"_account_id":9311,"name":"Tristan Cacqueray","email":"tdecacqu@redhat.com","username":"tristanC"},"change_message_id":"97c75edaf57cdd68b7eedf07f8e8683eeed1e369","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":8,"id":"50c4c2d5_541ecc74","updated":"2022-09-01 13:56:18.000000000","message":"This specification seems to address the many pain points of managing build node images, I\u0027m looking forward making use of it. Thanks!","commit_id":"781a8470d3790e6257fa5e813e9d8750c41b72bc"},{"author":{"_account_id":1,"name":"James E. Blair","email":"jim@acmegating.com","username":"corvus"},"change_message_id":"d4b62e133e4e1cc6a8791d587069d005bd40f138","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":8,"id":"b73ca1f0_13604a1d","in_reply_to":"6fbc8677_15325673","updated":"2022-09-01 21:13:17.000000000","message":"Good point.  I omitted it because I think we\u0027ve got a really good handle on that now in Nodepool with the way we use metadata, and I don\u0027t think that needs to change at all.","commit_id":"781a8470d3790e6257fa5e813e9d8750c41b72bc"}],"doc/source/developer/specs/nodepool-in-zuul.rst":[{"author":{"_account_id":2,"name":"Monty Taylor","email":"mordred@inaugust.com","username":"mordred"},"change_message_id":"5712c5043817a2096556f0f4aab3e9b30d284bc4","unresolved":true,"context_lines":[{"line_number":5,"context_line":"   are not currently available in Zuul.  They may change significantly"},{"line_number":6,"context_line":"   before final implementation, or may never be fully completed."},{"line_number":7,"context_line":""},{"line_number":8,"context_line":"The following specification describes plan to move Nodepool\u0027s"},{"line_number":9,"context_line":"functionality into Zuul and end development of Nodepool as a separate"},{"line_number":10,"context_line":"application.  This will allow for more node and image related features"},{"line_number":11,"context_line":"as well as simpler maintenance and deployment."}],"source_content_type":"text/x-rst","patch_set":1,"id":"97b30b6b_33094f85","line":8,"range":{"start_line":8,"start_character":38,"end_line":8,"end_character":42},"updated":"2022-07-14 18:27:21.000000000","message":"nit: a/the plan","commit_id":"032f625fd751f08ee572babe681088aa418c0447"},{"author":{"_account_id":1,"name":"James E. Blair","email":"jim@acmegating.com","username":"corvus"},"change_message_id":"761c9d23340ad66cab3e024af7c4ee8f27f5fe3e","unresolved":false,"context_lines":[{"line_number":5,"context_line":"   are not currently available in Zuul.  They may change significantly"},{"line_number":6,"context_line":"   before final implementation, or may never be fully completed."},{"line_number":7,"context_line":""},{"line_number":8,"context_line":"The following specification describes plan to move Nodepool\u0027s"},{"line_number":9,"context_line":"functionality into Zuul and end development of Nodepool as a separate"},{"line_number":10,"context_line":"application.  This will allow for more node and image related features"},{"line_number":11,"context_line":"as well as simpler maintenance and deployment."}],"source_content_type":"text/x-rst","patch_set":1,"id":"0ba2bae6_04af9494","line":8,"range":{"start_line":8,"start_character":38,"end_line":8,"end_character":42},"in_reply_to":"97b30b6b_33094f85","updated":"2022-07-15 00:27:37.000000000","message":"Done","commit_id":"032f625fd751f08ee572babe681088aa418c0447"},{"author":{"_account_id":2,"name":"Monty Taylor","email":"mordred@inaugust.com","username":"mordred"},"change_message_id":"5712c5043817a2096556f0f4aab3e9b30d284bc4","unresolved":true,"context_lines":[{"line_number":45,"context_line":"This spec contemplates a new Zuul component to handle image and node"},{"line_number":46,"context_line":"management: zuul-launcher.  Much of the Nodepool configuration will"},{"line_number":47,"context_line":"become Zuul configuration as well.  That is detailed in its own"},{"line_number":48,"context_line":"section, but for now, it\u0027s enough to know that the Zuul system as a"},{"line_number":49,"context_line":"whole will know what images and node labels are present in the"},{"line_number":50,"context_line":"configuration."},{"line_number":51,"context_line":""},{"line_number":52,"context_line":"Image Management"},{"line_number":53,"context_line":"----------------"}],"source_content_type":"text/x-rst","patch_set":1,"id":"664fb173_62c6b633","line":50,"range":{"start_line":48,"start_character":48,"end_line":50,"end_character":14},"updated":"2022-07-14 18:27:21.000000000","message":"That seems like a lovely win - as there is no way today to validate that a node label is real or not.","commit_id":"032f625fd751f08ee572babe681088aa418c0447"},{"author":{"_account_id":1,"name":"James E. Blair","email":"jim@acmegating.com","username":"corvus"},"change_message_id":"8777f8f2976362a26f3511458ab9e27e2140f1cf","unresolved":false,"context_lines":[{"line_number":47,"context_line":"become Zuul configuration as well.  That is detailed in its own"},{"line_number":48,"context_line":"section, but for now, it\u0027s enough to know that the Zuul system as a"},{"line_number":49,"context_line":"whole will know what images and node labels are present in the"},{"line_number":50,"context_line":"configuration."},{"line_number":51,"context_line":""},{"line_number":52,"context_line":"Image Management"},{"line_number":53,"context_line":"----------------"}],"source_content_type":"text/x-rst","patch_set":1,"id":"540230a8_0f4c4503","line":50,"updated":"2022-07-14 21:20:03.000000000","message":"Yeah, that can become a config error now instead of a runtime error.","commit_id":"032f625fd751f08ee572babe681088aa418c0447"},{"author":{"_account_id":1,"name":"James E. Blair","email":"jim@acmegating.com","username":"corvus"},"change_message_id":"761c9d23340ad66cab3e024af7c4ee8f27f5fe3e","unresolved":false,"context_lines":[{"line_number":45,"context_line":"This spec contemplates a new Zuul component to handle image and node"},{"line_number":46,"context_line":"management: zuul-launcher.  Much of the Nodepool configuration will"},{"line_number":47,"context_line":"become Zuul configuration as well.  That is detailed in its own"},{"line_number":48,"context_line":"section, but for now, it\u0027s enough to know that the Zuul system as a"},{"line_number":49,"context_line":"whole will know what images and node labels are present in the"},{"line_number":50,"context_line":"configuration."},{"line_number":51,"context_line":""},{"line_number":52,"context_line":"Image Management"},{"line_number":53,"context_line":"----------------"}],"source_content_type":"text/x-rst","patch_set":1,"id":"7395ecc0_02f8fe5e","line":50,"range":{"start_line":48,"start_character":48,"end_line":50,"end_character":14},"in_reply_to":"664fb173_62c6b633","updated":"2022-07-15 00:27:37.000000000","message":"Ack","commit_id":"032f625fd751f08ee572babe681088aa418c0447"},{"author":{"_account_id":2,"name":"Monty Taylor","email":"mordred@inaugust.com","username":"mordred"},"change_message_id":"5712c5043817a2096556f0f4aab3e9b30d284bc4","unresolved":true,"context_lines":[{"line_number":54,"context_line":""},{"line_number":55,"context_line":"Part of nodepool-builder\u0027s functionality is important to have as a"},{"line_number":56,"context_line":"long-running daemon, and part of what it does would make more sense as"},{"line_number":57,"context_line":"a Zuul job.  By moving the actual image build into a Zuul job, we can"},{"line_number":58,"context_line":"make the activity more visible to users of the system.  It will be"},{"line_number":59,"context_line":"easier for users to test changes to image builds (inasmuch as they can"},{"line_number":60,"context_line":"propose a change and a check job can run on that change to see if the"}],"source_content_type":"text/x-rst","patch_set":1,"id":"f2b262c7_1179d80e","line":57,"range":{"start_line":57,"start_character":34,"end_line":57,"end_character":61},"updated":"2022-07-14 18:27:21.000000000","message":"FWIW - maybe another point, or maybe not - is that people have from time to time expressed a desire to use something other than dib to build images. Zuul ultimately doesn\u0027t care what is used to create an image file - and Zuul jobs can kind of do whatever you want them to do. So it adds flexibility without also having to invent a new execution mechanism in nodepool.","commit_id":"032f625fd751f08ee572babe681088aa418c0447"},{"author":{"_account_id":1,"name":"James E. Blair","email":"jim@acmegating.com","username":"corvus"},"change_message_id":"8777f8f2976362a26f3511458ab9e27e2140f1cf","unresolved":false,"context_lines":[{"line_number":54,"context_line":""},{"line_number":55,"context_line":"Part of nodepool-builder\u0027s functionality is important to have as a"},{"line_number":56,"context_line":"long-running daemon, and part of what it does would make more sense as"},{"line_number":57,"context_line":"a Zuul job.  By moving the actual image build into a Zuul job, we can"},{"line_number":58,"context_line":"make the activity more visible to users of the system.  It will be"},{"line_number":59,"context_line":"easier for users to test changes to image builds (inasmuch as they can"},{"line_number":60,"context_line":"propose a change and a check job can run on that change to see if the"}],"source_content_type":"text/x-rst","patch_set":1,"id":"03c06ac9_23673d97","line":57,"updated":"2022-07-14 21:20:03.000000000","message":"Yes absolutely; I figure we\u0027ll have roles in zuul-jobs for running diskimage builder, but folks can do whatever to split out an image file.","commit_id":"032f625fd751f08ee572babe681088aa418c0447"},{"author":{"_account_id":1,"name":"James E. Blair","email":"jim@acmegating.com","username":"corvus"},"change_message_id":"761c9d23340ad66cab3e024af7c4ee8f27f5fe3e","unresolved":false,"context_lines":[{"line_number":54,"context_line":""},{"line_number":55,"context_line":"Part of nodepool-builder\u0027s functionality is important to have as a"},{"line_number":56,"context_line":"long-running daemon, and part of what it does would make more sense as"},{"line_number":57,"context_line":"a Zuul job.  By moving the actual image build into a Zuul job, we can"},{"line_number":58,"context_line":"make the activity more visible to users of the system.  It will be"},{"line_number":59,"context_line":"easier for users to test changes to image builds (inasmuch as they can"},{"line_number":60,"context_line":"propose a change and a check job can run on that change to see if the"}],"source_content_type":"text/x-rst","patch_set":1,"id":"f76ac547_876be0e8","line":57,"range":{"start_line":57,"start_character":34,"end_line":57,"end_character":61},"in_reply_to":"f2b262c7_1179d80e","updated":"2022-07-15 00:27:37.000000000","message":"Ack","commit_id":"032f625fd751f08ee572babe681088aa418c0447"},{"author":{"_account_id":2,"name":"Monty Taylor","email":"mordred@inaugust.com","username":"mordred"},"change_message_id":"5712c5043817a2096556f0f4aab3e9b30d284bc4","unresolved":true,"context_lines":[{"line_number":56,"context_line":"long-running daemon, and part of what it does would make more sense as"},{"line_number":57,"context_line":"a Zuul job.  By moving the actual image build into a Zuul job, we can"},{"line_number":58,"context_line":"make the activity more visible to users of the system.  It will be"},{"line_number":59,"context_line":"easier for users to test changes to image builds (inasmuch as they can"},{"line_number":60,"context_line":"propose a change and a check job can run on that change to see if the"},{"line_number":61,"context_line":"image builds sucessfully).  Build history and logs will be visible in"},{"line_number":62,"context_line":"the usual way in the Zuul web interface."}],"source_content_type":"text/x-rst","patch_set":1,"id":"6039229c_2bb01bff","line":59,"range":{"start_line":59,"start_character":21,"end_line":59,"end_character":48},"updated":"2022-07-14 18:27:21.000000000","message":"Yolanda would be so happy.","commit_id":"032f625fd751f08ee572babe681088aa418c0447"},{"author":{"_account_id":1,"name":"James E. Blair","email":"jim@acmegating.com","username":"corvus"},"change_message_id":"761c9d23340ad66cab3e024af7c4ee8f27f5fe3e","unresolved":false,"context_lines":[{"line_number":56,"context_line":"long-running daemon, and part of what it does would make more sense as"},{"line_number":57,"context_line":"a Zuul job.  By moving the actual image build into a Zuul job, we can"},{"line_number":58,"context_line":"make the activity more visible to users of the system.  It will be"},{"line_number":59,"context_line":"easier for users to test changes to image builds (inasmuch as they can"},{"line_number":60,"context_line":"propose a change and a check job can run on that change to see if the"},{"line_number":61,"context_line":"image builds sucessfully).  Build history and logs will be visible in"},{"line_number":62,"context_line":"the usual way in the Zuul web interface."}],"source_content_type":"text/x-rst","patch_set":1,"id":"83b52d42_d3f52918","line":59,"range":{"start_line":59,"start_character":21,"end_line":59,"end_character":48},"in_reply_to":"6039229c_2bb01bff","updated":"2022-07-15 00:27:37.000000000","message":"Ack","commit_id":"032f625fd751f08ee572babe681088aa418c0447"},{"author":{"_account_id":1,"name":"James E. Blair","email":"jim@acmegating.com","username":"corvus"},"change_message_id":"8777f8f2976362a26f3511458ab9e27e2140f1cf","unresolved":false,"context_lines":[{"line_number":103,"context_line":""},{"line_number":104,"context_line":"Currently nodepool-builder will build an image under three"},{"line_number":105,"context_line":"circumstances: 1) the image (or the image in a particular format) is"},{"line_number":106,"context_line":"missing; 2) a user has directly requested a build; 3) on an automatic"},{"line_number":107,"context_line":"interval (typically daily).  To map this into Zuul, we will use Zuul\u0027s"},{"line_number":108,"context_line":"existing pipeline functionality, but we will add a new trigger for"},{"line_number":109,"context_line":"case #1.  Case #2 can be handled by a manual Zuul enqueue command, and"}],"source_content_type":"text/x-rst","patch_set":1,"id":"3062b93c_0001a2f6","line":106,"updated":"2022-07-14 21:20:03.000000000","message":"Correct, I wasn\u0027t anticipating doing that.  But I think that would be possible.  We can have it periodically audit all parts of the system, and if an upload is missing, re-upload (and revalidate it -- but we\u0027d have to be careful to make sure the \"in-service\" flag is a one-way state transition -- we don\u0027t take images out of service once they go in).","commit_id":"032f625fd751f08ee572babe681088aa418c0447"},{"author":{"_account_id":2,"name":"Monty Taylor","email":"mordred@inaugust.com","username":"mordred"},"change_message_id":"5712c5043817a2096556f0f4aab3e9b30d284bc4","unresolved":true,"context_lines":[{"line_number":102,"context_line":"to deal with many variations of a similar process."},{"line_number":103,"context_line":""},{"line_number":104,"context_line":"Currently nodepool-builder will build an image under three"},{"line_number":105,"context_line":"circumstances: 1) the image (or the image in a particular format) is"},{"line_number":106,"context_line":"missing; 2) a user has directly requested a build; 3) on an automatic"},{"line_number":107,"context_line":"interval (typically daily).  To map this into Zuul, we will use Zuul\u0027s"},{"line_number":108,"context_line":"existing pipeline functionality, but we will add a new trigger for"},{"line_number":109,"context_line":"case #1.  Case #2 can be handled by a manual Zuul enqueue command, and"}],"source_content_type":"text/x-rst","patch_set":1,"id":"70df031c_ea715c9e","line":106,"range":{"start_line":105,"start_character":65,"end_line":106,"end_character":8},"updated":"2022-07-14 18:27:21.000000000","message":"The case described below is about images missing from zuul\u0027s zk state. There isn\u0027t anything for discovering that a remote image is missing when we had previously uploaded it, updating the state and re-emitting the zuul trigger, is there?","commit_id":"032f625fd751f08ee572babe681088aa418c0447"},{"author":{"_account_id":1,"name":"James E. Blair","email":"jim@acmegating.com","username":"corvus"},"change_message_id":"761c9d23340ad66cab3e024af7c4ee8f27f5fe3e","unresolved":false,"context_lines":[{"line_number":102,"context_line":"to deal with many variations of a similar process."},{"line_number":103,"context_line":""},{"line_number":104,"context_line":"Currently nodepool-builder will build an image under three"},{"line_number":105,"context_line":"circumstances: 1) the image (or the image in a particular format) is"},{"line_number":106,"context_line":"missing; 2) a user has directly requested a build; 3) on an automatic"},{"line_number":107,"context_line":"interval (typically daily).  To map this into Zuul, we will use Zuul\u0027s"},{"line_number":108,"context_line":"existing pipeline functionality, but we will add a new trigger for"},{"line_number":109,"context_line":"case #1.  Case #2 can be handled by a manual Zuul enqueue command, and"}],"source_content_type":"text/x-rst","patch_set":1,"id":"69dcd06d_afd9c709","line":106,"range":{"start_line":105,"start_character":65,"end_line":106,"end_character":8},"in_reply_to":"70df031c_ea715c9e","updated":"2022-07-15 00:27:37.000000000","message":"Ack","commit_id":"032f625fd751f08ee572babe681088aa418c0447"},{"author":{"_account_id":1,"name":"James E. Blair","email":"jim@acmegating.com","username":"corvus"},"change_message_id":"8777f8f2976362a26f3511458ab9e27e2140f1cf","unresolved":false,"context_lines":[{"line_number":124,"context_line":"jobs matching the image that needs building are run.  Second, it will"},{"line_number":125,"context_line":"allow Zuul to determine which formats are needed for that image (based"},{"line_number":126,"context_line":"on which providers are configured to use it) and include that"},{"line_number":127,"context_line":"information as job data."},{"line_number":128,"context_line":""},{"line_number":129,"context_line":"The job will be responsible for building the image and uploading the"},{"line_number":130,"context_line":"result to some storage system.  The URLs for each image format built"}],"source_content_type":"text/x-rst","patch_set":1,"id":"2284f436_ad6e6f3d","line":127,"updated":"2022-07-14 21:20:03.000000000","message":"Oh interesting.  I think that would work with what\u0027s written here without any changes.  You\u0027d just make sure both the parent and child jobs had the image name matcher so they both run, then the parent job can return the raw image artifact, and the child job can fetch that and do the conversion and return the vhd artifact.  The zuul reporter would look at all the jobs for artifacts, so it would see both.  And if the parent job returned an intermediate artifact the child job used but the zuul driver isn\u0027t configured to look for, that\u0027s fine too, it\u0027ll just ignore that.","commit_id":"032f625fd751f08ee572babe681088aa418c0447"},{"author":{"_account_id":2,"name":"Monty Taylor","email":"mordred@inaugust.com","username":"mordred"},"change_message_id":"5712c5043817a2096556f0f4aab3e9b30d284bc4","unresolved":true,"context_lines":[{"line_number":121,"context_line":"Jobs will include an extra attribute to indicate they build a"},{"line_number":122,"context_line":"particular image.  This serves two purposes; first, in the case of an"},{"line_number":123,"context_line":"`image-build` trigger event, it will act as a matcher so that only"},{"line_number":124,"context_line":"jobs matching the image that needs building are run.  Second, it will"},{"line_number":125,"context_line":"allow Zuul to determine which formats are needed for that image (based"},{"line_number":126,"context_line":"on which providers are configured to use it) and include that"},{"line_number":127,"context_line":"information as job data."},{"line_number":128,"context_line":""},{"line_number":129,"context_line":"The job will be responsible for building the image and uploading the"},{"line_number":130,"context_line":"result to some storage system.  The URLs for each image format built"}],"source_content_type":"text/x-rst","patch_set":1,"id":"52909554_09b8c043","line":127,"range":{"start_line":124,"start_character":54,"end_line":127,"end_character":24},"updated":"2022-07-14 18:27:21.000000000","message":"Some image formats are harder than others. Would it be possible to have a parent job build the raw image and a child job do the raw-to-format conversion? Thinking about using a fedora node to build a fedora image and then an ubuntu node to convert it to vhd. Granted - the fedora image could just podman run a vhd-util container","commit_id":"032f625fd751f08ee572babe681088aa418c0447"},{"author":{"_account_id":1,"name":"James E. Blair","email":"jim@acmegating.com","username":"corvus"},"change_message_id":"761c9d23340ad66cab3e024af7c4ee8f27f5fe3e","unresolved":false,"context_lines":[{"line_number":121,"context_line":"Jobs will include an extra attribute to indicate they build a"},{"line_number":122,"context_line":"particular image.  This serves two purposes; first, in the case of an"},{"line_number":123,"context_line":"`image-build` trigger event, it will act as a matcher so that only"},{"line_number":124,"context_line":"jobs matching the image that needs building are run.  Second, it will"},{"line_number":125,"context_line":"allow Zuul to determine which formats are needed for that image (based"},{"line_number":126,"context_line":"on which providers are configured to use it) and include that"},{"line_number":127,"context_line":"information as job data."},{"line_number":128,"context_line":""},{"line_number":129,"context_line":"The job will be responsible for building the image and uploading the"},{"line_number":130,"context_line":"result to some storage system.  The URLs for each image format built"}],"source_content_type":"text/x-rst","patch_set":1,"id":"a77238e5_820f629f","line":127,"range":{"start_line":124,"start_character":54,"end_line":127,"end_character":24},"in_reply_to":"52909554_09b8c043","updated":"2022-07-15 00:27:37.000000000","message":"Ack","commit_id":"032f625fd751f08ee572babe681088aa418c0447"},{"author":{"_account_id":2,"name":"Monty Taylor","email":"mordred@inaugust.com","username":"mordred"},"change_message_id":"5712c5043817a2096556f0f4aab3e9b30d284bc4","unresolved":true,"context_lines":[{"line_number":178,"context_line":""},{"line_number":179,"context_line":"Zuul will update internal records in ZooKeeper for the image to record"},{"line_number":180,"context_line":"the storage URLs.  The zuul-launcher process will then start"},{"line_number":181,"context_line":"background processes to download the images from the storage system"},{"line_number":182,"context_line":"and upload them to the configured providers (much as nodepool-builder"},{"line_number":183,"context_line":"does now with files on disk).  As a special case, it may detect that"},{"line_number":184,"context_line":"the image files are stored in a location that a provider can access"}],"source_content_type":"text/x-rst","patch_set":1,"id":"d1628ca6_f2bda5ee","line":181,"range":{"start_line":181,"start_character":25,"end_line":181,"end_character":67},"updated":"2022-07-14 18:27:21.000000000","message":"This could get fun with builder nodes being far away from the storage system that is near to the zuul-launcher.","commit_id":"032f625fd751f08ee572babe681088aa418c0447"},{"author":{"_account_id":1,"name":"James E. Blair","email":"jim@acmegating.com","username":"corvus"},"change_message_id":"8777f8f2976362a26f3511458ab9e27e2140f1cf","unresolved":false,"context_lines":[{"line_number":178,"context_line":""},{"line_number":179,"context_line":"Zuul will update internal records in ZooKeeper for the image to record"},{"line_number":180,"context_line":"the storage URLs.  The zuul-launcher process will then start"},{"line_number":181,"context_line":"background processes to download the images from the storage system"},{"line_number":182,"context_line":"and upload them to the configured providers (much as nodepool-builder"},{"line_number":183,"context_line":"does now with files on disk).  As a special case, it may detect that"},{"line_number":184,"context_line":"the image files are stored in a location that a provider can access"}],"source_content_type":"text/x-rst","patch_set":1,"id":"e38f45f5_2b1b0984","line":181,"updated":"2022-07-14 21:20:03.000000000","message":"Yes, there may be more really long network transfers in this system than we see right now with the builders.  A site like opendev with multiple options for storing image files might want to carefully select what object storage to use for them.  Also consider gzipping them and uploading them with an appropriate content-encoding header, etc.","commit_id":"032f625fd751f08ee572babe681088aa418c0447"},{"author":{"_account_id":1,"name":"James E. Blair","email":"jim@acmegating.com","username":"corvus"},"change_message_id":"761c9d23340ad66cab3e024af7c4ee8f27f5fe3e","unresolved":false,"context_lines":[{"line_number":178,"context_line":""},{"line_number":179,"context_line":"Zuul will update internal records in ZooKeeper for the image to record"},{"line_number":180,"context_line":"the storage URLs.  The zuul-launcher process will then start"},{"line_number":181,"context_line":"background processes to download the images from the storage system"},{"line_number":182,"context_line":"and upload them to the configured providers (much as nodepool-builder"},{"line_number":183,"context_line":"does now with files on disk).  As a special case, it may detect that"},{"line_number":184,"context_line":"the image files are stored in a location that a provider can access"}],"source_content_type":"text/x-rst","patch_set":1,"id":"79645d4d_075b9755","line":181,"range":{"start_line":181,"start_character":25,"end_line":181,"end_character":67},"in_reply_to":"d1628ca6_f2bda5ee","updated":"2022-07-15 00:27:37.000000000","message":"Ack","commit_id":"032f625fd751f08ee572babe681088aa418c0447"},{"author":{"_account_id":2,"name":"Monty Taylor","email":"mordred@inaugust.com","username":"mordred"},"change_message_id":"5712c5043817a2096556f0f4aab3e9b30d284bc4","unresolved":true,"context_lines":[{"line_number":221,"context_line":"fulfilled with a node built from the image under test.  This process"},{"line_number":222,"context_line":"may repeat for each of the providers using that image (normal pipeline"},{"line_number":223,"context_line":"queue deduplication rules may need a special case to allow this)."},{"line_number":224,"context_line":"Once the validation jobs pass, the entry in ZooKeeper will be updated"},{"line_number":225,"context_line":"and the image will go into regular service."},{"line_number":226,"context_line":""},{"line_number":227,"context_line":"A more specific process definition follows:"}],"source_content_type":"text/x-rst","patch_set":1,"id":"6e71a4b9_e052a7a6","line":224,"range":{"start_line":224,"start_character":1,"end_line":224,"end_character":29},"updated":"2022-07-14 18:27:21.000000000","message":"Does an image with a failed validation indicate to zuul that it should emit the event for needing one of these images?","commit_id":"032f625fd751f08ee572babe681088aa418c0447"},{"author":{"_account_id":1,"name":"James E. Blair","email":"jim@acmegating.com","username":"corvus"},"change_message_id":"8777f8f2976362a26f3511458ab9e27e2140f1cf","unresolved":false,"context_lines":[{"line_number":221,"context_line":"fulfilled with a node built from the image under test.  This process"},{"line_number":222,"context_line":"may repeat for each of the providers using that image (normal pipeline"},{"line_number":223,"context_line":"queue deduplication rules may need a special case to allow this)."},{"line_number":224,"context_line":"Once the validation jobs pass, the entry in ZooKeeper will be updated"},{"line_number":225,"context_line":"and the image will go into regular service."},{"line_number":226,"context_line":""},{"line_number":227,"context_line":"A more specific process definition follows:"}],"source_content_type":"text/x-rst","patch_set":1,"id":"a36a7099_03d699ee","line":224,"updated":"2022-07-14 21:20:03.000000000","message":"Yes, I think at that point we should emit an image-delete event, probably wait until that image is cleaned up (so that we don\u0027t pile up a bunch of bad images in the object storage system) then emit another build-image event.\n\nPerhaps there should be a configurable interval too, so we might wait an extra X hours?","commit_id":"032f625fd751f08ee572babe681088aa418c0447"},{"author":{"_account_id":1,"name":"James E. Blair","email":"jim@acmegating.com","username":"corvus"},"change_message_id":"761c9d23340ad66cab3e024af7c4ee8f27f5fe3e","unresolved":false,"context_lines":[{"line_number":221,"context_line":"fulfilled with a node built from the image under test.  This process"},{"line_number":222,"context_line":"may repeat for each of the providers using that image (normal pipeline"},{"line_number":223,"context_line":"queue deduplication rules may need a special case to allow this)."},{"line_number":224,"context_line":"Once the validation jobs pass, the entry in ZooKeeper will be updated"},{"line_number":225,"context_line":"and the image will go into regular service."},{"line_number":226,"context_line":""},{"line_number":227,"context_line":"A more specific process definition follows:"}],"source_content_type":"text/x-rst","patch_set":1,"id":"108a4e0d_ddfc4925","line":224,"range":{"start_line":224,"start_character":1,"end_line":224,"end_character":29},"in_reply_to":"6e71a4b9_e052a7a6","updated":"2022-07-15 00:27:37.000000000","message":"Ack","commit_id":"032f625fd751f08ee572babe681088aa418c0447"},{"author":{"_account_id":2,"name":"Monty Taylor","email":"mordred@inaugust.com","username":"mordred"},"change_message_id":"5712c5043817a2096556f0f4aab3e9b30d284bc4","unresolved":true,"context_lines":[{"line_number":274,"context_line":"require us to support snapshot image builds, but in case we want to"},{"line_number":275,"context_line":"add support in the future, we should have a plan for it."},{"line_number":276,"context_line":""},{"line_number":277,"context_line":"The image build job in Zuul could, instead of running"},{"line_number":278,"context_line":"diskimage-builder, act on the remote node to prepare it for a"},{"line_number":279,"context_line":"snapshot.  A special job attribute could indicate that it is a"},{"line_number":280,"context_line":"snapshot image job, and instead of having the zuul-launcher component"}],"source_content_type":"text/x-rst","patch_set":1,"id":"45579aac_abaddfb8","line":277,"range":{"start_line":277,"start_character":0,"end_line":277,"end_character":34},"updated":"2022-07-14 18:27:21.000000000","message":"This spec describes a build job storing the location of the image in an artifact. Potentially instead of an artifact, in the case of snapshots, it could store the location of the image id in the cloud? Then the upload step can be a no-op but could trigger the validation workflow and the internal data upkeep.\n\nThis could likely be very achievable for people with only one cloud.","commit_id":"032f625fd751f08ee572babe681088aa418c0447"},{"author":{"_account_id":1,"name":"James E. Blair","email":"jim@acmegating.com","username":"corvus"},"change_message_id":"8777f8f2976362a26f3511458ab9e27e2140f1cf","unresolved":false,"context_lines":[{"line_number":274,"context_line":"require us to support snapshot image builds, but in case we want to"},{"line_number":275,"context_line":"add support in the future, we should have a plan for it."},{"line_number":276,"context_line":""},{"line_number":277,"context_line":"The image build job in Zuul could, instead of running"},{"line_number":278,"context_line":"diskimage-builder, act on the remote node to prepare it for a"},{"line_number":279,"context_line":"snapshot.  A special job attribute could indicate that it is a"},{"line_number":280,"context_line":"snapshot image job, and instead of having the zuul-launcher component"}],"source_content_type":"text/x-rst","patch_set":1,"id":"3a46c6b1_10c14dcc","line":277,"updated":"2022-07-14 21:20:03.000000000","message":"Well, what I was trying to get at here is that we should let zuul-launcher perform the actual API call to make the snapshot.  So the job runs on a node and does a bunch of stuff to the filesystem.  Then, instead of having the job call \"openstack image create\" or whatever (since that would need credentials in a zuul secret), we would just have the job end, and then zuul-launcher will perform that api call before deleting the node.  Then that image location gets stored in ZK.\n\nMostly, I described the above so that Zuul secrets with cloud creds aren\u0027t needed in the job.\n\nHaving said all that, I agree your process would work as well, with fewer changes needed in Zuul, just with the caveat about the creds needed to make the snapshot.","commit_id":"032f625fd751f08ee572babe681088aa418c0447"},{"author":{"_account_id":2,"name":"Monty Taylor","email":"mordred@inaugust.com","username":"mordred"},"change_message_id":"2860c802542f107c25d6d3eb4b9fc205c6c47001","unresolved":false,"context_lines":[{"line_number":274,"context_line":"require us to support snapshot image builds, but in case we want to"},{"line_number":275,"context_line":"add support in the future, we should have a plan for it."},{"line_number":276,"context_line":""},{"line_number":277,"context_line":"The image build job in Zuul could, instead of running"},{"line_number":278,"context_line":"diskimage-builder, act on the remote node to prepare it for a"},{"line_number":279,"context_line":"snapshot.  A special job attribute could indicate that it is a"},{"line_number":280,"context_line":"snapshot image job, and instead of having the zuul-launcher component"}],"source_content_type":"text/x-rst","patch_set":1,"id":"5edc4133_7d1c4778","line":277,"in_reply_to":"3a46c6b1_10c14dcc","updated":"2022-07-14 21:51:19.000000000","message":"Ah - totally - that\u0027s a great point. I think I was thinking about things like packer where the zuul node would be where you\u0027d run packer - but you\u0027d already need creds for packer to create _another_ VM to specialize. I like your version better, of course, because I don\u0027t like packer in the first place.","commit_id":"032f625fd751f08ee572babe681088aa418c0447"},{"author":{"_account_id":1,"name":"James E. Blair","email":"jim@acmegating.com","username":"corvus"},"change_message_id":"761c9d23340ad66cab3e024af7c4ee8f27f5fe3e","unresolved":false,"context_lines":[{"line_number":274,"context_line":"require us to support snapshot image builds, but in case we want to"},{"line_number":275,"context_line":"add support in the future, we should have a plan for it."},{"line_number":276,"context_line":""},{"line_number":277,"context_line":"The image build job in Zuul could, instead of running"},{"line_number":278,"context_line":"diskimage-builder, act on the remote node to prepare it for a"},{"line_number":279,"context_line":"snapshot.  A special job attribute could indicate that it is a"},{"line_number":280,"context_line":"snapshot image job, and instead of having the zuul-launcher component"}],"source_content_type":"text/x-rst","patch_set":1,"id":"50264ec8_0802b56d","line":277,"range":{"start_line":277,"start_character":0,"end_line":277,"end_character":34},"in_reply_to":"45579aac_abaddfb8","updated":"2022-07-15 00:27:37.000000000","message":"Ack","commit_id":"032f625fd751f08ee572babe681088aa418c0447"},{"author":{"_account_id":1,"name":"James E. Blair","email":"jim@acmegating.com","username":"corvus"},"change_message_id":"8777f8f2976362a26f3511458ab9e27e2140f1cf","unresolved":false,"context_lines":[{"line_number":353,"context_line":""},{"line_number":354,"context_line":"The configuration currently handled by Nodepool will be refactored and"},{"line_number":355,"context_line":"added to Zuul\u0027s tenant configuration file.  This means that it will be"},{"line_number":356,"context_line":"non-speculative and not loaded directly from git repos.  Instead it"},{"line_number":357,"context_line":"will be static configuration in the same way that tenants, global"},{"line_number":358,"context_line":"semaphores, and authorization rules are managed.  A reconfiguration"},{"line_number":359,"context_line":"event will be needed to prompt Zuul to reload the configuration file"}],"source_content_type":"text/x-rst","patch_set":1,"id":"99f75ecf_856173f9","line":356,"updated":"2022-07-14 21:20:03.000000000","message":"Yeah it would... I\u0027ve been thinking about that and it just seems messy though because we still want images to be \"global\".\n\nI mean, we could totally have a repo where we have \"image\" stanzas like below.  Let\u0027s use opendev as an example.  We could have \"opendev/images\" with that configuration in it.  And then we could include opendev/images in every tenant (but with \u0027include: [image]\u0027 in most tenants, and only include jobs, etc, in the \"opendev\" tenant).  Then every tenant gets access to those images.  If a tenant has extra \"image\" definitions, it gets access to those extra images as well.  For example, let\u0027s say we have an \"openstack/images\" repo where we make special devstack images for openstack.  So far so good.\n\nSo what if we have:\n\nopendev/images: has a \"debian-unstable\" image\nopenstack/images: has a \"devstack\" image\nzuul/images: decides to define its own \"devstack\" image\n\nSince our image names are global, we have a name conflict and it\u0027s a bit tricky to resolve because the actual image exists outside of the context of a tenant (I\u0027m taking it for granted we don\u0027t want to build/upload copies of images for each tenant).\n\nMaybe what we could do is, internally, we go ahead and use the canonical name.  So in ZooKeeper, we store image information at:\n\n/zuul/images/opendev.org%2fopendev%fimages%2fdebian-unstable/...\n\nAnd then when we go to run a job, we use the configuration objects in that tenant to resolve the label down to an image, and whatever repo defines that devstack image we use the full canonical image name.  So if the job says to use the \"devstack\" label and the \"devstack\" label says to use the \"devstack\" image, if that job is running in the openstack tenant, it\u0027s going to get the \"opendev.org/openstack/images/devstack\" and if a job runs in the zuul tenant, it\u0027s going to get \"opendev.org/zuul/images/devstack\".  Which is probably what each expects.\n\nThat might actually work.","commit_id":"032f625fd751f08ee572babe681088aa418c0447"},{"author":{"_account_id":2,"name":"Monty Taylor","email":"mordred@inaugust.com","username":"mordred"},"change_message_id":"5712c5043817a2096556f0f4aab3e9b30d284bc4","unresolved":true,"context_lines":[{"line_number":353,"context_line":""},{"line_number":354,"context_line":"The configuration currently handled by Nodepool will be refactored and"},{"line_number":355,"context_line":"added to Zuul\u0027s tenant configuration file.  This means that it will be"},{"line_number":356,"context_line":"non-speculative and not loaded directly from git repos.  Instead it"},{"line_number":357,"context_line":"will be static configuration in the same way that tenants, global"},{"line_number":358,"context_line":"semaphores, and authorization rules are managed.  A reconfiguration"},{"line_number":359,"context_line":"event will be needed to prompt Zuul to reload the configuration file"}],"source_content_type":"text/x-rst","patch_set":1,"id":"d86a4e6e_8e1933f9","line":356,"range":{"start_line":356,"start_character":0,"end_line":356,"end_character":16},"updated":"2022-07-14 18:27:21.000000000","message":"awww. you know it would be totally cool to have job use an new image and depend-on the change to add the image to the system. ;)","commit_id":"032f625fd751f08ee572babe681088aa418c0447"},{"author":{"_account_id":1,"name":"James E. Blair","email":"jim@acmegating.com","username":"corvus"},"change_message_id":"761c9d23340ad66cab3e024af7c4ee8f27f5fe3e","unresolved":false,"context_lines":[{"line_number":353,"context_line":""},{"line_number":354,"context_line":"The configuration currently handled by Nodepool will be refactored and"},{"line_number":355,"context_line":"added to Zuul\u0027s tenant configuration file.  This means that it will be"},{"line_number":356,"context_line":"non-speculative and not loaded directly from git repos.  Instead it"},{"line_number":357,"context_line":"will be static configuration in the same way that tenants, global"},{"line_number":358,"context_line":"semaphores, and authorization rules are managed.  A reconfiguration"},{"line_number":359,"context_line":"event will be needed to prompt Zuul to reload the configuration file"}],"source_content_type":"text/x-rst","patch_set":1,"id":"874fc1ea_5efcc1ee","line":356,"range":{"start_line":356,"start_character":0,"end_line":356,"end_character":16},"in_reply_to":"d86a4e6e_8e1933f9","updated":"2022-07-15 00:27:37.000000000","message":"Ack","commit_id":"032f625fd751f08ee572babe681088aa418c0447"},{"author":{"_account_id":27582,"name":"Simon Westphahl","email":"simon.westphahl@bmw.de","username":"simon.westphahl"},"change_message_id":"58cada5484d2538cc631cf9479249943232a4433","unresolved":true,"context_lines":[{"line_number":399,"context_line":"   - image:"},{"line_number":400,"context_line":"       name: debian-unstable"},{"line_number":401,"context_line":""},{"line_number":402,"context_line":"   - provider:"},{"line_number":403,"context_line":"       name: rax-dfw"},{"line_number":404,"context_line":"       driver: openstack"},{"line_number":405,"context_line":"       pools:"}],"source_content_type":"text/x-rst","patch_set":3,"id":"78d737b8_495bf2cd","line":402,"updated":"2022-07-21 12:43:17.000000000","message":"Given the use case of a shared repository that is part of multiple tenants and defines some generic labels/images and a provider, we should allow for attaching additional, tenant specific labels to an existing provider pool.\n\nOtherwise each tenant would need a separate provider and/or pool config for its own labels.","commit_id":"961a5c3b5df925939ebb0031682f9113317b9a18"},{"author":{"_account_id":1,"name":"James E. Blair","email":"jim@acmegating.com","username":"corvus"},"change_message_id":"dc40bb40c1186559c75882d29e7a83b192a4d205","unresolved":false,"context_lines":[{"line_number":399,"context_line":"   - image:"},{"line_number":400,"context_line":"       name: debian-unstable"},{"line_number":401,"context_line":""},{"line_number":402,"context_line":"   - provider:"},{"line_number":403,"context_line":"       name: rax-dfw"},{"line_number":404,"context_line":"       driver: openstack"},{"line_number":405,"context_line":"       pools:"}],"source_content_type":"text/x-rst","patch_set":3,"id":"bf86ccf2_c78411ac","line":402,"in_reply_to":"78d737b8_495bf2cd","updated":"2022-07-23 22:28:04.000000000","message":"Good point, I\u0027ll elaborate on the configuration syntax a bit and make sure we can do that.","commit_id":"961a5c3b5df925939ebb0031682f9113317b9a18"},{"author":{"_account_id":27582,"name":"Simon Westphahl","email":"simon.westphahl@bmw.de","username":"simon.westphahl"},"change_message_id":"58cada5484d2538cc631cf9479249943232a4433","unresolved":true,"context_lines":[{"line_number":401,"context_line":""},{"line_number":402,"context_line":"   - provider:"},{"line_number":403,"context_line":"       name: rax-dfw"},{"line_number":404,"context_line":"       driver: openstack"},{"line_number":405,"context_line":"       pools:"},{"line_number":406,"context_line":"         - name: main"},{"line_number":407,"context_line":"           labels:"}],"source_content_type":"text/x-rst","patch_set":3,"id":"fe6b557a_6bba3960","line":404,"updated":"2022-07-21 12:43:17.000000000","message":"nit: Since providers will probably be configured in the Zuul config similar to the connections, we don\u0027t have to name the driver here.","commit_id":"961a5c3b5df925939ebb0031682f9113317b9a18"},{"author":{"_account_id":1,"name":"James E. Blair","email":"jim@acmegating.com","username":"corvus"},"change_message_id":"dc40bb40c1186559c75882d29e7a83b192a4d205","unresolved":false,"context_lines":[{"line_number":401,"context_line":""},{"line_number":402,"context_line":"   - provider:"},{"line_number":403,"context_line":"       name: rax-dfw"},{"line_number":404,"context_line":"       driver: openstack"},{"line_number":405,"context_line":"       pools:"},{"line_number":406,"context_line":"         - name: main"},{"line_number":407,"context_line":"           labels:"}],"source_content_type":"text/x-rst","patch_set":3,"id":"d0305216_0ac669ec","line":404,"in_reply_to":"fe6b557a_6bba3960","updated":"2022-07-23 22:28:04.000000000","message":"Yep.","commit_id":"961a5c3b5df925939ebb0031682f9113317b9a18"},{"author":{"_account_id":1,"name":"James E. Blair","email":"jim@acmegating.com","username":"corvus"},"change_message_id":"d3aadd8524718b895d054fa969dce85bff83ab9e","unresolved":false,"context_lines":[{"line_number":387,"context_line":"necessary since they will be encoded in Zuul jobs.  This will reduce"},{"line_number":388,"context_line":"the complexity of the configuration significantly."},{"line_number":389,"context_line":""},{"line_number":390,"context_line":"The provider configuration will be similar to what is provided to"},{"line_number":391,"context_line":"nodepool-launcher now, except that we will take the opportunity to"},{"line_number":392,"context_line":"make it more \"Zuul-like\".  Instead of a top-level dictionary, we will"},{"line_number":393,"context_line":"have a list.  We will standardize on attributes used across drivers"}],"source_content_type":"text/x-rst","patch_set":4,"id":"cd0ae64f_7d65cc76","line":390,"updated":"2022-07-25 17:26:48.000000000","message":"How about we add a command line argument to zuul-launcher telling it to limit to a set of connections?  (eg \"zuul-launcher --provider rackspace --provider ovh\")  I suggest that so that we can keep all connections in zuul.conf (we recently had issues with zuul-web needing extra connection info which it didn\u0027t otherwise use in order to fully parse the configuration; I could see something similar happening here).\n\nOr we could add \"enabled\u003dfalse\" to connections which would tell that launcher not to handle that connection.\n\nIn summary, 3 options:\n1) List connections in [zuul-launcher]\n2) Command line argument\n3) Enabled flag in [connection foo] sections\n\nI think we can do [(1 xor 3) or 2].  But really, having only one way to do it would be best.  I lean toward only doing #2.","commit_id":"2ccc41a16d15c6330c83ff917c239fd5c0cdcbea"},{"author":{"_account_id":16068,"name":"Tobias Henkel","email":"tobias.henkel@bmw.de","username":"tobias.henkel"},"change_message_id":"26f49161a99d07c9c319adde6e9b2690c0d613b5","unresolved":true,"context_lines":[{"line_number":387,"context_line":"necessary since they will be encoded in Zuul jobs.  This will reduce"},{"line_number":388,"context_line":"the complexity of the configuration significantly."},{"line_number":389,"context_line":""},{"line_number":390,"context_line":"The provider configuration will be similar to what is provided to"},{"line_number":391,"context_line":"nodepool-launcher now, except that we will take the opportunity to"},{"line_number":392,"context_line":"make it more \"Zuul-like\".  Instead of a top-level dictionary, we will"},{"line_number":393,"context_line":"have a list.  We will standardize on attributes used across drivers"}],"source_content_type":"text/x-rst","patch_set":4,"id":"11aa2752_a849f4d3","line":390,"updated":"2022-07-24 11:51:06.000000000","message":"I think what\u0027s missing here is how can we define which launcher can work with which provider. We have the use case that our nodepool-builders are disjoint (some are for onprem some for aws) and the images we build should not cross the onprem/aws boundary.\n\nI guess one way could be by only having a subset of connections specified in the individual zuul.conf files and ignoring all providers that link to an unknown connection.\n\nA second way could be to list all connections in the [zuul-launcher] section that should be serviced by a launcher.","commit_id":"2ccc41a16d15c6330c83ff917c239fd5c0cdcbea"},{"author":{"_account_id":1,"name":"James E. Blair","email":"jim@acmegating.com","username":"corvus"},"change_message_id":"2708265501399520ca579790965d16d6b9232ee0","unresolved":true,"context_lines":[{"line_number":387,"context_line":"necessary since they will be encoded in Zuul jobs.  This will reduce"},{"line_number":388,"context_line":"the complexity of the configuration significantly."},{"line_number":389,"context_line":""},{"line_number":390,"context_line":"The provider configuration will be similar to what is provided to"},{"line_number":391,"context_line":"nodepool-launcher now, except that we will take the opportunity to"},{"line_number":392,"context_line":"make it more \"Zuul-like\".  Instead of a top-level dictionary, we will"},{"line_number":393,"context_line":"have a list.  We will standardize on attributes used across drivers"}],"source_content_type":"text/x-rst","patch_set":4,"id":"9d8ebbdc_496d8dbb","line":390,"in_reply_to":"11aa2752_a849f4d3","updated":"2022-07-25 17:30:21.000000000","message":"How about we add a command line argument to zuul-launcher telling it to limit to a set of connections?  (eg \"zuul-launcher --provider rackspace --provider ovh\")  I suggest that so that we can keep all connections in zuul.conf (we recently had issues with zuul-web needing extra connection info which it didn\u0027t otherwise use in order to fully parse the configuration; I could see something similar happening here).\n\nOr we could add \"enabled\u003dfalse\" to connections which would tell that launcher not to handle that connection.\n\nIn summary, 3 options:\n1) List connections in [zuul-launcher]\n2) Command line argument\n3) Enabled flag in [connection foo] sections\n\nI think we can do [(1 xor 3) or 2].  But really, having only one way to do it would be best.  I lean toward only doing #2.","commit_id":"2ccc41a16d15c6330c83ff917c239fd5c0cdcbea"},{"author":{"_account_id":1,"name":"James E. Blair","email":"jim@acmegating.com","username":"corvus"},"change_message_id":"9313012343bc4a68db49dcc68df1e991f0c885d2","unresolved":false,"context_lines":[{"line_number":387,"context_line":"necessary since they will be encoded in Zuul jobs.  This will reduce"},{"line_number":388,"context_line":"the complexity of the configuration significantly."},{"line_number":389,"context_line":""},{"line_number":390,"context_line":"The provider configuration will be similar to what is provided to"},{"line_number":391,"context_line":"nodepool-launcher now, except that we will take the opportunity to"},{"line_number":392,"context_line":"make it more \"Zuul-like\".  Instead of a top-level dictionary, we will"},{"line_number":393,"context_line":"have a list.  We will standardize on attributes used across drivers"}],"source_content_type":"text/x-rst","patch_set":4,"id":"34fcc545_d9574e12","line":390,"in_reply_to":"9d8ebbdc_496d8dbb","updated":"2022-07-29 00:38:26.000000000","message":"I will add a command line argument in the next version.","commit_id":"2ccc41a16d15c6330c83ff917c239fd5c0cdcbea"},{"author":{"_account_id":16068,"name":"Tobias Henkel","email":"tobias.henkel@bmw.de","username":"tobias.henkel"},"change_message_id":"c086502f2fd1023878decd86e4f6a7379b0ebd93","unresolved":false,"context_lines":[{"line_number":387,"context_line":"necessary since they will be encoded in Zuul jobs.  This will reduce"},{"line_number":388,"context_line":"the complexity of the configuration significantly."},{"line_number":389,"context_line":""},{"line_number":390,"context_line":"The provider configuration will be similar to what is provided to"},{"line_number":391,"context_line":"nodepool-launcher now, except that we will take the opportunity to"},{"line_number":392,"context_line":"make it more \"Zuul-like\".  Instead of a top-level dictionary, we will"},{"line_number":393,"context_line":"have a list.  We will standardize on attributes used across drivers"}],"source_content_type":"text/x-rst","patch_set":4,"id":"def7784d_76fc6da8","line":390,"in_reply_to":"cd0ae64f_7d65cc76","updated":"2022-07-26 15:01:29.000000000","message":"I think command line args are just fine. We do essentially the same with our config generation script for nodepool.","commit_id":"2ccc41a16d15c6330c83ff917c239fd5c0cdcbea"},{"author":{"_account_id":16068,"name":"Tobias Henkel","email":"tobias.henkel@bmw.de","username":"tobias.henkel"},"change_message_id":"26f49161a99d07c9c319adde6e9b2690c0d613b5","unresolved":true,"context_lines":[{"line_number":410,"context_line":"       min-ready: 1"},{"line_number":411,"context_line":""},{"line_number":412,"context_line":"   - label:"},{"line_number":413,"context_line":"       name: ubuntu"},{"line_number":414,"context_line":""},{"line_number":415,"context_line":"   - provider:"},{"line_number":416,"context_line":"       name: rax-dfw"}],"source_content_type":"text/x-rst","patch_set":4,"id":"9aaa8a9a_91dbc9d7","line":413,"updated":"2022-07-24 11:51:06.000000000","message":"Here few thoughts while thinking about our use cases.\n\nOur users should not have/need knowledge about the instnace flavors that are used/available in the backend. Having the possibility to define virtual flavors could achieve that. \n\nMany of our labels are specified by our projects to run in specific regions aka providers. It could be tedious to have labels and provider/pool attachments distributed. I think it might make sense to also allow to specify the pools that should serve a label directly on the label. In order to achieve that we\u0027d need to allow specifying the image, virtual flavor (I think this would likely be required for this use case) and possibly further information on the label stanza.\n\nWe also need to restrict the available flavors that are available to tenants. This could also achieved by defining virtual labels and somehow restrict defining them via the tenant config to certain repos via the include/exclude of object types in the tenant config. In order to facilitate the config it might make sense to define default includes/excludes on a tenant (e.g. exclude by default certain nodepool object types) and enable them specifically on certain repos that are intended to hold nodepool config.","commit_id":"2ccc41a16d15c6330c83ff917c239fd5c0cdcbea"},{"author":{"_account_id":1,"name":"James E. Blair","email":"jim@acmegating.com","username":"corvus"},"change_message_id":"d3aadd8524718b895d054fa969dce85bff83ab9e","unresolved":false,"context_lines":[{"line_number":410,"context_line":"       min-ready: 1"},{"line_number":411,"context_line":""},{"line_number":412,"context_line":"   - label:"},{"line_number":413,"context_line":"       name: ubuntu"},{"line_number":414,"context_line":""},{"line_number":415,"context_line":"   - provider:"},{"line_number":416,"context_line":"       name: rax-dfw"}],"source_content_type":"text/x-rst","patch_set":4,"id":"7d474a86_e9aa9c49","line":413,"updated":"2022-07-25 17:26:48.000000000","message":"I feel like I don\u0027t have enough information from this comment to fully understand what you want.  I think most of the confusion comes from your use of the word \"users\" and where you want to draw the line on what they know and don\u0027t know.\n\nSo far, this spec isn\u0027t designed to really change any of that.  Right now, in order to update the nodepool configuration, you need to know everything about the clouds nodepool is using.  A Zuul user doesn\u0027t need to know that -- their only interface with nodepool is the \"label\" concept.  The idea is that the Zuul/Nodepool system operator tells them what labels are available.  (If some or all of the Zuul users also happen to understand the clouds and the nodepool configuration, that\u0027s fine of course, but it\u0027s not necessary.)\n\nFundamentally, the idea is that a Zuul user doesn\u0027t have to know anything more about the test resource configuration than a label.  That\u0027s the unit of abstraction.\n\nWith this spec, nothing about that has to change -- all of these are new configuration objects, and they can be restricted to only be loaded from certain repos.  So users can still use abstract/opaque labels to specify what resources they want, and the people responsible for the current nodepool configuration can still be responsible for it by maintaining the repos with these new configuration objects in them.\n\nHaving said all that, of course we can change things, but I think in order to do that, we should start from a common point and work from there.  Given that the current system is designed to be mantained by people who know all the details of the clouds and provide labels to end users, what do you want to change starting from that point?\n\nMaybe we could define some specific use cases.\n\nRegarding the last point -- it is already the case that any Zuul configuration object can be included or excluded from any repo, and tenants can specify default sets for include/exclude (and in fact, you can make any arbitrary grouping of projects with a given include/exclude set).  That would certainly be true for every new configuration object as well, so anything included in this spec.  That is absolutely the basis for access control for all of this.","commit_id":"2ccc41a16d15c6330c83ff917c239fd5c0cdcbea"},{"author":{"_account_id":1,"name":"James E. Blair","email":"jim@acmegating.com","username":"corvus"},"change_message_id":"2708265501399520ca579790965d16d6b9232ee0","unresolved":true,"context_lines":[{"line_number":410,"context_line":"       min-ready: 1"},{"line_number":411,"context_line":""},{"line_number":412,"context_line":"   - label:"},{"line_number":413,"context_line":"       name: ubuntu"},{"line_number":414,"context_line":""},{"line_number":415,"context_line":"   - provider:"},{"line_number":416,"context_line":"       name: rax-dfw"}],"source_content_type":"text/x-rst","patch_set":4,"id":"c040a0c4_7d62f737","line":413,"in_reply_to":"9aaa8a9a_91dbc9d7","updated":"2022-07-25 17:30:21.000000000","message":"I feel like I don\u0027t have enough information from this comment to fully understand what you want.  I think most of the confusion comes from your use of the word \"users\" and where you want to draw the line on what they know and don\u0027t know.\n\nSo far, this spec isn\u0027t designed to really change any of that.  Right now, in order to update the nodepool configuration, you need to know everything about the clouds nodepool is using.  A Zuul user doesn\u0027t need to know that -- their only interface with nodepool is the \"label\" concept.  The idea is that the Zuul/Nodepool system operator tells them what labels are available.  (If some or all of the Zuul users also happen to understand the clouds and the nodepool configuration, that\u0027s fine of course, but it\u0027s not necessary.)\n\nFundamentally, the idea is that a Zuul user doesn\u0027t have to know anything more about the test resource configuration than a label.  That\u0027s the unit of abstraction.\n\nWith this spec, nothing about that has to change -- all of these are new configuration objects, and they can be restricted to only be loaded from certain repos.  So users can still use abstract/opaque labels to specify what resources they want, and the people responsible for the current nodepool configuration can still be responsible for it by maintaining the repos with these new configuration objects in them.\n\nHaving said all that, of course we can change things, but I think in order to do that, we should start from a common point and work from there.  Given that the current system is designed to be mantained by people who know all the details of the clouds and provide labels to end users, what do you want to change starting from that point?\n\nMaybe we could define some specific use cases.\n\nRegarding the last point -- it is already the case that any Zuul configuration object can be included or excluded from any repo, and tenants can specify default sets for include/exclude (and in fact, you can make any arbitrary grouping of projects with a given include/exclude set).  That would certainly be true for every new configuration object as well, so anything included in this spec.  That is absolutely the basis for access control for all of this.","commit_id":"2ccc41a16d15c6330c83ff917c239fd5c0cdcbea"},{"author":{"_account_id":1,"name":"James E. Blair","email":"jim@acmegating.com","username":"corvus"},"change_message_id":"9313012343bc4a68db49dcc68df1e991f0c885d2","unresolved":false,"context_lines":[{"line_number":410,"context_line":"       min-ready: 1"},{"line_number":411,"context_line":""},{"line_number":412,"context_line":"   - label:"},{"line_number":413,"context_line":"       name: ubuntu"},{"line_number":414,"context_line":""},{"line_number":415,"context_line":"   - provider:"},{"line_number":416,"context_line":"       name: rax-dfw"}],"source_content_type":"text/x-rst","patch_set":4,"id":"d3b09d1b_a15a58cf","line":413,"in_reply_to":"c040a0c4_7d62f737","updated":"2022-07-29 00:38:26.000000000","message":"I chatted with Tobias about this some, and I think this is the gist of it: we can take this opportunity to make the configuration more user accessible.  The main new use case is: a tenant administrator (ie, a power user who is not responsible for the entire system, only the content within their tenant) should be able to manage the lifecycle of a diskimage without help from the system administrator.  This spec was already anticipating that for maintaining the jobs that build images, but image lifecycle also includes adding and deleting images, so users will need to be able to attach them to providers (if the admin allows it).\n\nAnother point Tobias raised: being able to abstract flavors will make label management across multiple clouds much easier and will remove a lot of boilerplate configuration.\n\nI think the next version will address these points.","commit_id":"2ccc41a16d15c6330c83ff917c239fd5c0cdcbea"},{"author":{"_account_id":16068,"name":"Tobias Henkel","email":"tobias.henkel@bmw.de","username":"tobias.henkel"},"change_message_id":"26f49161a99d07c9c319adde6e9b2690c0d613b5","unresolved":true,"context_lines":[{"line_number":429,"context_line":"       # image-specific info (like config-drive) or username."},{"line_number":430,"context_line":"       zuul-images:"},{"line_number":431,"context_line":"         - name: centos-7"},{"line_number":432,"context_line":"           config-drive: true"},{"line_number":433,"context_line":"       cloud-images:"},{"line_number":434,"context_line":"         - name: ubuntu"},{"line_number":435,"context_line":"           image-name: ibm-ubuntu-20-04-3-minimal-amd64-1"}],"source_content_type":"text/x-rst","patch_set":4,"id":"b4d85d32_f50ec454","line":432,"updated":"2022-07-24 11:51:06.000000000","message":"It would also be great to have config-drive (we need to enable this on every onprem image) also as a global flag on the openstack provider or pool. But I think that\u0027s useful independent of this spec.","commit_id":"2ccc41a16d15c6330c83ff917c239fd5c0cdcbea"},{"author":{"_account_id":1,"name":"James E. Blair","email":"jim@acmegating.com","username":"corvus"},"change_message_id":"d3aadd8524718b895d054fa969dce85bff83ab9e","unresolved":false,"context_lines":[{"line_number":429,"context_line":"       # image-specific info (like config-drive) or username."},{"line_number":430,"context_line":"       zuul-images:"},{"line_number":431,"context_line":"         - name: centos-7"},{"line_number":432,"context_line":"           config-drive: true"},{"line_number":433,"context_line":"       cloud-images:"},{"line_number":434,"context_line":"         - name: ubuntu"},{"line_number":435,"context_line":"           image-name: ibm-ubuntu-20-04-3-minimal-amd64-1"}],"source_content_type":"text/x-rst","patch_set":4,"id":"39d3372a_ffe13c5f","line":432,"updated":"2022-07-25 17:26:48.000000000","message":"Yeah, I think we could add a default flag for that.","commit_id":"2ccc41a16d15c6330c83ff917c239fd5c0cdcbea"},{"author":{"_account_id":1,"name":"James E. Blair","email":"jim@acmegating.com","username":"corvus"},"change_message_id":"2708265501399520ca579790965d16d6b9232ee0","unresolved":false,"context_lines":[{"line_number":429,"context_line":"       # image-specific info (like config-drive) or username."},{"line_number":430,"context_line":"       zuul-images:"},{"line_number":431,"context_line":"         - name: centos-7"},{"line_number":432,"context_line":"           config-drive: true"},{"line_number":433,"context_line":"       cloud-images:"},{"line_number":434,"context_line":"         - name: ubuntu"},{"line_number":435,"context_line":"           image-name: ibm-ubuntu-20-04-3-minimal-amd64-1"}],"source_content_type":"text/x-rst","patch_set":4,"id":"60cc6fc6_fd13cf08","line":432,"in_reply_to":"b4d85d32_f50ec454","updated":"2022-07-25 17:30:21.000000000","message":"Yeah, I think we could add a default flag for that.","commit_id":"2ccc41a16d15c6330c83ff917c239fd5c0cdcbea"},{"author":{"_account_id":16068,"name":"Tobias Henkel","email":"tobias.henkel@bmw.de","username":"tobias.henkel"},"change_message_id":"26f49161a99d07c9c319adde6e9b2690c0d613b5","unresolved":true,"context_lines":[{"line_number":530,"context_line":""},{"line_number":531,"context_line":"To make the transition as minimally disruptive as possible, we will"},{"line_number":532,"context_line":"support both systems in Zuul, and allow for selection of one system or"},{"line_number":533,"context_line":"the other on a tenant-by-tenant basis.  A tenant configuration flag"},{"line_number":534,"context_line":"will be added to instruct the Zuul scheduler to place node requests in"},{"line_number":535,"context_line":"either the ``/nodepool`` or ``/zuul/node-requests`` path in ZooKeeper."},{"line_number":536,"context_line":""}],"source_content_type":"text/x-rst","patch_set":4,"id":"5d6835bf_92ce5df6","line":533,"updated":"2022-07-24 11:51:06.000000000","message":"We have tenants with ~100 images that are very hard to switch as a big bang. I think we\u0027d need a more fine grained way of switching on a label-by-label basis.\n\nMaybe we could do that by placing node-requests for known labels in /zuul/node-requests and unknown labels in /nodepool until the tenant is marked fully migrated which would then also enable e.g. label validation in job config.","commit_id":"2ccc41a16d15c6330c83ff917c239fd5c0cdcbea"},{"author":{"_account_id":1,"name":"James E. Blair","email":"jim@acmegating.com","username":"corvus"},"change_message_id":"d3aadd8524718b895d054fa969dce85bff83ab9e","unresolved":false,"context_lines":[{"line_number":530,"context_line":""},{"line_number":531,"context_line":"To make the transition as minimally disruptive as possible, we will"},{"line_number":532,"context_line":"support both systems in Zuul, and allow for selection of one system or"},{"line_number":533,"context_line":"the other on a tenant-by-tenant basis.  A tenant configuration flag"},{"line_number":534,"context_line":"will be added to instruct the Zuul scheduler to place node requests in"},{"line_number":535,"context_line":"either the ``/nodepool`` or ``/zuul/node-requests`` path in ZooKeeper."},{"line_number":536,"context_line":""}],"source_content_type":"text/x-rst","patch_set":4,"id":"f5241cf8_a1019336","line":533,"updated":"2022-07-25 17:26:48.000000000","message":"Why does the number of images affect whether a tenant switches or not?  You can have all the images ready to go before switching the tenant over (because the image builds can be done before handling any node requests).  And in general, you can switch a smaller tenant over first to verify that node requests are working correctly.\n\nThe label validation is needed for more than just the job config, it\u0027s also needed for provider config, etc.  Even today, it\u0027s a Nodepool config error for a provider to have a label entry for a label that doesn\u0027t exist.  So if we made the presence of a label in a tenant be an indication of whether we use old or new requests, we would need to extend that to the pool configuration as well.  So basically, to turn on a label in a tenant, you would have to add both the \"label\" object and also add that label to a pool.\n\nGiven that it\u0027s anticipated that many label and pool definitions would be kept in a central repo that is added to multiple tenants, this behavior (even just the \"label\" behavior alone) really means that under your proposal, switching to the new system would be label-by-label but for all tenants at once, unless you combined the two, so that there was a tri-state tenant flag:\n\n1) Old requests only\n2) New requests for known labels, otherwise old requests\n3) New requests only (validation enabled)\n\nThen you could start with all tenants at #1, then switch a small tenant to #2 and add some labels, then switch a large tenant to #2 and add more labels until they are all using new requests, then switch all tenants to #2 then #3.\n\nIs that what you had in mind?","commit_id":"2ccc41a16d15c6330c83ff917c239fd5c0cdcbea"},{"author":{"_account_id":1,"name":"James E. Blair","email":"jim@acmegating.com","username":"corvus"},"change_message_id":"2708265501399520ca579790965d16d6b9232ee0","unresolved":true,"context_lines":[{"line_number":530,"context_line":""},{"line_number":531,"context_line":"To make the transition as minimally disruptive as possible, we will"},{"line_number":532,"context_line":"support both systems in Zuul, and allow for selection of one system or"},{"line_number":533,"context_line":"the other on a tenant-by-tenant basis.  A tenant configuration flag"},{"line_number":534,"context_line":"will be added to instruct the Zuul scheduler to place node requests in"},{"line_number":535,"context_line":"either the ``/nodepool`` or ``/zuul/node-requests`` path in ZooKeeper."},{"line_number":536,"context_line":""}],"source_content_type":"text/x-rst","patch_set":4,"id":"9ac0a638_c4b3eb00","line":533,"in_reply_to":"5d6835bf_92ce5df6","updated":"2022-07-25 17:30:21.000000000","message":"Why does the number of images affect whether a tenant switches or not?  You can have all the images ready to go before switching the tenant over (because the image builds can be done before handling any node requests).  And in general, you can switch a smaller tenant over first to verify that node requests are working correctly.\n\nThe label validation is needed for more than just the job config, it\u0027s also needed for provider config, etc.  Even today, it\u0027s a Nodepool config error for a provider to have a label entry for a label that doesn\u0027t exist.  So if we made the presence of a label in a tenant be an indication of whether we use old or new requests, we would need to extend that to the pool configuration as well.  So basically, to turn on a label in a tenant, you would have to add both the \"label\" object and also add that label to a pool.\n\nGiven that it\u0027s anticipated that many label and pool definitions would be kept in a central repo that is added to multiple tenants, this behavior (even just the \"label\" behavior alone) really means that under your proposal, switching to the new system would be label-by-label but for all tenants at once, unless you combined the two, so that there was a tri-state tenant flag:\n\n1) Old requests only\n2) New requests for known labels, otherwise old requests\n3) New requests only (validation enabled)\n\nThen you could start with all tenants at #1, then switch a small tenant to #2 and add some labels, then switch a large tenant to #2 and add more labels until they are all using new requests, then switch all tenants to #2 then #3.\n\nIs that what you had in mind?","commit_id":"2ccc41a16d15c6330c83ff917c239fd5c0cdcbea"},{"author":{"_account_id":1,"name":"James E. Blair","email":"jim@acmegating.com","username":"corvus"},"change_message_id":"9313012343bc4a68db49dcc68df1e991f0c885d2","unresolved":false,"context_lines":[{"line_number":530,"context_line":""},{"line_number":531,"context_line":"To make the transition as minimally disruptive as possible, we will"},{"line_number":532,"context_line":"support both systems in Zuul, and allow for selection of one system or"},{"line_number":533,"context_line":"the other on a tenant-by-tenant basis.  A tenant configuration flag"},{"line_number":534,"context_line":"will be added to instruct the Zuul scheduler to place node requests in"},{"line_number":535,"context_line":"either the ``/nodepool`` or ``/zuul/node-requests`` path in ZooKeeper."},{"line_number":536,"context_line":""}],"source_content_type":"text/x-rst","patch_set":4,"id":"6f5bd305_7dc9b879","line":533,"in_reply_to":"9ac0a638_c4b3eb00","updated":"2022-07-29 00:38:26.000000000","message":"I\u0027m including a label-by-label + tenant flag process in the next revision.","commit_id":"2ccc41a16d15c6330c83ff917c239fd5c0cdcbea"},{"author":{"_account_id":27582,"name":"Simon Westphahl","email":"simon.westphahl@bmw.de","username":"simon.westphahl"},"change_message_id":"a675b75f5a869d3dcafdd5bc08142cf26c86899e","unresolved":true,"context_lines":[{"line_number":583,"context_line":"       name: rax-dfw-devstack"},{"line_number":584,"context_line":"       section: rax-dfw"},{"line_number":585,"context_line":"       # The images can be attached to the pool just like the provider."},{"line_number":586,"context_line":"       images:"},{"line_number":587,"context_line":"         - name: devstack"},{"line_number":588,"context_line":"           config-drive: true"},{"line_number":589,"context_line":"       labels:"}],"source_content_type":"text/x-rst","patch_set":5,"id":"8f7f26f9_945b5ee0","line":586,"updated":"2022-08-18 14:17:48.000000000","message":"Do we have to list all images here or only when we want to override some image config on the provider level?\n\nI\u0027m asking since the label already specifies the associated image. If we just need this for overriding image attributes we should maybe call this \"image-overrides\" instead.","commit_id":"803eb70e419964485e0f043f5b1b5a85968a9ef9"},{"author":{"_account_id":1,"name":"James E. Blair","email":"jim@acmegating.com","username":"corvus"},"change_message_id":"4c68cfbbdd6069e5d9ba4ec4154a30128b45f220","unresolved":false,"context_lines":[{"line_number":583,"context_line":"       name: rax-dfw-devstack"},{"line_number":584,"context_line":"       section: rax-dfw"},{"line_number":585,"context_line":"       # The images can be attached to the pool just like the provider."},{"line_number":586,"context_line":"       images:"},{"line_number":587,"context_line":"         - name: devstack"},{"line_number":588,"context_line":"           config-drive: true"},{"line_number":589,"context_line":"       labels:"}],"source_content_type":"text/x-rst","patch_set":5,"id":"3ec55d89_4a5b6865","line":586,"in_reply_to":"8f7f26f9_945b5ee0","updated":"2022-08-18 20:47:36.000000000","message":"This is the same as the \"images\" attribute on the \"section\" object, that way a tenant can (if permitted) configure cloud/section-specific attributes of tenant-specific images.  In other words, a section might have image \"A\" and a provider might add image \"B\" to that, so that ultimately both A and B are available.\n\nI don\u0027t think I specified what should happen if we have \"A\" on both.  That should probably be an error?  Because we probably don\u0027t want a provider overriding something set on a section?\n\nWith that in mind, I think I\u0027d prefer to keep it as \"images\", but if we want to change it, I think it should be in both places.","commit_id":"803eb70e419964485e0f043f5b1b5a85968a9ef9"},{"author":{"_account_id":5263,"name":"Jeremy Stanley","display_name":"fungi","email":"fungi@yuggoth.org","username":"fungi","status":"missing, presumed fed"},"change_message_id":"b1d33e587fc8d3b0f6c1cd23d4ab7e29143612d9","unresolved":false,"context_lines":[{"line_number":85,"context_line":"can build an image for Zuul without relying on any particular cloud"},{"line_number":86,"context_line":"provider images.  A Zuul system whose operator wants to use custom"},{"line_number":87,"context_line":"images will need to bootstrap that process, and under the proposed"},{"line_number":88,"context_line":"system where images are build in Zuul jobs, that would need to be done"},{"line_number":89,"context_line":"using a stock cloud image.  In other words, to bootstrap a system such"},{"line_number":90,"context_line":"as OpenDev from scratch, the operators would need to use a stock cloud"},{"line_number":91,"context_line":"image to run the job to build the custom image.  Once a custom image"}],"source_content_type":"text/x-rst","patch_set":8,"id":"4fe7b194_5f4583eb","line":88,"updated":"2022-09-08 15:23:17.000000000","message":"Nit: are built","commit_id":"781a8470d3790e6257fa5e813e9d8750c41b72bc"},{"author":{"_account_id":4146,"name":"Clark Boylan","email":"cboylan@sapwetik.org","username":"cboylan"},"change_message_id":"7be0b064a7c79187fa42b085a39538393c044538","unresolved":true,"context_lines":[{"line_number":107,"context_line":"interval (typically daily).  To map this into Zuul, we will use Zuul\u0027s"},{"line_number":108,"context_line":"existing pipeline functionality, but we will add a new trigger for"},{"line_number":109,"context_line":"case #1.  Case #2 can be handled by a manual Zuul enqueue command, and"},{"line_number":110,"context_line":"case #3 by a periodic pipeline trigger."},{"line_number":111,"context_line":""},{"line_number":112,"context_line":"Since Zuul knows what images are configured and what their current"},{"line_number":113,"context_line":"states are, it will be able to emit trigger events when it detects"}],"source_content_type":"text/x-rst","patch_set":8,"id":"e8b52490_17c7bdbf","line":110,"updated":"2022-09-01 19:55:00.000000000","message":"For case #2 I don\u0027t think it is currently possible to enqueue a build for a single job. Instead we have to enqueue a trigger event which will build all matching jobs. Will we update the enqueue tooling to enable us to enqueue a single build job as described for the image build trigger below? I think this will be important for manually triggered builds as we often don\u0027t want to build the world due to contention of resources when a single image needs updating.","commit_id":"781a8470d3790e6257fa5e813e9d8750c41b72bc"},{"author":{"_account_id":4146,"name":"Clark Boylan","email":"cboylan@sapwetik.org","username":"cboylan"},"change_message_id":"ade2bd9a806816fc01a19b60b541a3ddcef87075","unresolved":false,"context_lines":[{"line_number":107,"context_line":"interval (typically daily).  To map this into Zuul, we will use Zuul\u0027s"},{"line_number":108,"context_line":"existing pipeline functionality, but we will add a new trigger for"},{"line_number":109,"context_line":"case #1.  Case #2 can be handled by a manual Zuul enqueue command, and"},{"line_number":110,"context_line":"case #3 by a periodic pipeline trigger."},{"line_number":111,"context_line":""},{"line_number":112,"context_line":"Since Zuul knows what images are configured and what their current"},{"line_number":113,"context_line":"states are, it will be able to emit trigger events when it detects"}],"source_content_type":"text/x-rst","patch_set":8,"id":"e71b08b9_004c167c","line":110,"in_reply_to":"6074d05b_29bc34ca","updated":"2022-09-07 16:55:18.000000000","message":"I think this comment is sufficient. I just wanted to make sure it was on your radar before we committed to something that may have been clunky from a manual management perspective.","commit_id":"781a8470d3790e6257fa5e813e9d8750c41b72bc"},{"author":{"_account_id":1,"name":"James E. Blair","email":"jim@acmegating.com","username":"corvus"},"change_message_id":"d4b62e133e4e1cc6a8791d587069d005bd40f138","unresolved":true,"context_lines":[{"line_number":107,"context_line":"interval (typically daily).  To map this into Zuul, we will use Zuul\u0027s"},{"line_number":108,"context_line":"existing pipeline functionality, but we will add a new trigger for"},{"line_number":109,"context_line":"case #1.  Case #2 can be handled by a manual Zuul enqueue command, and"},{"line_number":110,"context_line":"case #3 by a periodic pipeline trigger."},{"line_number":111,"context_line":""},{"line_number":112,"context_line":"Since Zuul knows what images are configured and what their current"},{"line_number":113,"context_line":"states are, it will be able to emit trigger events when it detects"}],"source_content_type":"text/x-rst","patch_set":8,"id":"6074d05b_29bc34ca","line":110,"in_reply_to":"e8b52490_17c7bdbf","updated":"2022-09-01 21:13:17.000000000","message":"I don\u0027t think we\u0027d specify a particular job, but we would specify a particular image or set of images.  See the paragraph from lines 121-127 for a description of the new image attribute for jobs that we would expect the pipeline to match.  I think what\u0027s missing is a description of the fact that the triggering image attribute described in lines 112-119 would carry over to the enqueued item and be used to match the jobs.  With a new attribute on the queue item, that\u0027s conceptually compatible with direct enqueueing an item.  Maybe this comment is enough to clarify that, or I can add a few more sentences to that effect.","commit_id":"781a8470d3790e6257fa5e813e9d8750c41b72bc"},{"author":{"_account_id":4146,"name":"Clark Boylan","email":"cboylan@sapwetik.org","username":"cboylan"},"change_message_id":"7be0b064a7c79187fa42b085a39538393c044538","unresolved":true,"context_lines":[{"line_number":128,"context_line":""},{"line_number":129,"context_line":"The job will be responsible for building the image and uploading the"},{"line_number":130,"context_line":"result to some storage system.  The URLs for each image format built"},{"line_number":131,"context_line":"should be returned to Zuul as artifacts."},{"line_number":132,"context_line":""},{"line_number":133,"context_line":"Finally, the `zuul` driver reporter will accept parameters which will"},{"line_number":134,"context_line":"tell it to search the result data for these artifact URLs and update"}],"source_content_type":"text/x-rst","patch_set":8,"id":"29bdf149_f1f70924","line":131,"updated":"2022-09-01 19:55:00.000000000","message":"One advantage of the current system compared to this proposed system is that we can keep a copy of our images long term in local build storage. At times this is also a disadvantage when we can\u0027t delete images and they build up and fill the builder disk.\n\nWith artifact uploads in opendev I believe we have a 30 day expiry. We may need to think about variable expiries for different object types like images.\n\nI don\u0027t think this is really a problem. More of a point to consider as this rolls out.","commit_id":"781a8470d3790e6257fa5e813e9d8750c41b72bc"},{"author":{"_account_id":1,"name":"James E. Blair","email":"jim@acmegating.com","username":"corvus"},"change_message_id":"d4b62e133e4e1cc6a8791d587069d005bd40f138","unresolved":true,"context_lines":[{"line_number":128,"context_line":""},{"line_number":129,"context_line":"The job will be responsible for building the image and uploading the"},{"line_number":130,"context_line":"result to some storage system.  The URLs for each image format built"},{"line_number":131,"context_line":"should be returned to Zuul as artifacts."},{"line_number":132,"context_line":""},{"line_number":133,"context_line":"Finally, the `zuul` driver reporter will accept parameters which will"},{"line_number":134,"context_line":"tell it to search the result data for these artifact URLs and update"}],"source_content_type":"text/x-rst","patch_set":8,"id":"acdcd474_a1eeafae","line":131,"in_reply_to":"29bdf149_f1f70924","updated":"2022-09-01 21:13:17.000000000","message":"Yeah, I think this actually adds flexibility.  To continue the opendev example, we don\u0027t have to use our existing log storage locations, we can set up new locations with different retention periods (and we may prefer to use a subset of the places we use for log storage).","commit_id":"781a8470d3790e6257fa5e813e9d8750c41b72bc"},{"author":{"_account_id":4146,"name":"Clark Boylan","email":"cboylan@sapwetik.org","username":"cboylan"},"change_message_id":"ade2bd9a806816fc01a19b60b541a3ddcef87075","unresolved":false,"context_lines":[{"line_number":128,"context_line":""},{"line_number":129,"context_line":"The job will be responsible for building the image and uploading the"},{"line_number":130,"context_line":"result to some storage system.  The URLs for each image format built"},{"line_number":131,"context_line":"should be returned to Zuul as artifacts."},{"line_number":132,"context_line":""},{"line_number":133,"context_line":"Finally, the `zuul` driver reporter will accept parameters which will"},{"line_number":134,"context_line":"tell it to search the result data for these artifact URLs and update"}],"source_content_type":"text/x-rst","patch_set":8,"id":"9e2d63ea_918c0e0e","line":131,"in_reply_to":"acdcd474_a1eeafae","updated":"2022-09-07 16:55:18.000000000","message":"Ack","commit_id":"781a8470d3790e6257fa5e813e9d8750c41b72bc"},{"author":{"_account_id":5263,"name":"Jeremy Stanley","display_name":"fungi","email":"fungi@yuggoth.org","username":"fungi","status":"missing, presumed fed"},"change_message_id":"b1d33e587fc8d3b0f6c1cd23d4ab7e29143612d9","unresolved":false,"context_lines":[{"line_number":153,"context_line":""},{"line_number":154,"context_line":"   - job:"},{"line_number":155,"context_line":"       name: build-debian-unstable-image"},{"line_number":156,"context_line":"       image-build-name: debian-unstable"},{"line_number":157,"context_line":""},{"line_number":158,"context_line":"This job would run whenever Zuul determines it needs a new"},{"line_number":159,"context_line":"debian-unstable image or daily at midnight.  Once the job completes,"}],"source_content_type":"text/x-rst","patch_set":8,"id":"dee595cd_bf05f988","line":156,"updated":"2022-09-08 15:23:17.000000000","message":"Would such job definitions be limited to trusted config repos, or otherwise controlled via ACL in order to prevent arbitrary users of the system from adding images? And would the system enforce uniqueness for the image-build-name parameter (at least for non-abstract jobs)? This may be related to the tenant discussion farther down, though I\u0027m also concerned with the potential for limiting who has permission to do this within a given tenant.","commit_id":"781a8470d3790e6257fa5e813e9d8750c41b72bc"},{"author":{"_account_id":1,"name":"James E. Blair","email":"jim@acmegating.com","username":"corvus"},"change_message_id":"f7aeb3a212a2e681bdf73eec88844097f520c728","unresolved":false,"context_lines":[{"line_number":153,"context_line":""},{"line_number":154,"context_line":"   - job:"},{"line_number":155,"context_line":"       name: build-debian-unstable-image"},{"line_number":156,"context_line":"       image-build-name: debian-unstable"},{"line_number":157,"context_line":""},{"line_number":158,"context_line":"This job would run whenever Zuul determines it needs a new"},{"line_number":159,"context_line":"debian-unstable image or daily at midnight.  Once the job completes,"}],"source_content_type":"text/x-rst","patch_set":8,"id":"af774dd9_c8d383bd","line":156,"in_reply_to":"dee595cd_bf05f988","updated":"2022-09-08 18:02:26.000000000","message":"The jobs could be defined anywhere jobs are normally defined -- but they won\u0027t do anything special like upload images unless they are attached to an image building pipeline (which has certain triggers and reporters which are able to be restricted -- see my conversation with Clark).  So the way you say \"this tenant can or can not add images\" is to supply or allow that kind of pipeline in the tenant.\n\nThe image name is... let\u0027s call it... \"implicitly unique\".  Because behind the scenes, Zuul will map that image name to an \"image\" stanza and the image stanza will always internally use a fully qualified name including the repo where it is defined.  So \"debian-unstable\" in opendev really means something like \"(opendev.org/opendev/images, debian-unstable)\".  That\u0027s described in lines 375-388).  So a single tenant can only have one \"debian-unstable\" image and Zuul will enforce that uniqueness in the usual way.  But \"debian-unstable\" in two different tenants might mean the same image, or they might not, depending on whether the debian-unstable image stanza is literally in the same repo or in a different one.\n\nFor opendev, we will almost certainly only have one definition and share it among all the tenants.  That mimics our current behavior.","commit_id":"781a8470d3790e6257fa5e813e9d8750c41b72bc"},{"author":{"_account_id":4146,"name":"Clark Boylan","email":"cboylan@sapwetik.org","username":"cboylan"},"change_message_id":"7be0b064a7c79187fa42b085a39538393c044538","unresolved":true,"context_lines":[{"line_number":156,"context_line":"       image-build-name: debian-unstable"},{"line_number":157,"context_line":""},{"line_number":158,"context_line":"This job would run whenever Zuul determines it needs a new"},{"line_number":159,"context_line":"debian-unstable image or daily at midnight.  Once the job completes,"},{"line_number":160,"context_line":"because of the ``image-built: true`` report, it will look for artifact"},{"line_number":161,"context_line":"data like this:"},{"line_number":162,"context_line":""}],"source_content_type":"text/x-rst","patch_set":8,"id":"d9c4e8ee_86df6364","line":159,"updated":"2022-09-01 19:55:00.000000000","message":"Currently image builds are well spread out in OpenDev which is nice because it avoids overwhelming the storage systems in clouds when the image build completes and we shift to uploading the images. Shifting to triggering all the image builds at midnight (or relatively close using jitter) may result in undesireable thundering herds.","commit_id":"781a8470d3790e6257fa5e813e9d8750c41b72bc"},{"author":{"_account_id":4146,"name":"Clark Boylan","email":"cboylan@sapwetik.org","username":"cboylan"},"change_message_id":"ade2bd9a806816fc01a19b60b541a3ddcef87075","unresolved":false,"context_lines":[{"line_number":156,"context_line":"       image-build-name: debian-unstable"},{"line_number":157,"context_line":""},{"line_number":158,"context_line":"This job would run whenever Zuul determines it needs a new"},{"line_number":159,"context_line":"debian-unstable image or daily at midnight.  Once the job completes,"},{"line_number":160,"context_line":"because of the ``image-built: true`` report, it will look for artifact"},{"line_number":161,"context_line":"data like this:"},{"line_number":162,"context_line":""}],"source_content_type":"text/x-rst","patch_set":8,"id":"c7a82f54_490cb96f","line":159,"in_reply_to":"6105e627_4f2400fb","updated":"2022-09-07 16:55:18.000000000","message":"Good point, and definitely something that OpenDev should plan to do to avoid this problem.","commit_id":"781a8470d3790e6257fa5e813e9d8750c41b72bc"},{"author":{"_account_id":1,"name":"James E. Blair","email":"jim@acmegating.com","username":"corvus"},"change_message_id":"d4b62e133e4e1cc6a8791d587069d005bd40f138","unresolved":true,"context_lines":[{"line_number":156,"context_line":"       image-build-name: debian-unstable"},{"line_number":157,"context_line":""},{"line_number":158,"context_line":"This job would run whenever Zuul determines it needs a new"},{"line_number":159,"context_line":"debian-unstable image or daily at midnight.  Once the job completes,"},{"line_number":160,"context_line":"because of the ``image-built: true`` report, it will look for artifact"},{"line_number":161,"context_line":"data like this:"},{"line_number":162,"context_line":""}],"source_content_type":"text/x-rst","patch_set":8,"id":"6105e627_4f2400fb","line":159,"in_reply_to":"d9c4e8ee_86df6364","updated":"2022-09-01 21:13:17.000000000","message":"We can use a Zuul semaphore to limit the number of concurrent builds.","commit_id":"781a8470d3790e6257fa5e813e9d8750c41b72bc"},{"author":{"_account_id":5263,"name":"Jeremy Stanley","display_name":"fungi","email":"fungi@yuggoth.org","username":"fungi","status":"missing, presumed fed"},"change_message_id":"b1d33e587fc8d3b0f6c1cd23d4ab7e29143612d9","unresolved":false,"context_lines":[{"line_number":178,"context_line":""},{"line_number":179,"context_line":"Zuul will update internal records in ZooKeeper for the image to record"},{"line_number":180,"context_line":"the storage URLs.  The zuul-launcher process will then start"},{"line_number":181,"context_line":"background processes to download the images from the storage system"},{"line_number":182,"context_line":"and upload them to the configured providers (much as nodepool-builder"},{"line_number":183,"context_line":"does now with files on disk).  As a special case, it may detect that"},{"line_number":184,"context_line":"the image files are stored in a location that a provider can access"}],"source_content_type":"text/x-rst","patch_set":8,"id":"cfa4f3bc_bb542941","line":181,"updated":"2022-09-08 15:23:17.000000000","message":"While I don\u0027t think it\u0027s a reason to avoid this design, it\u0027s worth considering that these are potentially very large files and we would now be copying them back and forth over the network multiple times (as opposed to the current model where they\u0027re uploaded directly from local storage). Not only does this imply a potentially significant increase in bandwidth usage, but also provides additional opportunities for network and service issues adversely impacting those long-running file transfers, degrading the stability of the image upload.","commit_id":"781a8470d3790e6257fa5e813e9d8750c41b72bc"},{"author":{"_account_id":1,"name":"James E. Blair","email":"jim@acmegating.com","username":"corvus"},"change_message_id":"f7aeb3a212a2e681bdf73eec88844097f520c728","unresolved":false,"context_lines":[{"line_number":178,"context_line":""},{"line_number":179,"context_line":"Zuul will update internal records in ZooKeeper for the image to record"},{"line_number":180,"context_line":"the storage URLs.  The zuul-launcher process will then start"},{"line_number":181,"context_line":"background processes to download the images from the storage system"},{"line_number":182,"context_line":"and upload them to the configured providers (much as nodepool-builder"},{"line_number":183,"context_line":"does now with files on disk).  As a special case, it may detect that"},{"line_number":184,"context_line":"the image files are stored in a location that a provider can access"}],"source_content_type":"text/x-rst","patch_set":8,"id":"49e8a365_4e57c69b","line":181,"in_reply_to":"cfa4f3bc_bb542941","updated":"2022-09-08 18:02:26.000000000","message":"Yes, compared to the current system, this may add 2 transfers for an image, if a given zuul-launcher has access to all providers.  If it is scoped to fewer providers, more transfers may be needed.","commit_id":"781a8470d3790e6257fa5e813e9d8750c41b72bc"},{"author":{"_account_id":5263,"name":"Jeremy Stanley","display_name":"fungi","email":"fungi@yuggoth.org","username":"fungi","status":"missing, presumed fed"},"change_message_id":"b1d33e587fc8d3b0f6c1cd23d4ab7e29143612d9","unresolved":false,"context_lines":[{"line_number":183,"context_line":"does now with files on disk).  As a special case, it may detect that"},{"line_number":184,"context_line":"the image files are stored in a location that a provider can access"},{"line_number":185,"context_line":"directly for import and may be able to import directly from the"},{"line_number":186,"context_line":"storage location rather than downloading locally first."},{"line_number":187,"context_line":""},{"line_number":188,"context_line":"To handle image validation, a flag will be stored for each image"},{"line_number":189,"context_line":"upload indicating whether it has been validated.  The example above"}],"source_content_type":"text/x-rst","patch_set":8,"id":"ab3f0811_b9283d4d","line":186,"updated":"2022-09-08 15:23:17.000000000","message":"This is a useful feature though, which may mitigate some of my points above.","commit_id":"781a8470d3790e6257fa5e813e9d8750c41b72bc"},{"author":{"_account_id":4146,"name":"Clark Boylan","email":"cboylan@sapwetik.org","username":"cboylan"},"change_message_id":"7be0b064a7c79187fa42b085a39538393c044538","unresolved":true,"context_lines":[{"line_number":245,"context_line":"* Lock each upload it can handle"},{"line_number":246,"context_line":"* Unlocks the image while retaining the upload locks"},{"line_number":247,"context_line":"* Downloads artifact (if needed) and uploads images to provider"},{"line_number":248,"context_line":"* If upload requires validation, it enqueues an `image-validate` zuul driver trigger event"},{"line_number":249,"context_line":"* Unlocks upload"},{"line_number":250,"context_line":""},{"line_number":251,"context_line":"The locking sequence is so that a single launcher can perform multiple"}],"source_content_type":"text/x-rst","patch_set":8,"id":"4b0f6595_d1ab4870","line":248,"updated":"2022-09-01 19:55:00.000000000","message":"This trigger event will identify the specific build of an image right? Otherwise we might not be verifying the correct build of an image.\n\nAdditionally this may be necessary to manually trigger a reupload of an image. For example if an upload was corrupted in the target cloud we might need to trigger a delete for a specific cloud image and then reupload it.","commit_id":"781a8470d3790e6257fa5e813e9d8750c41b72bc"},{"author":{"_account_id":4146,"name":"Clark Boylan","email":"cboylan@sapwetik.org","username":"cboylan"},"change_message_id":"ade2bd9a806816fc01a19b60b541a3ddcef87075","unresolved":false,"context_lines":[{"line_number":245,"context_line":"* Lock each upload it can handle"},{"line_number":246,"context_line":"* Unlocks the image while retaining the upload locks"},{"line_number":247,"context_line":"* Downloads artifact (if needed) and uploads images to provider"},{"line_number":248,"context_line":"* If upload requires validation, it enqueues an `image-validate` zuul driver trigger event"},{"line_number":249,"context_line":"* Unlocks upload"},{"line_number":250,"context_line":""},{"line_number":251,"context_line":"The locking sequence is so that a single launcher can perform multiple"}],"source_content_type":"text/x-rst","patch_set":8,"id":"2871534f_819de160","line":248,"in_reply_to":"48f32f0a_9a73e567","updated":"2022-09-07 16:55:18.000000000","message":"Ack","commit_id":"781a8470d3790e6257fa5e813e9d8750c41b72bc"},{"author":{"_account_id":1,"name":"James E. Blair","email":"jim@acmegating.com","username":"corvus"},"change_message_id":"d4b62e133e4e1cc6a8791d587069d005bd40f138","unresolved":true,"context_lines":[{"line_number":245,"context_line":"* Lock each upload it can handle"},{"line_number":246,"context_line":"* Unlocks the image while retaining the upload locks"},{"line_number":247,"context_line":"* Downloads artifact (if needed) and uploads images to provider"},{"line_number":248,"context_line":"* If upload requires validation, it enqueues an `image-validate` zuul driver trigger event"},{"line_number":249,"context_line":"* Unlocks upload"},{"line_number":250,"context_line":""},{"line_number":251,"context_line":"The locking sequence is so that a single launcher can perform multiple"}],"source_content_type":"text/x-rst","patch_set":8,"id":"48f32f0a_9a73e567","line":248,"in_reply_to":"4b0f6595_d1ab4870","updated":"2022-09-01 21:13:17.000000000","message":"Yes, we\u0027ll include that info on the trigger event and queue item as well so that the node request will use exactly that build (which is otherwise not yet in service).\n\nI agree we should handle the delete use-case.  I think we can handle it with manual commands similar to how we do now -- if we have a command to delete the upload from the provider and remove the image id from the path described at line 237 then this cycle can restart.","commit_id":"781a8470d3790e6257fa5e813e9d8750c41b72bc"},{"author":{"_account_id":4146,"name":"Clark Boylan","email":"cboylan@sapwetik.org","username":"cboylan"},"change_message_id":"7be0b064a7c79187fa42b085a39538393c044538","unresolved":true,"context_lines":[{"line_number":256,"context_line":"Zuul-launcher will delete the uploads from the providers.  The `zuul`"},{"line_number":257,"context_line":"driver emits an `image-delete` event with item data for the image"},{"line_number":258,"context_line":"artifact.  This will trigger an image-delete job that can delete the"},{"line_number":259,"context_line":"artifact from the cloud storage."},{"line_number":260,"context_line":""},{"line_number":261,"context_line":"All of these pipeline definitions should typically be in a single"},{"line_number":262,"context_line":"tenant (but need not be), but the images they build are potentially"}],"source_content_type":"text/x-rst","patch_set":8,"id":"1ec900ed_be08da0d","line":259,"updated":"2022-09-01 19:55:00.000000000","message":"Does this imply the local launcher copy of the image is ephemeral and may be deleted aggressively? I think that may be a good idea since the images can grow to be quite large.","commit_id":"781a8470d3790e6257fa5e813e9d8750c41b72bc"},{"author":{"_account_id":1,"name":"James E. Blair","email":"jim@acmegating.com","username":"corvus"},"change_message_id":"d4b62e133e4e1cc6a8791d587069d005bd40f138","unresolved":true,"context_lines":[{"line_number":256,"context_line":"Zuul-launcher will delete the uploads from the providers.  The `zuul`"},{"line_number":257,"context_line":"driver emits an `image-delete` event with item data for the image"},{"line_number":258,"context_line":"artifact.  This will trigger an image-delete job that can delete the"},{"line_number":259,"context_line":"artifact from the cloud storage."},{"line_number":260,"context_line":""},{"line_number":261,"context_line":"All of these pipeline definitions should typically be in a single"},{"line_number":262,"context_line":"tenant (but need not be), but the images they build are potentially"}],"source_content_type":"text/x-rst","patch_set":8,"id":"6d7c8491_eedd2f8a","line":259,"in_reply_to":"1ec900ed_be08da0d","updated":"2022-09-01 21:13:17.000000000","message":"Yes, I envision that it would delete its local copy as soon as it has performed all the uploads that it can.","commit_id":"781a8470d3790e6257fa5e813e9d8750c41b72bc"},{"author":{"_account_id":4146,"name":"Clark Boylan","email":"cboylan@sapwetik.org","username":"cboylan"},"change_message_id":"ade2bd9a806816fc01a19b60b541a3ddcef87075","unresolved":false,"context_lines":[{"line_number":256,"context_line":"Zuul-launcher will delete the uploads from the providers.  The `zuul`"},{"line_number":257,"context_line":"driver emits an `image-delete` event with item data for the image"},{"line_number":258,"context_line":"artifact.  This will trigger an image-delete job that can delete the"},{"line_number":259,"context_line":"artifact from the cloud storage."},{"line_number":260,"context_line":""},{"line_number":261,"context_line":"All of these pipeline definitions should typically be in a single"},{"line_number":262,"context_line":"tenant (but need not be), but the images they build are potentially"}],"source_content_type":"text/x-rst","patch_set":8,"id":"d314a8ae_3f9a727b","line":259,"in_reply_to":"6d7c8491_eedd2f8a","updated":"2022-09-07 16:55:18.000000000","message":"Ack","commit_id":"781a8470d3790e6257fa5e813e9d8750c41b72bc"},{"author":{"_account_id":4146,"name":"Clark Boylan","email":"cboylan@sapwetik.org","username":"cboylan"},"change_message_id":"7be0b064a7c79187fa42b085a39538393c044538","unresolved":true,"context_lines":[{"line_number":263,"context_line":"available to each tenant that includes the image definition"},{"line_number":264,"context_line":"configuration object (see the Configuration section below).  Any repo"},{"line_number":265,"context_line":"in a tenant with an image build pipeline will be able to cause images"},{"line_number":266,"context_line":"to be built and uploaded to providers."},{"line_number":267,"context_line":""},{"line_number":268,"context_line":"Snapshot Images"},{"line_number":269,"context_line":"~~~~~~~~~~~~~~~"}],"source_content_type":"text/x-rst","patch_set":8,"id":"bf60c5ca_06523f0c","line":266,"updated":"2022-09-01 19:55:00.000000000","message":"I worry that this sort of access control isn\u0027t sufficient. In the past tenants have largely been able to self manage pipelines, but that didn\u0027t have an impact of behind the scenes cloud resources. We may need to indicate if a tenant is allowed to build images in the zuul tenant config.","commit_id":"781a8470d3790e6257fa5e813e9d8750c41b72bc"},{"author":{"_account_id":4146,"name":"Clark Boylan","email":"cboylan@sapwetik.org","username":"cboylan"},"change_message_id":"ade2bd9a806816fc01a19b60b541a3ddcef87075","unresolved":true,"context_lines":[{"line_number":263,"context_line":"available to each tenant that includes the image definition"},{"line_number":264,"context_line":"configuration object (see the Configuration section below).  Any repo"},{"line_number":265,"context_line":"in a tenant with an image build pipeline will be able to cause images"},{"line_number":266,"context_line":"to be built and uploaded to providers."},{"line_number":267,"context_line":""},{"line_number":268,"context_line":"Snapshot Images"},{"line_number":269,"context_line":"~~~~~~~~~~~~~~~"}],"source_content_type":"text/x-rst","patch_set":8,"id":"df573a9c_ddb97d71","line":266,"in_reply_to":"b7ad0e9b_a8af89a1","updated":"2022-09-07 16:55:18.000000000","message":"The problem is that making images that boot in the various clouds is actually quite difficult. We also have to think about uploaded image quotas. I\u0027m concerned that allowing this broadly will produce two problems:\n\n 1) Users thinking clouds don\u0027t work because they don\u0027t understand how to configure networking on boot. Or otherwise making assumptions like everything is kvm (when xen is possible).\n 2) Users adding images that they never remove when they are no longer needed. OpenDev has a difficult enough time pruning images already and I know people will leave the 10 year old ubuntu image around even though it shouldn\u0027t be used and likely isn\u0027t used. This will impact the clouds negatively through unnecessary resource consumption.\n \nI like the idea of making this configurable and then we can think about this on a tenant by tenant basis at least.","commit_id":"781a8470d3790e6257fa5e813e9d8750c41b72bc"},{"author":{"_account_id":1,"name":"James E. Blair","email":"jim@acmegating.com","username":"corvus"},"change_message_id":"d4b62e133e4e1cc6a8791d587069d005bd40f138","unresolved":true,"context_lines":[{"line_number":263,"context_line":"available to each tenant that includes the image definition"},{"line_number":264,"context_line":"configuration object (see the Configuration section below).  Any repo"},{"line_number":265,"context_line":"in a tenant with an image build pipeline will be able to cause images"},{"line_number":266,"context_line":"to be built and uploaded to providers."},{"line_number":267,"context_line":""},{"line_number":268,"context_line":"Snapshot Images"},{"line_number":269,"context_line":"~~~~~~~~~~~~~~~"}],"source_content_type":"text/x-rst","patch_set":8,"id":"b7ad0e9b_a8af89a1","line":266,"in_reply_to":"bf60c5ca_06523f0c","updated":"2022-09-01 21:13:17.000000000","message":"That makes sense.  We have generally assumed quite a large range of permitted actions from config repos, but already have an `allowed-triggers` tenant config option to restrict what pipelines can do.  Since this is proposed as an enhancement to the `zuul` trigger, that alone isn\u0027t sufficient; we may need to augment that with something like `allowed-zuul-events` or maybe just a boolean `allow-image-builds`.  With that, we can prevent people from adding image builds to pipelines in certain tenants.\n\nHaving said that, I personally think this would be a boon even for opendev to allow tenants to manage their own images.  Most of opendev\u0027s images should be common and built in the opendev tenant, but I could see the utility of tenant-specific images.\n\nPlus with the ability to use job inheritance, we could tune our caching situation a bit better (eg, devstack images are built from a child job that just caches some more blobs).","commit_id":"781a8470d3790e6257fa5e813e9d8750c41b72bc"},{"author":{"_account_id":1,"name":"James E. Blair","email":"jim@acmegating.com","username":"corvus"},"change_message_id":"f755128aa0b149df10338c7d15e38648ba17acd7","unresolved":true,"context_lines":[{"line_number":263,"context_line":"available to each tenant that includes the image definition"},{"line_number":264,"context_line":"configuration object (see the Configuration section below).  Any repo"},{"line_number":265,"context_line":"in a tenant with an image build pipeline will be able to cause images"},{"line_number":266,"context_line":"to be built and uploaded to providers."},{"line_number":267,"context_line":""},{"line_number":268,"context_line":"Snapshot Images"},{"line_number":269,"context_line":"~~~~~~~~~~~~~~~"}],"source_content_type":"text/x-rst","patch_set":8,"id":"dcf0be00_cc6e1864","line":266,"in_reply_to":"df573a9c_ddb97d71","updated":"2022-09-07 17:09:47.000000000","message":"I agree the use case should be supported.  I\u0027m on board with adding allow-image-builds as a tenant config option.","commit_id":"781a8470d3790e6257fa5e813e9d8750c41b72bc"},{"author":{"_account_id":4146,"name":"Clark Boylan","email":"cboylan@sapwetik.org","username":"cboylan"},"change_message_id":"51e4ebb86032ee9c0d587e5a65b40c32f0d60ba5","unresolved":false,"context_lines":[{"line_number":263,"context_line":"available to each tenant that includes the image definition"},{"line_number":264,"context_line":"configuration object (see the Configuration section below).  Any repo"},{"line_number":265,"context_line":"in a tenant with an image build pipeline will be able to cause images"},{"line_number":266,"context_line":"to be built and uploaded to providers."},{"line_number":267,"context_line":""},{"line_number":268,"context_line":"Snapshot Images"},{"line_number":269,"context_line":"~~~~~~~~~~~~~~~"}],"source_content_type":"text/x-rst","patch_set":8,"id":"c8e2aac4_5757dabd","line":266,"in_reply_to":"df573a9c_ddb97d71","updated":"2022-09-07 17:18:08.000000000","message":"Per matrix it sounds like we are all on baord with making this configurable. I don\u0027t think that needs a full respin as we appear to be in agreement.","commit_id":"781a8470d3790e6257fa5e813e9d8750c41b72bc"},{"author":{"_account_id":4146,"name":"Clark Boylan","email":"cboylan@sapwetik.org","username":"cboylan"},"change_message_id":"7be0b064a7c79187fa42b085a39538393c044538","unresolved":true,"context_lines":[{"line_number":282,"context_line":"record that information in ZooKeeper.  Unlike an image-build job, an"},{"line_number":283,"context_line":"image-snapshot job would need to run in each provider (similar to how"},{"line_number":284,"context_line":"it is proposed that an image-validate job will run in each provider)."},{"line_number":285,"context_line":"An image-delete job would not be required."},{"line_number":286,"context_line":""},{"line_number":287,"context_line":""},{"line_number":288,"context_line":"Node Management"}],"source_content_type":"text/x-rst","patch_set":8,"id":"cdec6b3e_0be03209","line":285,"updated":"2022-09-01 19:55:00.000000000","message":"I don\u0027t know if this is possible, but if you can download a snapshotted image after taking the snapshot then upload it to typical cloud storage the rest of the previously described process could continue to be used.","commit_id":"781a8470d3790e6257fa5e813e9d8750c41b72bc"},{"author":{"_account_id":4146,"name":"Clark Boylan","email":"cboylan@sapwetik.org","username":"cboylan"},"change_message_id":"ade2bd9a806816fc01a19b60b541a3ddcef87075","unresolved":false,"context_lines":[{"line_number":282,"context_line":"record that information in ZooKeeper.  Unlike an image-build job, an"},{"line_number":283,"context_line":"image-snapshot job would need to run in each provider (similar to how"},{"line_number":284,"context_line":"it is proposed that an image-validate job will run in each provider)."},{"line_number":285,"context_line":"An image-delete job would not be required."},{"line_number":286,"context_line":""},{"line_number":287,"context_line":""},{"line_number":288,"context_line":"Node Management"}],"source_content_type":"text/x-rst","patch_set":8,"id":"dd196b76_0a4193f1","line":285,"in_reply_to":"8f442f23_5e4e4369","updated":"2022-09-07 16:55:18.000000000","message":"Ack","commit_id":"781a8470d3790e6257fa5e813e9d8750c41b72bc"},{"author":{"_account_id":1,"name":"James E. Blair","email":"jim@acmegating.com","username":"corvus"},"change_message_id":"d4b62e133e4e1cc6a8791d587069d005bd40f138","unresolved":true,"context_lines":[{"line_number":282,"context_line":"record that information in ZooKeeper.  Unlike an image-build job, an"},{"line_number":283,"context_line":"image-snapshot job would need to run in each provider (similar to how"},{"line_number":284,"context_line":"it is proposed that an image-validate job will run in each provider)."},{"line_number":285,"context_line":"An image-delete job would not be required."},{"line_number":286,"context_line":""},{"line_number":287,"context_line":""},{"line_number":288,"context_line":"Node Management"}],"source_content_type":"text/x-rst","patch_set":8,"id":"8f442f23_5e4e4369","line":285,"in_reply_to":"cdec6b3e_0be03209","updated":"2022-09-01 21:13:17.000000000","message":"That might be possible in some environments; if it\u0027s supported universally, that sounds like it would be a good implementation.","commit_id":"781a8470d3790e6257fa5e813e9d8750c41b72bc"},{"author":{"_account_id":5263,"name":"Jeremy Stanley","display_name":"fungi","email":"fungi@yuggoth.org","username":"fungi","status":"missing, presumed fed"},"change_message_id":"18fc09c61e5b4b9ab1984392a8c3adb253d7df07","unresolved":false,"context_lines":[{"line_number":303,"context_line":"  assigning providers intentionally, we can more efficiently utilize"},{"line_number":304,"context_line":"  providers."},{"line_number":305,"context_line":""},{"line_number":306,"context_line":"* Fulfilling node requests from multiple providers: by designing"},{"line_number":307,"context_line":"  zuul-launcher for cooperative work, we can have nodesets that"},{"line_number":308,"context_line":"  request nodes which are fulfilled by different providers.  Generally"},{"line_number":309,"context_line":"  we should favor the same provider for a set of nodes (since they may"}],"source_content_type":"text/x-rst","patch_set":8,"id":"245eb5a0_e9d6fb95","line":306,"updated":"2022-09-08 18:28:50.000000000","message":"Good point. At least from the OpenDev perspective, the only behavior change here would be that suddenly Zuul might agree to run jobs which were previously not possible (like mixing node labels we currently offer from mutually exclusive providers, i.e. x86 and arm64, or in other deployments it might mean the ability for mixing labels tied to different node drivers entirely).","commit_id":"781a8470d3790e6257fa5e813e9d8750c41b72bc"},{"author":{"_account_id":5263,"name":"Jeremy Stanley","display_name":"fungi","email":"fungi@yuggoth.org","username":"fungi","status":"missing, presumed fed"},"change_message_id":"b1d33e587fc8d3b0f6c1cd23d4ab7e29143612d9","unresolved":false,"context_lines":[{"line_number":303,"context_line":"  assigning providers intentionally, we can more efficiently utilize"},{"line_number":304,"context_line":"  providers."},{"line_number":305,"context_line":""},{"line_number":306,"context_line":"* Fulfilling node requests from multiple providers: by designing"},{"line_number":307,"context_line":"  zuul-launcher for cooperative work, we can have nodesets that"},{"line_number":308,"context_line":"  request nodes which are fulfilled by different providers.  Generally"},{"line_number":309,"context_line":"  we should favor the same provider for a set of nodes (since they may"}],"source_content_type":"text/x-rst","patch_set":8,"id":"7abbe285_1b07b489","line":306,"updated":"2022-09-08 15:23:17.000000000","message":"It seems like this is something which should probably be allowed or denied within the nodeset (or potentially by the consuming jobs), since some jobs may rely heavily on local connectivity between nodes.","commit_id":"781a8470d3790e6257fa5e813e9d8750c41b72bc"},{"author":{"_account_id":1,"name":"James E. Blair","email":"jim@acmegating.com","username":"corvus"},"change_message_id":"585045ab1efd018edd3eb71076a92ea6362b5775","unresolved":false,"context_lines":[{"line_number":303,"context_line":"  assigning providers intentionally, we can more efficiently utilize"},{"line_number":304,"context_line":"  providers."},{"line_number":305,"context_line":""},{"line_number":306,"context_line":"* Fulfilling node requests from multiple providers: by designing"},{"line_number":307,"context_line":"  zuul-launcher for cooperative work, we can have nodesets that"},{"line_number":308,"context_line":"  request nodes which are fulfilled by different providers.  Generally"},{"line_number":309,"context_line":"  we should favor the same provider for a set of nodes (since they may"}],"source_content_type":"text/x-rst","patch_set":8,"id":"675b349e_088aa66c","line":306,"in_reply_to":"245eb5a0_e9d6fb95","updated":"2022-09-08 19:49:10.000000000","message":"Yep, so it should be a possible enhancement that is backwards compatible with current behaviors.","commit_id":"781a8470d3790e6257fa5e813e9d8750c41b72bc"},{"author":{"_account_id":1,"name":"James E. Blair","email":"jim@acmegating.com","username":"corvus"},"change_message_id":"f7aeb3a212a2e681bdf73eec88844097f520c728","unresolved":false,"context_lines":[{"line_number":303,"context_line":"  assigning providers intentionally, we can more efficiently utilize"},{"line_number":304,"context_line":"  providers."},{"line_number":305,"context_line":""},{"line_number":306,"context_line":"* Fulfilling node requests from multiple providers: by designing"},{"line_number":307,"context_line":"  zuul-launcher for cooperative work, we can have nodesets that"},{"line_number":308,"context_line":"  request nodes which are fulfilled by different providers.  Generally"},{"line_number":309,"context_line":"  we should favor the same provider for a set of nodes (since they may"}],"source_content_type":"text/x-rst","patch_set":8,"id":"f1736f7f_ed32a309","line":306,"in_reply_to":"7abbe285_1b07b489","updated":"2022-09-08 18:02:26.000000000","message":"I think at least in the initial implementation we would expect that if a provider can supply multiple labels within a nodeset, we would have it fulfill as many as possible, and only yield nodes to other providers which cannot fulfill them.  That would make the system work as it does today transparently.","commit_id":"781a8470d3790e6257fa5e813e9d8750c41b72bc"},{"author":{"_account_id":4146,"name":"Clark Boylan","email":"cboylan@sapwetik.org","username":"cboylan"},"change_message_id":"7be0b064a7c79187fa42b085a39538393c044538","unresolved":true,"context_lines":[{"line_number":310,"context_line":"  need to communicate over a LAN), but if that is not feasible,"},{"line_number":311,"context_line":"  allowing multiple providers to fulfill a request will permit"},{"line_number":312,"context_line":"  nodesets with diverse node types (e.g., VM + static, or VM +"},{"line_number":313,"context_line":"  container)."},{"line_number":314,"context_line":""},{"line_number":315,"context_line":"Each zuul-launcher process will execute a number of processing loops"},{"line_number":316,"context_line":"in series; first a global request processing loop, and then a"}],"source_content_type":"text/x-rst","patch_set":8,"id":"d0c5a635_b89bfc2f","line":313,"updated":"2022-09-01 19:55:00.000000000","message":"There is another goal that might be good to add which is treating requests in a more queue like fashion. Basically we know who is next in line for node type foo and instead of assigning a specific instance boot to that request we can just hand the next successful but to the next item in the queue.\n\nThis gets a little tricky when you have to deal with provider constraints for locality but I think for a lot of jobs this would make users experience a bit more predictable and expected.","commit_id":"781a8470d3790e6257fa5e813e9d8750c41b72bc"},{"author":{"_account_id":1,"name":"James E. Blair","email":"jim@acmegating.com","username":"corvus"},"change_message_id":"d4b62e133e4e1cc6a8791d587069d005bd40f138","unresolved":true,"context_lines":[{"line_number":310,"context_line":"  need to communicate over a LAN), but if that is not feasible,"},{"line_number":311,"context_line":"  allowing multiple providers to fulfill a request will permit"},{"line_number":312,"context_line":"  nodesets with diverse node types (e.g., VM + static, or VM +"},{"line_number":313,"context_line":"  container)."},{"line_number":314,"context_line":""},{"line_number":315,"context_line":"Each zuul-launcher process will execute a number of processing loops"},{"line_number":316,"context_line":"in series; first a global request processing loop, and then a"}],"source_content_type":"text/x-rst","patch_set":8,"id":"f67209df_b942db6c","line":313,"in_reply_to":"d0c5a635_b89bfc2f","updated":"2022-09-01 21:13:17.000000000","message":"I can think of some ways to do that, but so far, they are all very complex and involve having two queues (one queue with the nodeset requests then a second for individual or locality-bound groups of nodes).\n\nIf we do want to set this as a goal, I think the existing algorithm laid out below would need to be replaced and it would be a significant revision of this spec.\n\nPersonally, I\u0027m not sure the benefits are going to be worth the complexity, since even the current system mostly starts jobs in the order you expect (and most of the differences are due to different node types or providers).  But I\u0027m open to the revision if people feel this is important.  It would be harder to implement later.","commit_id":"781a8470d3790e6257fa5e813e9d8750c41b72bc"},{"author":{"_account_id":4146,"name":"Clark Boylan","email":"cboylan@sapwetik.org","username":"cboylan"},"change_message_id":"ade2bd9a806816fc01a19b60b541a3ddcef87075","unresolved":false,"context_lines":[{"line_number":310,"context_line":"  need to communicate over a LAN), but if that is not feasible,"},{"line_number":311,"context_line":"  allowing multiple providers to fulfill a request will permit"},{"line_number":312,"context_line":"  nodesets with diverse node types (e.g., VM + static, or VM +"},{"line_number":313,"context_line":"  container)."},{"line_number":314,"context_line":""},{"line_number":315,"context_line":"Each zuul-launcher process will execute a number of processing loops"},{"line_number":316,"context_line":"in series; first a global request processing loop, and then a"}],"source_content_type":"text/x-rst","patch_set":8,"id":"cdc08672_962805b5","line":313,"in_reply_to":"f67209df_b942db6c","updated":"2022-09-07 16:55:18.000000000","message":"I mostly wanted to call that out as addressing the confusion users have when they see jobs start ahead of theirs for the same node type would be nice. If we are going to address that this seems like a good opportunity for it.\n\nThat said I agree it is a relatively minor thing and something that we can educate users about rather than building more complexity into the tool. I\u0027m good with keeping the old node request assignment behavior instead.","commit_id":"781a8470d3790e6257fa5e813e9d8750c41b72bc"},{"author":{"_account_id":4146,"name":"Clark Boylan","email":"cboylan@sapwetik.org","username":"cboylan"},"change_message_id":"7be0b064a7c79187fa42b085a39538393c044538","unresolved":true,"context_lines":[{"line_number":385,"context_line":"definitions which should be available to a tenant should be included"},{"line_number":386,"context_line":"in that tenant\u0027s config.  Again using the OpenDev example, the"},{"line_number":387,"context_line":"hypothetical `opendev/images` repository should be included in every"},{"line_number":388,"context_line":"OpenDev tenant so all of those images are available."},{"line_number":389,"context_line":""},{"line_number":390,"context_line":"Within a tenant, image names must be unique (otherwise it is a tenant"},{"line_number":391,"context_line":"configuration error, similar to a job name collision)."}],"source_content_type":"text/x-rst","patch_set":8,"id":"e0e5cbc1_598f868b","line":388,"updated":"2022-09-01 19:55:00.000000000","message":"I think this may be at odds with my earlier suggestion of limiting which tenants can define images. Maybe we need a new privileged type of repo? In particular I do not see us wanting to allow any of our tenants in OpenDev to be able to create random images without OpenDev\u0027s review. There are too many gotchas about image format types and how to boot with networking for all the various clouds to allow people to have at it on their own. On top of that images in a cloud represent resource utilization that we\u0027ll want to manage.","commit_id":"781a8470d3790e6257fa5e813e9d8750c41b72bc"},{"author":{"_account_id":4146,"name":"Clark Boylan","email":"cboylan@sapwetik.org","username":"cboylan"},"change_message_id":"ade2bd9a806816fc01a19b60b541a3ddcef87075","unresolved":false,"context_lines":[{"line_number":385,"context_line":"definitions which should be available to a tenant should be included"},{"line_number":386,"context_line":"in that tenant\u0027s config.  Again using the OpenDev example, the"},{"line_number":387,"context_line":"hypothetical `opendev/images` repository should be included in every"},{"line_number":388,"context_line":"OpenDev tenant so all of those images are available."},{"line_number":389,"context_line":""},{"line_number":390,"context_line":"Within a tenant, image names must be unique (otherwise it is a tenant"},{"line_number":391,"context_line":"configuration error, similar to a job name collision)."}],"source_content_type":"text/x-rst","patch_set":8,"id":"202d413b_8f9f9aab","line":388,"in_reply_to":"52b66f62_51656262","updated":"2022-09-07 16:55:18.000000000","message":"Ack","commit_id":"781a8470d3790e6257fa5e813e9d8750c41b72bc"},{"author":{"_account_id":1,"name":"James E. Blair","email":"jim@acmegating.com","username":"corvus"},"change_message_id":"d4b62e133e4e1cc6a8791d587069d005bd40f138","unresolved":true,"context_lines":[{"line_number":385,"context_line":"definitions which should be available to a tenant should be included"},{"line_number":386,"context_line":"in that tenant\u0027s config.  Again using the OpenDev example, the"},{"line_number":387,"context_line":"hypothetical `opendev/images` repository should be included in every"},{"line_number":388,"context_line":"OpenDev tenant so all of those images are available."},{"line_number":389,"context_line":""},{"line_number":390,"context_line":"Within a tenant, image names must be unique (otherwise it is a tenant"},{"line_number":391,"context_line":"configuration error, similar to a job name collision)."}],"source_content_type":"text/x-rst","patch_set":8,"id":"52b66f62_51656262","line":388,"in_reply_to":"e0e5cbc1_598f868b","updated":"2022-09-01 21:13:17.000000000","message":"I don\u0027t think it\u0027s at odds.  As described above, we can add an option to the tenant config to enable/disable building images.  With that in mind, this just describes a repo with all the image definitions and in opendev\u0027s case, we would put them all in that repo and then add that to all the tenants.  Only the opendev tenant would actually build them.\n\nHaving said that, and just as a parenthetical aside that doesn\u0027t really affect this spec, I do think we should consider allowing tenant-specific builds.  I don\u0027t think there\u0027s much harm in that if we provide good base image build jobs.  I only mention this to suggest keeping an open mind for what we might do with opendev.  We\u0027ll definitely support the current \"only the admins touch images\" approach.  But we don\u0027t have to be limited to it if we choose.","commit_id":"781a8470d3790e6257fa5e813e9d8750c41b72bc"},{"author":{"_account_id":4146,"name":"Clark Boylan","email":"cboylan@sapwetik.org","username":"cboylan"},"change_message_id":"7be0b064a7c79187fa42b085a39538393c044538","unresolved":true,"context_lines":[{"line_number":521,"context_line":"   - section:"},{"line_number":522,"context_line":"       name: rax-base"},{"line_number":523,"context_line":"       abstract: true"},{"line_number":524,"context_line":"       connection: rackspace"},{"line_number":525,"context_line":"       boot-timeout: 120"},{"line_number":526,"context_line":"       launch-timeout: 600"},{"line_number":527,"context_line":"       key-name: infra-root-keys-2020-05-13"}],"source_content_type":"text/x-rst","patch_set":8,"id":"68bb41fb_82b1534c","line":524,"updated":"2022-09-01 19:55:00.000000000","message":"An example of the new connection details showing how we interface with external auth info like a clouds.yaml and various regions might be helpful. Though I suspect that is largely a mechanical mapping.","commit_id":"781a8470d3790e6257fa5e813e9d8750c41b72bc"},{"author":{"_account_id":1,"name":"James E. Blair","email":"jim@acmegating.com","username":"corvus"},"change_message_id":"d4b62e133e4e1cc6a8791d587069d005bd40f138","unresolved":true,"context_lines":[{"line_number":521,"context_line":"   - section:"},{"line_number":522,"context_line":"       name: rax-base"},{"line_number":523,"context_line":"       abstract: true"},{"line_number":524,"context_line":"       connection: rackspace"},{"line_number":525,"context_line":"       boot-timeout: 120"},{"line_number":526,"context_line":"       launch-timeout: 600"},{"line_number":527,"context_line":"       key-name: infra-root-keys-2020-05-13"}],"source_content_type":"text/x-rst","patch_set":8,"id":"7f123cd3_c41f8e2f","line":524,"in_reply_to":"68bb41fb_82b1534c","updated":"2022-09-01 21:13:17.000000000","message":"Yes, I think it\u0027s pretty mechanical -- I think it would look sort of like zuul\u0027s existing connections with the necessary fields from nodepool mapped in.  So zuul.conf would look like:\n```\n[connection rackspace]\ndriver\u003dopenstack\ncloud\u003drax\n```","commit_id":"781a8470d3790e6257fa5e813e9d8750c41b72bc"},{"author":{"_account_id":4146,"name":"Clark Boylan","email":"cboylan@sapwetik.org","username":"cboylan"},"change_message_id":"ade2bd9a806816fc01a19b60b541a3ddcef87075","unresolved":false,"context_lines":[{"line_number":521,"context_line":"   - section:"},{"line_number":522,"context_line":"       name: rax-base"},{"line_number":523,"context_line":"       abstract: true"},{"line_number":524,"context_line":"       connection: rackspace"},{"line_number":525,"context_line":"       boot-timeout: 120"},{"line_number":526,"context_line":"       launch-timeout: 600"},{"line_number":527,"context_line":"       key-name: infra-root-keys-2020-05-13"}],"source_content_type":"text/x-rst","patch_set":8,"id":"1eb25155_75949d45","line":524,"in_reply_to":"7f123cd3_c41f8e2f","updated":"2022-09-07 16:55:18.000000000","message":"Ack","commit_id":"781a8470d3790e6257fa5e813e9d8750c41b72bc"},{"author":{"_account_id":4146,"name":"Clark Boylan","email":"cboylan@sapwetik.org","username":"cboylan"},"change_message_id":"7be0b064a7c79187fa42b085a39538393c044538","unresolved":true,"context_lines":[{"line_number":541,"context_line":"         - name: ubuntu"},{"line_number":542,"context_line":"           # This is a cloud image, so the specific cloud image name is required"},{"line_number":543,"context_line":"           image-name: ibm-ubuntu-20-04-3-minimal-amd64-1"},{"line_number":544,"context_line":"           # Other information may be provided"},{"line_number":545,"context_line":"           # username ..."},{"line_number":546,"context_line":"           # python-path: ..."},{"line_number":547,"context_line":"           # shell-type: ..."}],"source_content_type":"text/x-rst","patch_set":8,"id":"94f0f00f_e9b82bdf","line":544,"updated":"2022-09-01 19:55:00.000000000","message":"Are we thinking other information would replace or override/supplement the default in the image definition in the case where both define this info?","commit_id":"781a8470d3790e6257fa5e813e9d8750c41b72bc"},{"author":{"_account_id":1,"name":"James E. Blair","email":"jim@acmegating.com","username":"corvus"},"change_message_id":"bd2c3ef969f4b222d220336892540ce3829f68a7","unresolved":true,"context_lines":[{"line_number":541,"context_line":"         - name: ubuntu"},{"line_number":542,"context_line":"           # This is a cloud image, so the specific cloud image name is required"},{"line_number":543,"context_line":"           image-name: ibm-ubuntu-20-04-3-minimal-amd64-1"},{"line_number":544,"context_line":"           # Other information may be provided"},{"line_number":545,"context_line":"           # username ..."},{"line_number":546,"context_line":"           # python-path: ..."},{"line_number":547,"context_line":"           # shell-type: ..."}],"source_content_type":"text/x-rst","patch_set":8,"id":"2ca50b47_2b9130da","line":544,"in_reply_to":"27bd18e6_ba5bf4a1","updated":"2022-09-02 15:11:54.000000000","message":"Reproducing the relevant lines:\n\n469 * ``image`` stanza                                                                              \n470 * ``flavor`` stanza\n471 * ``label`` stanza\n472 * ``section`` stanza (top level)\n473 * ``image`` within ``section``\n474 * ``flavor`` within ``section``\n475 * ``provider`` stanza (top level)\n476 * ``label`` within ``provider``\n\nCurrently, given X images and Y providers, we set that flag X*Y times.  With this proposal, we could do the same by setting it using line 473.  Or we could set it only once per image (X times) by setting it using line 469.  Or we could set it once per provider (Y times) by setting it using line 475.  Or we could set it once per cloud (let\u0027s call that O(1) time) by setting using line 472 and section inheritance.","commit_id":"781a8470d3790e6257fa5e813e9d8750c41b72bc"},{"author":{"_account_id":27582,"name":"Simon Westphahl","email":"simon.westphahl@bmw.de","username":"simon.westphahl"},"change_message_id":"1186a14befc51af0175bf5e7b02f07184f3339cb","unresolved":false,"context_lines":[{"line_number":541,"context_line":"         - name: ubuntu"},{"line_number":542,"context_line":"           # This is a cloud image, so the specific cloud image name is required"},{"line_number":543,"context_line":"           image-name: ibm-ubuntu-20-04-3-minimal-amd64-1"},{"line_number":544,"context_line":"           # Other information may be provided"},{"line_number":545,"context_line":"           # username ..."},{"line_number":546,"context_line":"           # python-path: ..."},{"line_number":547,"context_line":"           # shell-type: ..."}],"source_content_type":"text/x-rst","patch_set":8,"id":"5769934f_0773cdd2","line":544,"in_reply_to":"2ca50b47_2b9130da","updated":"2022-09-07 17:02:14.000000000","message":"ok, thanks! Missed that bit about top-level overrides.","commit_id":"781a8470d3790e6257fa5e813e9d8750c41b72bc"},{"author":{"_account_id":1,"name":"James E. Blair","email":"jim@acmegating.com","username":"corvus"},"change_message_id":"d4b62e133e4e1cc6a8791d587069d005bd40f138","unresolved":true,"context_lines":[{"line_number":541,"context_line":"         - name: ubuntu"},{"line_number":542,"context_line":"           # This is a cloud image, so the specific cloud image name is required"},{"line_number":543,"context_line":"           image-name: ibm-ubuntu-20-04-3-minimal-amd64-1"},{"line_number":544,"context_line":"           # Other information may be provided"},{"line_number":545,"context_line":"           # username ..."},{"line_number":546,"context_line":"           # python-path: ..."},{"line_number":547,"context_line":"           # shell-type: ..."}],"source_content_type":"text/x-rst","patch_set":8,"id":"9f8d6514_f07f0ca0","line":544,"in_reply_to":"94f0f00f_e9b82bdf","updated":"2022-09-01 21:13:17.000000000","message":"Yes both override and supplement.  The order of precedence is in lines 469-576.\n\nThat addresses one of Tobias\u0027s pet peeves about having to say \"config-drive: true\" everywhere.  :)","commit_id":"781a8470d3790e6257fa5e813e9d8750c41b72bc"},{"author":{"_account_id":27582,"name":"Simon Westphahl","email":"simon.westphahl@bmw.de","username":"simon.westphahl"},"change_message_id":"36d547f86ad0fa173548d469a07f4d23f100fd8d","unresolved":true,"context_lines":[{"line_number":541,"context_line":"         - name: ubuntu"},{"line_number":542,"context_line":"           # This is a cloud image, so the specific cloud image name is required"},{"line_number":543,"context_line":"           image-name: ibm-ubuntu-20-04-3-minimal-amd64-1"},{"line_number":544,"context_line":"           # Other information may be provided"},{"line_number":545,"context_line":"           # username ..."},{"line_number":546,"context_line":"           # python-path: ..."},{"line_number":547,"context_line":"           # shell-type: ..."}],"source_content_type":"text/x-rst","patch_set":8,"id":"27bd18e6_ba5bf4a1","line":544,"in_reply_to":"9f8d6514_f07f0ca0","updated":"2022-09-02 06:00:46.000000000","message":"Just wondering how this solves the issue of having to define/override it for every image? Can the image name be a regex and match multiple images? Otherwise it would just be defined in a different location (image stanza vs. image in section).","commit_id":"781a8470d3790e6257fa5e813e9d8750c41b72bc"},{"author":{"_account_id":4146,"name":"Clark Boylan","email":"cboylan@sapwetik.org","username":"cboylan"},"change_message_id":"7be0b064a7c79187fa42b085a39538393c044538","unresolved":true,"context_lines":[{"line_number":696,"context_line":"interfaces.  As of writing, this uses another 420M of disk space."},{"line_number":697,"context_line":"Since our primary method of distribution at this point is container"},{"line_number":698,"context_line":"images, if the additional space is a concern, we could restrict the"},{"line_number":699,"context_line":"installation of these dependencies to only the zuul-launcher image."},{"line_number":700,"context_line":""},{"line_number":701,"context_line":"Diskimage-Builder Testing"},{"line_number":702,"context_line":"-------------------------"}],"source_content_type":"text/x-rst","patch_set":8,"id":"f4d3ec9e_f1f06bfa","line":699,"updated":"2022-09-01 19:55:00.000000000","message":"I think we should do that similar to how we only install all the extra ansible stuff in the executor image.","commit_id":"781a8470d3790e6257fa5e813e9d8750c41b72bc"},{"author":{"_account_id":4146,"name":"Clark Boylan","email":"cboylan@sapwetik.org","username":"cboylan"},"change_message_id":"ade2bd9a806816fc01a19b60b541a3ddcef87075","unresolved":false,"context_lines":[{"line_number":696,"context_line":"interfaces.  As of writing, this uses another 420M of disk space."},{"line_number":697,"context_line":"Since our primary method of distribution at this point is container"},{"line_number":698,"context_line":"images, if the additional space is a concern, we could restrict the"},{"line_number":699,"context_line":"installation of these dependencies to only the zuul-launcher image."},{"line_number":700,"context_line":""},{"line_number":701,"context_line":"Diskimage-Builder Testing"},{"line_number":702,"context_line":"-------------------------"}],"source_content_type":"text/x-rst","patch_set":8,"id":"911b6b44_cc8072c8","line":699,"in_reply_to":"d3d78a4e_92bce4b4","updated":"2022-09-07 16:55:18.000000000","message":"Ack","commit_id":"781a8470d3790e6257fa5e813e9d8750c41b72bc"},{"author":{"_account_id":1,"name":"James E. Blair","email":"jim@acmegating.com","username":"corvus"},"change_message_id":"d4b62e133e4e1cc6a8791d587069d005bd40f138","unresolved":true,"context_lines":[{"line_number":696,"context_line":"interfaces.  As of writing, this uses another 420M of disk space."},{"line_number":697,"context_line":"Since our primary method of distribution at this point is container"},{"line_number":698,"context_line":"images, if the additional space is a concern, we could restrict the"},{"line_number":699,"context_line":"installation of these dependencies to only the zuul-launcher image."},{"line_number":700,"context_line":""},{"line_number":701,"context_line":"Diskimage-Builder Testing"},{"line_number":702,"context_line":"-------------------------"}],"source_content_type":"text/x-rst","patch_set":8,"id":"d3d78a4e_92bce4b4","line":699,"in_reply_to":"f4d3ec9e_f1f06bfa","updated":"2022-09-01 21:13:17.000000000","message":"That\u0027s the way I\u0027m leaning too.","commit_id":"781a8470d3790e6257fa5e813e9d8750c41b72bc"}]}