)]}'
{"doc/source/developer/specs/container-build-resources.rst":[{"author":{"_account_id":6488,"name":"Clint Byrum","email":"clint@fewbar.com","username":"clint-fewbar"},"change_message_id":"aae2592a17706be3c2d231d62a542afaae39bbe7","unresolved":false,"context_lines":[{"line_number":21,"context_line":"* Containers that behave like a machine"},{"line_number":22,"context_line":"* Native container workflow"},{"line_number":23,"context_line":""},{"line_number":24,"context_line":"Finally, there are a multiple container environments.  Kubernetes and"},{"line_number":25,"context_line":"OpenShift (an open source distribution of Kubernetes) are popular"},{"line_number":26,"context_line":"environments which provide significant infrastructure to help us more"},{"line_number":27,"context_line":"easily integrate them with Zuul, so this document will focus on these."}],"source_content_type":"text/x-rst","patch_set":1,"id":"5f7c97a3_af3bfcce","line":24,"updated":"2018-05-23 17:22:35.000000000","message":"s/there are a/there are/","commit_id":"676f09796afbb1cc5949978427af038352935943"},{"author":{"_account_id":9311,"name":"Tristan Cacqueray","email":"tdecacqu@redhat.com","username":"tristanC"},"change_message_id":"309c2b0b91eec70ff9a56abe9d05b63c7e66db29","unresolved":false,"context_lines":[{"line_number":81,"context_line":"discussion of how the git repo state can be synchronised, see"},{"line_number":82,"context_line":":ref:`git-repo-sync`."},{"line_number":83,"context_line":""},{"line_number":84,"context_line":".. _kubectl connection plugin: https://docs.ansible.com/ansible/2.5/plugins/connection/kubectl.html"},{"line_number":85,"context_line":""},{"line_number":86,"context_line":".. _container-native:"},{"line_number":87,"context_line":""}],"source_content_type":"text/x-rst","patch_set":1,"id":"bf659307_14bd3ad7","line":84,"updated":"2018-04-11 14:49:31.000000000","message":"Perhaps we should list the tasks to look for when running with the kubectl or raw ansible module? For example synchronize or zuul_stream may work differently.\n\nOr are we expecting a new set of zuul-jobs when using container build resources?","commit_id":"676f09796afbb1cc5949978427af038352935943"},{"author":{"_account_id":1,"name":"James E. Blair","email":"jim@acmegating.com","username":"corvus"},"change_message_id":"9ecd1b41f6951989ebd5147ed9f7c9871ce9c7fb","unresolved":false,"context_lines":[{"line_number":81,"context_line":"discussion of how the git repo state can be synchronised, see"},{"line_number":82,"context_line":":ref:`git-repo-sync`."},{"line_number":83,"context_line":""},{"line_number":84,"context_line":".. _kubectl connection plugin: https://docs.ansible.com/ansible/2.5/plugins/connection/kubectl.html"},{"line_number":85,"context_line":""},{"line_number":86,"context_line":".. _container-native:"},{"line_number":87,"context_line":""}],"source_content_type":"text/x-rst","patch_set":1,"id":"bf659307_78281d85","line":84,"updated":"2018-04-11 23:29:15.000000000","message":"Tristan, indeed -- synchronize may or may not work (I\u0027m not sure about that).  I don\u0027t have an answer for zuul_stream, but I bet we could, if we wanted, plumb the container stdout to the log streaming system.  Aside from those things (which are handled in base jobs, and therefore, we can solve them once for the whole system), I think that most simple jobs should \"just work\".  That is, once you\u0027ve got the git repos copied over, whether \"tox\" is executed by kubectl or ssh the results should be the same.\n\nWe should definitely update this to include this info.\n\nClark, ssh-ing into a container is certainly a thing one might want to do, however, it requires setting up a service and ingress access to the cluster, which is not always guaranteed to be present.  Being able to support the non-ssh case means we can work in more environments (and possibly with smaller images).\n\nBut maybe we should support the ssh-case better as well?","commit_id":"676f09796afbb1cc5949978427af038352935943"},{"author":{"_account_id":1,"name":"James E. Blair","email":"jim@acmegating.com","username":"corvus"},"change_message_id":"eeecdccee8df9fee477cead0dd3c4f2e0d9a85cd","unresolved":false,"context_lines":[{"line_number":81,"context_line":"discussion of how the git repo state can be synchronised, see"},{"line_number":82,"context_line":":ref:`git-repo-sync`."},{"line_number":83,"context_line":""},{"line_number":84,"context_line":".. _kubectl connection plugin: https://docs.ansible.com/ansible/2.5/plugins/connection/kubectl.html"},{"line_number":85,"context_line":""},{"line_number":86,"context_line":".. _container-native:"},{"line_number":87,"context_line":""}],"source_content_type":"text/x-rst","patch_set":1,"id":"bf659307_333cf12d","line":84,"updated":"2018-04-12 15:55:31.000000000","message":"Upload-logs doesn\u0027t interact with the worker, so that\u0027s not relevant here.\n\nWhy would tox_install_sibling_packages not work?\n\nThe fetch roles do rely on synchronize.  We may be able to get synchronize to work with kubectl (if it works with docker, it may be able to work with kubectl).\n\nWe can *certainly* run a container with sshd, but I think this is much more compelling if we design for being able to execute simple container workloads without it.  Requiring ssh and ingress support is a big thing to ask and puts us at a disadvantage.\n\nOf course if it better fits the specific workload, a user *can* run sshd.  So to be clear about what we want, how about I update this to explicitly discuss the non-sshd use case, but also the sshd use-case, and what\u0027s needed to support both?","commit_id":"676f09796afbb1cc5949978427af038352935943"},{"author":{"_account_id":4146,"name":"Clark Boylan","email":"cboylan@sapwetik.org","username":"cboylan"},"change_message_id":"e9ca73812a16c88cf90d344c4fbd2b05f3ae7c98","unresolved":false,"context_lines":[{"line_number":81,"context_line":"discussion of how the git repo state can be synchronised, see"},{"line_number":82,"context_line":":ref:`git-repo-sync`."},{"line_number":83,"context_line":""},{"line_number":84,"context_line":".. _kubectl connection plugin: https://docs.ansible.com/ansible/2.5/plugins/connection/kubectl.html"},{"line_number":85,"context_line":""},{"line_number":86,"context_line":".. _container-native:"},{"line_number":87,"context_line":""}],"source_content_type":"text/x-rst","patch_set":1,"id":"bf659307_15ba116d","line":84,"in_reply_to":"bf659307_14bd3ad7","updated":"2018-04-11 18:29:08.000000000","message":"Out of curiosity if the intent is to run VM compute like containers why not make the command to run sshd and just ssh in like other VM/baremetal compute instances?\n\nThen synchronize and just about everything else should continue to work as is right?\n\nI think it may be simplest to just have the general compute base job that expects eg ssh and then that works regardless of being a container, vm, or baremetal.\n\nAdditionally doing it this way would more easily open the door to people using lxc instead of k8s and so on. Since the base interaction remains the same and you\u0027d just need to update nodepool to speak to the provisioner.","commit_id":"676f09796afbb1cc5949978427af038352935943"},{"author":{"_account_id":4146,"name":"Clark Boylan","email":"cboylan@sapwetik.org","username":"cboylan"},"change_message_id":"16f1702d845a3837da4c088af13228343ea404d5","unresolved":false,"context_lines":[{"line_number":81,"context_line":"discussion of how the git repo state can be synchronised, see"},{"line_number":82,"context_line":":ref:`git-repo-sync`."},{"line_number":83,"context_line":""},{"line_number":84,"context_line":".. _kubectl connection plugin: https://docs.ansible.com/ansible/2.5/plugins/connection/kubectl.html"},{"line_number":85,"context_line":""},{"line_number":86,"context_line":".. _container-native:"},{"line_number":87,"context_line":""}],"source_content_type":"text/x-rst","patch_set":1,"id":"9f6a8fd7_bcd1f7cf","line":84,"in_reply_to":"bf659307_333cf12d","updated":"2018-04-19 16:35:51.000000000","message":"++ I think it would be fine to support both, to make it easy for people that want container like VMs using a system like lxd and also to support people who already have a k8s and don\u0027t want to deploy something new but have similar functionality. I just didn\u0027t want to exclude non k8s systems by coupling tightly to k8s expectations.","commit_id":"676f09796afbb1cc5949978427af038352935943"},{"author":{"_account_id":9311,"name":"Tristan Cacqueray","email":"tdecacqu@redhat.com","username":"tristanC"},"change_message_id":"3fd64f10b35bcf72dc5b214cbb992e47c1c10608","unresolved":false,"context_lines":[{"line_number":81,"context_line":"discussion of how the git repo state can be synchronised, see"},{"line_number":82,"context_line":":ref:`git-repo-sync`."},{"line_number":83,"context_line":""},{"line_number":84,"context_line":".. _kubectl connection plugin: https://docs.ansible.com/ansible/2.5/plugins/connection/kubectl.html"},{"line_number":85,"context_line":""},{"line_number":86,"context_line":".. _container-native:"},{"line_number":87,"context_line":""}],"source_content_type":"text/x-rst","patch_set":1,"id":"bf659307_0ed99854","line":84,"in_reply_to":"bf659307_78281d85","updated":"2018-04-12 04:25:12.000000000","message":"James, there is also the custom task such as tox_install_sibling_packages which may not work too. Also note that according to the synchronize module source, it currently only works via ssh, docker client or a direct filesystem. This would be a big drawback as upload-logs and fetch-* roles relies on it.\n\nI agree with Clark, containers behaving like a machine should run sshd.","commit_id":"676f09796afbb1cc5949978427af038352935943"},{"author":{"_account_id":6488,"name":"Clint Byrum","email":"clint@fewbar.com","username":"clint-fewbar"},"change_message_id":"aae2592a17706be3c2d231d62a542afaae39bbe7","unresolved":false,"context_lines":[{"line_number":129,"context_line":""},{"line_number":130,"context_line":"OpenShift provides some features that make this easier, so we\u0027ll start"},{"line_number":131,"context_line":"there."},{"line_number":132,"context_line":""},{"line_number":133,"context_line":"We can ask Nodepool to create an `OpenShift project`_ for the use of"},{"line_number":134,"context_line":"the job.  That will create a private image repository for the project."},{"line_number":135,"context_line":"Service accounts in the project are automatically created with"}],"source_content_type":"text/x-rst","patch_set":1,"id":"5f7c97a3_cf3678e8","line":132,"updated":"2018-05-23 17:22:35.000000000","message":"This is in line with something I\u0027ve been thinking about, which is that instead of \u0027nodepool\u0027 it could be changed to \u0027thingpool\u0027. In many cases, I want more than just instances from my cloud. I want load balancers, volumes, images, etc. So, it would be interesting to see if we can change nodepool from being node/ssh centric to being API-generic, with OpenShift projects just being one of the many \"things\" it could do.","commit_id":"676f09796afbb1cc5949978427af038352935943"},{"author":{"_account_id":1,"name":"James E. Blair","email":"jim@acmegating.com","username":"corvus"},"change_message_id":"9ecd1b41f6951989ebd5147ed9f7c9871ce9c7fb","unresolved":false,"context_lines":[{"line_number":131,"context_line":"there."},{"line_number":132,"context_line":""},{"line_number":133,"context_line":"We can ask Nodepool to create an `OpenShift project`_ for the use of"},{"line_number":134,"context_line":"the job.  That will create a private image repository for the project."},{"line_number":135,"context_line":"Service accounts in the project are automatically created with"},{"line_number":136,"context_line":"``imagePullSecrets`` configured to use the private image repository [#f1]_."},{"line_number":137,"context_line":"We can have Zuul use one of the default service accouns, or have"}],"source_content_type":"text/x-rst","patch_set":1,"id":"bf659307_982df196","line":134,"updated":"2018-04-11 23:29:15.000000000","message":"That looks to be an administrative choice.  But yeah, I guess we have \"quota\". :)","commit_id":"676f09796afbb1cc5949978427af038352935943"},{"author":{"_account_id":3099,"name":"David Shrewsbury","email":"dshrewsb@redhat.com","username":"dshrews"},"change_message_id":"62578b503d989f3633f799113c5c30eb1f26b41f","unresolved":false,"context_lines":[{"line_number":130,"context_line":"OpenShift provides some features that make this easier, so we\u0027ll start"},{"line_number":131,"context_line":"there."},{"line_number":132,"context_line":""},{"line_number":133,"context_line":"We can ask Nodepool to create an `OpenShift project`_ for the use of"},{"line_number":134,"context_line":"the job.  That will create a private image repository for the project."},{"line_number":135,"context_line":"Service accounts in the project are automatically created with"},{"line_number":136,"context_line":"``imagePullSecrets`` configured to use the private image repository [#f1]_."},{"line_number":137,"context_line":"We can have Zuul use one of the default service accouns, or have"}],"source_content_type":"text/x-rst","patch_set":1,"id":"bf659307_6e6709b9","line":134,"range":{"start_line":133,"start_character":0,"end_line":134,"end_character":70},"updated":"2018-04-11 13:27:06.000000000","message":"Will be interesting to consider any OpenShift limits here (which I\u0027m unfamiliar with). E.g., the link below discusses limits on the number of projects that can be created. \"admin\" level users do not have a limit, but not sure of the implications of Nodepool using an account at that level.\n\nhttps://docs.openshift.com/container-platform/3.3/admin_guide/managing_projects.html#limit-projects-per-user","commit_id":"676f09796afbb1cc5949978427af038352935943"},{"author":{"_account_id":3099,"name":"David Shrewsbury","email":"dshrewsb@redhat.com","username":"dshrews"},"change_message_id":"62578b503d989f3633f799113c5c30eb1f26b41f","unresolved":false,"context_lines":[{"line_number":134,"context_line":"the job.  That will create a private image repository for the project."},{"line_number":135,"context_line":"Service accounts in the project are automatically created with"},{"line_number":136,"context_line":"``imagePullSecrets`` configured to use the private image repository [#f1]_."},{"line_number":137,"context_line":"We can have Zuul use one of the default service accouns, or have"},{"line_number":138,"context_line":"Nodepool create a new one specifically for Zuul, and then when using"},{"line_number":139,"context_line":"the `k8s_raw Ansible module`_, the image registry will automatically be"},{"line_number":140,"context_line":"used."}],"source_content_type":"text/x-rst","patch_set":1,"id":"bf659307_2e0b51d6","line":137,"range":{"start_line":137,"start_character":48,"end_line":137,"end_character":55},"updated":"2018-04-11 13:27:06.000000000","message":"nit: s/accouns/accounts/","commit_id":"676f09796afbb1cc5949978427af038352935943"},{"author":{"_account_id":1,"name":"James E. Blair","email":"jim@acmegating.com","username":"corvus"},"change_message_id":"9ecd1b41f6951989ebd5147ed9f7c9871ce9c7fb","unresolved":false,"context_lines":[{"line_number":150,"context_line":".. _git-repo-sync:"},{"line_number":151,"context_line":""},{"line_number":152,"context_line":"Synchronizing Git Repos"},{"line_number":153,"context_line":"-----------------------"},{"line_number":154,"context_line":""},{"line_number":155,"context_line":"Our existing method of synchronizing git repositories onto a worker"},{"line_number":156,"context_line":"node relies on SSH.  It\u0027s possible to run an SSH daemon in a container"}],"source_content_type":"text/x-rst","patch_set":1,"id":"bf659307_381e255f","line":153,"updated":"2018-04-11 23:29:15.000000000","message":"IIUC, that\u0027s basically the \"run a process in a container to build an image\" model, so it might fit into the \u0027container native workflow\u0027 process.  The question is: how to get the speculative source code into that image.  It looks like s2i does that by \"downloading the source\" inside the container.  But Zuul v3 is a push system; there\u0027s no place to download the source from.  That\u0027s why this section describes pushing git repos into a container.\n\nBut if we push our git repo state onto a sidecar container attached to a pod running s2i with a shared volume, the s2i script could just be \"copy over the git repos from the shared volume\".  I don\u0027t know much about s2i though, so there\u0027s probably more to explore here.","commit_id":"676f09796afbb1cc5949978427af038352935943"},{"author":{"_account_id":9311,"name":"Tristan Cacqueray","email":"tdecacqu@redhat.com","username":"tristanC"},"change_message_id":"309c2b0b91eec70ff9a56abe9d05b63c7e66db29","unresolved":false,"context_lines":[{"line_number":150,"context_line":".. _git-repo-sync:"},{"line_number":151,"context_line":""},{"line_number":152,"context_line":"Synchronizing Git Repos"},{"line_number":153,"context_line":"-----------------------"},{"line_number":154,"context_line":""},{"line_number":155,"context_line":"Our existing method of synchronizing git repositories onto a worker"},{"line_number":156,"context_line":"node relies on SSH.  It\u0027s possible to run an SSH daemon in a container"}],"source_content_type":"text/x-rst","patch_set":1,"id":"bf659307_34be3eb4","line":153,"updated":"2018-04-11 14:49:31.000000000","message":"Perhaps this could be solved using the OpenShift s2i tool: https://docs.openshift.com/enterprise/3.0/creating_images/s2i.html","commit_id":"676f09796afbb1cc5949978427af038352935943"},{"author":{"_account_id":1,"name":"James E. Blair","email":"jim@acmegating.com","username":"corvus"},"change_message_id":"fc0429881e2797b678f926c8c67f56c5afe7269e","unresolved":false,"context_lines":[{"line_number":150,"context_line":".. _git-repo-sync:"},{"line_number":151,"context_line":""},{"line_number":152,"context_line":"Synchronizing Git Repos"},{"line_number":153,"context_line":"-----------------------"},{"line_number":154,"context_line":""},{"line_number":155,"context_line":"Our existing method of synchronizing git repositories onto a worker"},{"line_number":156,"context_line":"node relies on SSH.  It\u0027s possible to run an SSH daemon in a container"}],"source_content_type":"text/x-rst","patch_set":1,"id":"9f6a8fd7_f30ec7e3","line":153,"updated":"2018-05-02 14:38:54.000000000","message":"S2I is itself an image that is used to run a container process which builds an image.  To run it on the executor means requiring that the executor run a docker daemon.  That\u0027s out of scope.","commit_id":"676f09796afbb1cc5949978427af038352935943"},{"author":{"_account_id":1,"name":"James E. Blair","email":"jim@acmegating.com","username":"corvus"},"change_message_id":"eeecdccee8df9fee477cead0dd3c4f2e0d9a85cd","unresolved":false,"context_lines":[{"line_number":150,"context_line":".. _git-repo-sync:"},{"line_number":151,"context_line":""},{"line_number":152,"context_line":"Synchronizing Git Repos"},{"line_number":153,"context_line":"-----------------------"},{"line_number":154,"context_line":""},{"line_number":155,"context_line":"Our existing method of synchronizing git repositories onto a worker"},{"line_number":156,"context_line":"node relies on SSH.  It\u0027s possible to run an SSH daemon in a container"}],"source_content_type":"text/x-rst","patch_set":1,"id":"bf659307_d34495ba","line":153,"updated":"2018-04-12 15:55:31.000000000","message":"The executor is not remotely accessible.","commit_id":"676f09796afbb1cc5949978427af038352935943"},{"author":{"_account_id":9311,"name":"Tristan Cacqueray","email":"tdecacqu@redhat.com","username":"tristanC"},"change_message_id":"3fd64f10b35bcf72dc5b214cbb992e47c1c10608","unresolved":false,"context_lines":[{"line_number":150,"context_line":".. _git-repo-sync:"},{"line_number":151,"context_line":""},{"line_number":152,"context_line":"Synchronizing Git Repos"},{"line_number":153,"context_line":"-----------------------"},{"line_number":154,"context_line":""},{"line_number":155,"context_line":"Our existing method of synchronizing git repositories onto a worker"},{"line_number":156,"context_line":"node relies on SSH.  It\u0027s possible to run an SSH daemon in a container"}],"source_content_type":"text/x-rst","patch_set":1,"id":"bf659307_ee7a2422","line":153,"in_reply_to":"bf659307_381e255f","updated":"2018-04-12 04:25:12.000000000","message":"Couldn\u0027t we just make the s2i process fetch from the zuul-merger/executor? Container shouldn\u0027t have the tool to build and install the source, iiuc the image needs to be built before being started and tested.","commit_id":"676f09796afbb1cc5949978427af038352935943"},{"author":{"_account_id":9311,"name":"Tristan Cacqueray","email":"tdecacqu@redhat.com","username":"tristanC"},"change_message_id":"c1d9196b28e5aa13cf51baa61982f09ab79bc477","unresolved":false,"context_lines":[{"line_number":150,"context_line":".. _git-repo-sync:"},{"line_number":151,"context_line":""},{"line_number":152,"context_line":"Synchronizing Git Repos"},{"line_number":153,"context_line":"-----------------------"},{"line_number":154,"context_line":""},{"line_number":155,"context_line":"Our existing method of synchronizing git repositories onto a worker"},{"line_number":156,"context_line":"node relies on SSH.  It\u0027s possible to run an SSH daemon in a container"}],"source_content_type":"text/x-rst","patch_set":1,"id":"bf659307_056e38ff","line":153,"in_reply_to":"bf659307_d34495ba","updated":"2018-04-19 03:26:28.000000000","message":"Then maybe the s2i process could run on the executor?","commit_id":"676f09796afbb1cc5949978427af038352935943"},{"author":{"_account_id":4146,"name":"Clark Boylan","email":"cboylan@sapwetik.org","username":"cboylan"},"change_message_id":"e9ca73812a16c88cf90d344c4fbd2b05f3ae7c98","unresolved":false,"context_lines":[{"line_number":160,"context_line":"SSH server.  However, it\u0027s always possible to run commands in a"},{"line_number":161,"context_line":"container using kubectl with direct stdin/stdout connections without"},{"line_number":162,"context_line":"any of the service/ingress complications.  It should be possible to"},{"line_number":163,"context_line":"adapt our process to use this."},{"line_number":164,"context_line":""},{"line_number":165,"context_line":"Our current process will use a git cache if present on the worker"},{"line_number":166,"context_line":"image.  This is optional -- a Zuul user does not need a specially"}],"source_content_type":"text/x-rst","patch_set":1,"id":"bf659307_95014109","line":163,"updated":"2018-04-11 18:29:08.000000000","message":"I think some of this cumbersomeness may be beneficial if it doesn\u0027t tie you to a specific container tool like k8s and since you\u0027d have to run a long lived process in k8s anyways having that be sshd doesn\u0027t seem like too much effort?\n\nI think my biggest concern with this is we are calling it the  containers that behave like a machine but then tying it explicitly to k8s which does the opposite of behaving like a machine. I would expect users to that want containers to behave like a machine to be looking at alternatives like lxc/lxd since that is what they do (act like machines).\n\nMaybe we \"optimize\" k8s for this use case if k8s is chosen but not necessarily build the system around that expectation?","commit_id":"676f09796afbb1cc5949978427af038352935943"},{"author":{"_account_id":1,"name":"James E. Blair","email":"jim@acmegating.com","username":"corvus"},"change_message_id":"9ecd1b41f6951989ebd5147ed9f7c9871ce9c7fb","unresolved":false,"context_lines":[{"line_number":160,"context_line":"SSH server.  However, it\u0027s always possible to run commands in a"},{"line_number":161,"context_line":"container using kubectl with direct stdin/stdout connections without"},{"line_number":162,"context_line":"any of the service/ingress complications.  It should be possible to"},{"line_number":163,"context_line":"adapt our process to use this."},{"line_number":164,"context_line":""},{"line_number":165,"context_line":"Our current process will use a git cache if present on the worker"},{"line_number":166,"context_line":"image.  This is optional -- a Zuul user does not need a specially"}],"source_content_type":"text/x-rst","patch_set":1,"id":"bf659307_5823d9a7","line":163,"updated":"2018-04-11 23:29:15.000000000","message":"The ssh aspect is addressed above.  This process is in its own section because it has relevance to both \u0027containers like a machine\u0027 and \u0027container native workflow\u0027.  It\u0027s really hard for me to see how to build a container image based on a git repo in a container without this, and that\u0027s step 0 for container native.\n\nI do think this can apply to any container system where you can run a command in a container with stdin/stdout connected.","commit_id":"676f09796afbb1cc5949978427af038352935943"},{"author":{"_account_id":9311,"name":"Tristan Cacqueray","email":"tdecacqu@redhat.com","username":"tristanC"},"change_message_id":"3fd64f10b35bcf72dc5b214cbb992e47c1c10608","unresolved":false,"context_lines":[{"line_number":160,"context_line":"SSH server.  However, it\u0027s always possible to run commands in a"},{"line_number":161,"context_line":"container using kubectl with direct stdin/stdout connections without"},{"line_number":162,"context_line":"any of the service/ingress complications.  It should be possible to"},{"line_number":163,"context_line":"adapt our process to use this."},{"line_number":164,"context_line":""},{"line_number":165,"context_line":"Our current process will use a git cache if present on the worker"},{"line_number":166,"context_line":"image.  This is optional -- a Zuul user does not need a specially"}],"source_content_type":"text/x-rst","patch_set":1,"id":"bf659307_4e1470a0","line":163,"in_reply_to":"bf659307_5823d9a7","updated":"2018-04-12 04:25:12.000000000","message":"That sounds like a special use-case where the image is built inside a container. Projects may build the software on a regular system and only install the final package into an image.","commit_id":"676f09796afbb1cc5949978427af038352935943"},{"author":{"_account_id":6488,"name":"Clint Byrum","email":"clint@fewbar.com","username":"clint-fewbar"},"change_message_id":"aae2592a17706be3c2d231d62a542afaae39bbe7","unresolved":false,"context_lines":[{"line_number":168,"context_line":"a container environment, we can similarly have Nodepool build"},{"line_number":169,"context_line":"container images with a git repo cache.  The next step in the process"},{"line_number":170,"context_line":"can either start with one of those, or any other base image."},{"line_number":171,"context_line":""},{"line_number":172,"context_line":"Create a new pod based on either the git repo cache image, or a base"},{"line_number":173,"context_line":"image.  Ensure it has ``git`` installed.  If the pod is going to be"},{"line_number":174,"context_line":"used to run a single command (i.e., :ref:`container-machine`, or will"}],"source_content_type":"text/x-rst","patch_set":1,"id":"5f7c97a3_6f35e4d9","line":171,"updated":"2018-05-23 17:22:35.000000000","message":"I think I like the idea of building a container image with the git repos in it, and pushing that into wherever the k8s needs it. This will be an extremely quick process, just a FROM whatever-is-already-built and a push to a registry. I don\u0027t even think it needs to be all that isolated from other builds, it will get a unique ID and we can use that.","commit_id":"676f09796afbb1cc5949978427af038352935943"},{"author":{"_account_id":4146,"name":"Clark Boylan","email":"cboylan@sapwetik.org","username":"cboylan"},"change_message_id":"e9ca73812a16c88cf90d344c4fbd2b05f3ae7c98","unresolved":false,"context_lines":[{"line_number":220,"context_line":"container images once at the top, and then supports multiple jobs"},{"line_number":221,"context_line":"which deploy and exercise those images.  The use of a private image"},{"line_number":222,"context_line":"registry is particularly suited to this.  We should explore ways of"},{"line_number":223,"context_line":"supporting this use case."},{"line_number":224,"context_line":""},{"line_number":225,"context_line":"On the other hand, folks may want jobs in a buildset to be completely"},{"line_number":226,"context_line":"isolated from each other, so we may not want to simply assume that all"}],"source_content_type":"text/x-rst","patch_set":1,"id":"bf659307_30d40320","line":223,"updated":"2018-04-11 18:29:08.000000000","message":"Based on our current set of container tester users I expect that this particular use case will be very popular. And that for most users this is what they will want at least for their \"base\" testing.\n\n++ to supporting this use case.","commit_id":"676f09796afbb1cc5949978427af038352935943"},{"author":{"_account_id":4146,"name":"Clark Boylan","email":"cboylan@sapwetik.org","username":"cboylan"},"change_message_id":"e9ca73812a16c88cf90d344c4fbd2b05f3ae7c98","unresolved":false,"context_lines":[{"line_number":224,"context_line":""},{"line_number":225,"context_line":"On the other hand, folks may want jobs in a buildset to be completely"},{"line_number":226,"context_line":"isolated from each other, so we may not want to simply assume that all"},{"line_number":227,"context_line":"jobs in a buildset are related and use the same image registry."},{"line_number":228,"context_line":""},{"line_number":229,"context_line":"Some ideas we could explore:"},{"line_number":230,"context_line":""}],"source_content_type":"text/x-rst","patch_set":1,"id":"bf659307_90eaafd5","line":227,"updated":"2018-04-11 18:29:08.000000000","message":"Worth considering that using the build job then N test jobs model isn\u0027t limited to having a single build job with N test jobs.\n\nWe could set it up such that for each partition there was a separate build job X with N_x test jobs following it to handle the general case.\n\nThen you end up having the same mechanism regardless of need.","commit_id":"676f09796afbb1cc5949978427af038352935943"},{"author":{"_account_id":1,"name":"James E. Blair","email":"jim@acmegating.com","username":"corvus"},"change_message_id":"9ecd1b41f6951989ebd5147ed9f7c9871ce9c7fb","unresolved":false,"context_lines":[{"line_number":224,"context_line":""},{"line_number":225,"context_line":"On the other hand, folks may want jobs in a buildset to be completely"},{"line_number":226,"context_line":"isolated from each other, so we may not want to simply assume that all"},{"line_number":227,"context_line":"jobs in a buildset are related and use the same image registry."},{"line_number":228,"context_line":""},{"line_number":229,"context_line":"Some ideas we could explore:"},{"line_number":230,"context_line":""}],"source_content_type":"text/x-rst","patch_set":1,"id":"bf659307_f83bcdcd","line":227,"updated":"2018-04-11 23:29:15.000000000","message":"Yeah, IIUC, the suggestion on line 231 can be generalized as: each root node in a buildset\u0027s job graph gets its own registry.","commit_id":"676f09796afbb1cc5949978427af038352935943"},{"author":{"_account_id":4146,"name":"Clark Boylan","email":"cboylan@sapwetik.org","username":"cboylan"},"change_message_id":"e9ca73812a16c88cf90d344c4fbd2b05f3ae7c98","unresolved":false,"context_lines":[{"line_number":238,"context_line":"  instead implement it using Kubernetes primitives in a job at the top"},{"line_number":239,"context_line":"  of the buildset.  Add a facility that would allow the user to tell"},{"line_number":240,"context_line":"  Zuul to keep the resources used by that job (i.e., the registry"},{"line_number":241,"context_line":"  service) continually running until the end of the buildset."},{"line_number":242,"context_line":""},{"line_number":243,"context_line":"In order to support this, we may need to implement provider affinity"},{"line_number":244,"context_line":"for builds in a buildset in Nodepool so that we don\u0027t have to deal"}],"source_content_type":"text/x-rst","patch_set":1,"id":"bf659307_5090974a","line":241,"updated":"2018-04-11 18:29:08.000000000","message":"Similarly to my concerns about assuming k8s for the container like machine case, I think it may be beneficial to not assume openshift for the container orchestration case. For this particular case assuming k8s is probably fine since it appears to be the thing everyone is using, but not sure we should assume additional openshift features.","commit_id":"676f09796afbb1cc5949978427af038352935943"},{"author":{"_account_id":9311,"name":"Tristan Cacqueray","email":"tdecacqu@redhat.com","username":"tristanC"},"change_message_id":"3fd64f10b35bcf72dc5b214cbb992e47c1c10608","unresolved":false,"context_lines":[{"line_number":238,"context_line":"  instead implement it using Kubernetes primitives in a job at the top"},{"line_number":239,"context_line":"  of the buildset.  Add a facility that would allow the user to tell"},{"line_number":240,"context_line":"  Zuul to keep the resources used by that job (i.e., the registry"},{"line_number":241,"context_line":"  service) continually running until the end of the buildset."},{"line_number":242,"context_line":""},{"line_number":243,"context_line":"In order to support this, we may need to implement provider affinity"},{"line_number":244,"context_line":"for builds in a buildset in Nodepool so that we don\u0027t have to deal"}],"source_content_type":"text/x-rst","patch_set":1,"id":"bf659307_8ea7e850","line":241,"in_reply_to":"bf659307_5090974a","updated":"2018-04-12 04:25:12.000000000","message":"I agree with Clark, moreover OpenShift projects may already be using the included ci/cd pipelines. Shouldn\u0027t the spec be splitted in two, one section for image building (e.g. covering docker file or buildah script), and one section for image runtime covering different providers like runc, docker or k8s?","commit_id":"676f09796afbb1cc5949978427af038352935943"},{"author":{"_account_id":7069,"name":"Joshua Hesketh","email":"josh@nitrotech.org","username":"jhesketh"},"change_message_id":"a29033caff410ad6c4c009fb1c4deadb155f6a6c","unresolved":false,"context_lines":[{"line_number":238,"context_line":"  instead implement it using Kubernetes primitives in a job at the top"},{"line_number":239,"context_line":"  of the buildset.  Add a facility that would allow the user to tell"},{"line_number":240,"context_line":"  Zuul to keep the resources used by that job (i.e., the registry"},{"line_number":241,"context_line":"  service) continually running until the end of the buildset."},{"line_number":242,"context_line":""},{"line_number":243,"context_line":"In order to support this, we may need to implement provider affinity"},{"line_number":244,"context_line":"for builds in a buildset in Nodepool so that we don\u0027t have to deal"}],"source_content_type":"text/x-rst","patch_set":1,"id":"bf659307_100a0d78","line":241,"in_reply_to":"bf659307_8ea7e850","updated":"2018-04-12 09:19:44.000000000","message":"I think clarifying the building (and where it is built) vs the running vs the interacting/subsequent testing would be helpful (either in this spec or supplementary specs)","commit_id":"676f09796afbb1cc5949978427af038352935943"},{"author":{"_account_id":9311,"name":"Tristan Cacqueray","email":"tdecacqu@redhat.com","username":"tristanC"},"change_message_id":"309c2b0b91eec70ff9a56abe9d05b63c7e66db29","unresolved":false,"context_lines":[{"line_number":239,"context_line":"  of the buildset.  Add a facility that would allow the user to tell"},{"line_number":240,"context_line":"  Zuul to keep the resources used by that job (i.e., the registry"},{"line_number":241,"context_line":"  service) continually running until the end of the buildset."},{"line_number":242,"context_line":""},{"line_number":243,"context_line":"In order to support this, we may need to implement provider affinity"},{"line_number":244,"context_line":"for builds in a buildset in Nodepool so that we don\u0027t have to deal"},{"line_number":245,"context_line":"with ingress access to the registry (which may not be possible)."}],"source_content_type":"text/x-rst","patch_set":1,"id":"bf659307_f40f6685","line":242,"updated":"2018-04-11 14:49:31.000000000","message":"IIRC there was discussion about an intermediate job state (e.g. \"CONTINUE\") that a parent job could use to indicate the scheduler that child job can start while keeping the parent job running. This would let a \"build-image\" job host a registry that will be alive until all child job are finished.\n\nThis would be a handy generic alternative.","commit_id":"676f09796afbb1cc5949978427af038352935943"},{"author":{"_account_id":1,"name":"James E. Blair","email":"jim@acmegating.com","username":"corvus"},"change_message_id":"21335f4b8675bcb0a44fce64e7664acc99fb97c5","unresolved":false,"context_lines":[{"line_number":78,"context_line":"We can address both cases, but they will be handled a bit differently."},{"line_number":79,"context_line":"First, the case where the container does run SSHD:"},{"line_number":80,"context_line":""},{"line_number":81,"context_line":"Nodepool would need to create a Kubernetes service for the container."},{"line_number":82,"context_line":"If Nodepool and the Zuul executor are running in the same Kubernetes"},{"line_number":83,"context_line":"cluster, the container will be accessible to them, so Nodepool can"},{"line_number":84,"context_line":"return this information to Zuul and the service address can be added"}],"source_content_type":"text/x-rst","patch_set":2,"id":"5f7c97a3_7791ff45","line":81,"updated":"2018-07-05 22:32:24.000000000","message":"I think k8s-container-like-machine is still a sufficiently useful case to support that it\u0027s worth implementing and talking about here.  If I wanted to build a Zuul system that ran a bunch of simple test jobs in containers for efficiency, that\u0027s probably how I\u0027d do it today.\n\nI think that we\u0027ve hit the right balance here now with indicating that this spec is about targeting k8s, but many of the general concepts should apply elsewhere as well.\n\nI\u0027ve made one more revision in PS3 to indicate that it is Nodepool that tells Zuul to use the kubectl connection plugin for Ansible -- so by no means are we limited to using k8s for container-like-machine.  If Nodepool tells Zuul that it provided a label using some other system, that will be fine.  I hope that addresses your concern.","commit_id":"c1b8dacf43541a4a7fb09c8cb22c2e1827858f99"},{"author":{"_account_id":4146,"name":"Clark Boylan","email":"cboylan@sapwetik.org","username":"cboylan"},"change_message_id":"710c80978f36ef3d269b73610052ca26a6b86120","unresolved":false,"context_lines":[{"line_number":78,"context_line":"We can address both cases, but they will be handled a bit differently."},{"line_number":79,"context_line":"First, the case where the container does run SSHD:"},{"line_number":80,"context_line":""},{"line_number":81,"context_line":"Nodepool would need to create a Kubernetes service for the container."},{"line_number":82,"context_line":"If Nodepool and the Zuul executor are running in the same Kubernetes"},{"line_number":83,"context_line":"cluster, the container will be accessible to them, so Nodepool can"},{"line_number":84,"context_line":"return this information to Zuul and the service address can be added"}],"source_content_type":"text/x-rst","patch_set":2,"id":"5f7c97a3_08faa5bb","line":81,"range":{"start_line":81,"start_character":0,"end_line":81,"end_character":69},"updated":"2018-06-01 19:04:09.000000000","message":"I think this may have been the source of confusion on my part regarding \"machine like containers\". From my perspective you are most likely to interact with them without a kubernetes. LXD/LXC for example.\n\nMaybe we want to scope this spec to k8s support in nodepool/zuul and worry about other thingpool items including lxc/lxd and other machine like containers separately?\n\nMy previous concerns were that we were assuming k8s was the only way to get a machine like container hence the reliance on k8s specific access methods. I think that concern persists unless we want to scope this specifically to k8s rather than the bigger \"containers\".","commit_id":"c1b8dacf43541a4a7fb09c8cb22c2e1827858f99"},{"author":{"_account_id":1,"name":"James E. Blair","email":"jim@acmegating.com","username":"corvus"},"change_message_id":"21335f4b8675bcb0a44fce64e7664acc99fb97c5","unresolved":false,"context_lines":[{"line_number":111,"context_line":"If we want streaming output from a kubectl command, we may need to"},{"line_number":112,"context_line":"create a local fork of the kubectl connection plugin in order to"},{"line_number":113,"context_line":"connect it to the log streamer (much in the way we do for the command"},{"line_number":114,"context_line":"module)."},{"line_number":115,"context_line":""},{"line_number":116,"context_line":"Not all jobs will be expected to work in containers.  Some frequently"},{"line_number":117,"context_line":"used Ansible modules will not behave as expected when run with the"}],"source_content_type":"text/x-rst","patch_set":2,"id":"5f7c97a3_17920b4b","line":114,"updated":"2018-07-05 22:32:24.000000000","message":"Neat!","commit_id":"c1b8dacf43541a4a7fb09c8cb22c2e1827858f99"},{"author":{"_account_id":9311,"name":"Tristan Cacqueray","email":"tdecacqu@redhat.com","username":"tristanC"},"change_message_id":"1e4d1a6c1d3e27b4c65f6ec98baf16248735c4d4","unresolved":false,"context_lines":[{"line_number":111,"context_line":"If we want streaming output from a kubectl command, we may need to"},{"line_number":112,"context_line":"create a local fork of the kubectl connection plugin in order to"},{"line_number":113,"context_line":"connect it to the log streamer (much in the way we do for the command"},{"line_number":114,"context_line":"module)."},{"line_number":115,"context_line":""},{"line_number":116,"context_line":"Not all jobs will be expected to work in containers.  Some frequently"},{"line_number":117,"context_line":"used Ansible modules will not behave as expected when run with the"}],"source_content_type":"text/x-rst","patch_set":2,"id":"5f7c97a3_259eac53","line":114,"updated":"2018-06-14 04:33:53.000000000","message":"Turns out this is already working. Commands\u0027 logs do contain some initial \"waiting for logger\" but then the output is properly logged. Though if/when zuul_stream uses ssh port forward, then this may need some adjustments to work with the kubectl connection plugin.","commit_id":"c1b8dacf43541a4a7fb09c8cb22c2e1827858f99"},{"author":{"_account_id":9311,"name":"Tristan Cacqueray","email":"tdecacqu@redhat.com","username":"tristanC"},"change_message_id":"1e4d1a6c1d3e27b4c65f6ec98baf16248735c4d4","unresolved":false,"context_lines":[{"line_number":117,"context_line":"used Ansible modules will not behave as expected when run with the"},{"line_number":118,"context_line":"kubectl connection plugin.  Synchronize, in particular, may be"},{"line_number":119,"context_line":"problematic (though as there is support for synchronize with docker,"},{"line_number":120,"context_line":"that may be possible to overcome).  We will want to think about"},{"line_number":121,"context_line":"ways to keep the base job(s) as flexible as possible so they work with"},{"line_number":122,"context_line":"multiple connection types, but there may be limits.  Containers which"},{"line_number":123,"context_line":"run SSHD should not have these problems."}],"source_content_type":"text/x-rst","patch_set":2,"id":"5f7c97a3_e5a3d499","line":120,"updated":"2018-06-14 04:33:53.000000000","message":"At least oc has a synchronize command that will try rsync or tar usage over the k8s exec interface. So I think the synchronize module could be easily fixed to work with the kubectl connection plugin.","commit_id":"c1b8dacf43541a4a7fb09c8cb22c2e1827858f99"},{"author":{"_account_id":1,"name":"James E. Blair","email":"jim@acmegating.com","username":"corvus"},"change_message_id":"21335f4b8675bcb0a44fce64e7664acc99fb97c5","unresolved":false,"context_lines":[{"line_number":117,"context_line":"used Ansible modules will not behave as expected when run with the"},{"line_number":118,"context_line":"kubectl connection plugin.  Synchronize, in particular, may be"},{"line_number":119,"context_line":"problematic (though as there is support for synchronize with docker,"},{"line_number":120,"context_line":"that may be possible to overcome).  We will want to think about"},{"line_number":121,"context_line":"ways to keep the base job(s) as flexible as possible so they work with"},{"line_number":122,"context_line":"multiple connection types, but there may be limits.  Containers which"},{"line_number":123,"context_line":"run SSHD should not have these problems."}],"source_content_type":"text/x-rst","patch_set":2,"id":"5f7c97a3_3797073c","line":120,"updated":"2018-07-05 22:32:24.000000000","message":"Double neat!","commit_id":"c1b8dacf43541a4a7fb09c8cb22c2e1827858f99"},{"author":{"_account_id":1,"name":"James E. Blair","email":"jim@acmegating.com","username":"corvus"},"change_message_id":"21335f4b8675bcb0a44fce64e7664acc99fb97c5","unresolved":false,"context_lines":[{"line_number":179,"context_line":""},{"line_number":180,"context_line":"OpenShift provides some features that make this easier, so an"},{"line_number":181,"context_line":"OpenShift-specific driver could additonally do the following and"},{"line_number":182,"context_line":"reduce the complexity in the job:"},{"line_number":183,"context_line":""},{"line_number":184,"context_line":"We can ask Nodepool to create an `OpenShift project`_ for the use of"},{"line_number":185,"context_line":"the job.  That will create a private image repository for the project."}],"source_content_type":"text/x-rst","patch_set":2,"id":"5f7c97a3_d79fb34e","line":182,"updated":"2018-07-05 22:32:24.000000000","message":"It is to support k8s first, but the additional OpenShift functionality sounds very compelling and worth looking into.\n\nI worry about scope creep here -- there must be other applications which manage k8s clusters to scale with load.","commit_id":"c1b8dacf43541a4a7fb09c8cb22c2e1827858f99"},{"author":{"_account_id":1921,"name":"Andrea Frittoli","email":"andrea.frittoli@gmail.com","username":"andrea-frittoli"},"change_message_id":"000e5f91ec4e5600948827ab75ef7c290ec7d665","unresolved":false,"context_lines":[{"line_number":177,"context_line":"course, it will be useful to create reusable roles and jobs in"},{"line_number":178,"context_line":"zuul-jobs to implement this universally."},{"line_number":179,"context_line":""},{"line_number":180,"context_line":"OpenShift provides some features that make this easier, so an"},{"line_number":181,"context_line":"OpenShift-specific driver could additonally do the following and"},{"line_number":182,"context_line":"reduce the complexity in the job:"},{"line_number":183,"context_line":""},{"line_number":184,"context_line":"We can ask Nodepool to create an `OpenShift project`_ for the use of"},{"line_number":185,"context_line":"the job.  That will create a private image repository for the project."}],"source_content_type":"text/x-rst","patch_set":2,"id":"5f7c97a3_31b452f2","line":182,"range":{"start_line":180,"start_character":0,"end_line":182,"end_character":33},"updated":"2018-06-06 21:17:33.000000000","message":"This sounds great, as long as the priority is to provide standard k8s API - since resource providers may support k8s but not OpenShift.\n\nI wonder if it makes sense for node pool to expose a plugin interface that different cloud providers may implement to let nodepool even manage k8s clusters (add worker nodes, or even create / delete clusters).","commit_id":"c1b8dacf43541a4a7fb09c8cb22c2e1827858f99"},{"author":{"_account_id":1921,"name":"Andrea Frittoli","email":"andrea.frittoli@gmail.com","username":"andrea-frittoli"},"change_message_id":"000e5f91ec4e5600948827ab75ef7c290ec7d665","unresolved":false,"context_lines":[{"line_number":283,"context_line":"created a service token, that could be passed to the child jobs for"},{"line_number":284,"context_line":"their use when they start their own containers or pods."},{"line_number":285,"context_line":""},{"line_number":286,"context_line":"In order to support this, we may need to implement provider affinity"},{"line_number":287,"context_line":"for builds in a buildset in Nodepool so that we don\u0027t have to deal"},{"line_number":288,"context_line":"with ingress access to the registry (which may not be possible)."},{"line_number":289,"context_line":"Otherwise if a Nodepool had access to two Kubernetes clusters, we"},{"line_number":290,"context_line":"might assign a child job to a different cluster."}],"source_content_type":"text/x-rst","patch_set":2,"id":"5f7c97a3_118ef631","line":290,"range":{"start_line":286,"start_character":0,"end_line":290,"end_character":48},"updated":"2018-06-06 21:17:33.000000000","message":"+1. I think it\u0027s definitely worth trying to use the in-cluster capabilities.","commit_id":"c1b8dacf43541a4a7fb09c8cb22c2e1827858f99"},{"author":{"_account_id":27900,"name":"Artem Goncharov","email":"artem.goncharov@gmail.com","username":"gtema"},"change_message_id":"7ac6a28b2921435177d27aba5c6f5cad24604f33","unresolved":false,"context_lines":[{"line_number":64,"context_line":"downstream of the native `kubernetes python client`_, so they are"},{"line_number":65,"context_line":"compatible, but using the openshift client offers a superset of"},{"line_number":66,"context_line":"functionality, which may come in handy later.  Since the Ansible"},{"line_number":67,"context_line":"k8s_raw module uses the openshift client for this reason, we may want"},{"line_number":68,"context_line":"to as well."},{"line_number":69,"context_line":""},{"line_number":70,"context_line":"In kubernetes, a group of related containers forms a pod, which is the"}],"source_content_type":"text/x-rst","patch_set":4,"id":"5f7c97a3_26da7278","line":67,"range":{"start_line":67,"start_character":0,"end_line":67,"end_character":7},"updated":"2018-07-06 07:27:25.000000000","message":"use of `k8s_raw` is deprecated in favor of `k8s` (ansible 2.6)","commit_id":"1b2b1d0b8c34774df3012f7390311021c59b3642"},{"author":{"_account_id":1,"name":"James E. Blair","email":"jim@acmegating.com","username":"corvus"},"change_message_id":"2760d9efff45b1ea4c161749b702af4e43c2291a","unresolved":false,"context_lines":[{"line_number":73,"context_line":"single container is created.  Therefore, for this case, Nodepool will"},{"line_number":74,"context_line":"need to create a pod with a container.  Some aspects of the"},{"line_number":75,"context_line":"container\u0027s configuration (such as the image which is used) will need"},{"line_number":76,"context_line":"to be defined in the Nodepool configuration for the label.  These"},{"line_number":77,"context_line":"configuration values should be supplied in a manner typical of"},{"line_number":78,"context_line":"Nodepool\u0027s configuration format.  Note that very little customization"},{"line_number":79,"context_line":"is expected here -- more complex topologies should not be supported by"}],"source_content_type":"text/x-rst","patch_set":4,"id":"5f7c97a3_78cf4728","line":76,"updated":"2018-07-23 15:34:24.000000000","message":"Monty looked into this and we discussed it out-of-band.  The process described in (1) is not feasible with kubernetes, but we did come up with a plan for a git mirror system which could reduce or eliminate the need for building custom container images with in-place git repo caches.","commit_id":"1b2b1d0b8c34774df3012f7390311021c59b3642"},{"author":{"_account_id":1,"name":"James E. Blair","email":"jim@acmegating.com","username":"corvus"},"change_message_id":"37b33521aff18e454c9d55b9a37573bae06b67f2","unresolved":false,"context_lines":[{"line_number":73,"context_line":"single container is created.  Therefore, for this case, Nodepool will"},{"line_number":74,"context_line":"need to create a pod with a container.  Some aspects of the"},{"line_number":75,"context_line":"container\u0027s configuration (such as the image which is used) will need"},{"line_number":76,"context_line":"to be defined in the Nodepool configuration for the label.  These"},{"line_number":77,"context_line":"configuration values should be supplied in a manner typical of"},{"line_number":78,"context_line":"Nodepool\u0027s configuration format.  Note that very little customization"},{"line_number":79,"context_line":"is expected here -- more complex topologies should not be supported by"}],"source_content_type":"text/x-rst","patch_set":4,"id":"5f7c97a3_9a812dfd","line":76,"updated":"2018-07-09 20:56:37.000000000","message":"Okay, we\u0027ve got two suggestions here:\n\n1) Possibly using a two-container pod with the second container only holding the cached repos so that the image for the first container can be more vanilla.  Let\u0027s look into whether that would work, because if so, that seems like a better approach rather than building new images for each upstream image just to add the local cache.\n\n2) Specifying containers from Zuul.  If we agree (and I think we do) that there\u0027s still a place for nodepool providing a label-to-image mapping for containers (so that a local admin can provide their own local container images, or add certain local information (possibly the thing in #1 even), or restrict access to only defined container images, then I think the spec as written stands.  I think the suggestion to dynamically request container images from Zuul is a good one, but it opens some questions about implementation, usage, and access control which we should think about carefully and may distract us from the immediate goal.  So I\u0027d like to consider that separately later.\n\nHow\u0027s that sound?","commit_id":"1b2b1d0b8c34774df3012f7390311021c59b3642"},{"author":{"_account_id":1,"name":"James E. Blair","email":"jim@acmegating.com","username":"corvus"},"change_message_id":"dfeb05be7424e3031526774260eeea9fdd8998f2","unresolved":false,"context_lines":[{"line_number":73,"context_line":"single container is created.  Therefore, for this case, Nodepool will"},{"line_number":74,"context_line":"need to create a pod with a container.  Some aspects of the"},{"line_number":75,"context_line":"container\u0027s configuration (such as the image which is used) will need"},{"line_number":76,"context_line":"to be defined in the Nodepool configuration for the label.  These"},{"line_number":77,"context_line":"configuration values should be supplied in a manner typical of"},{"line_number":78,"context_line":"Nodepool\u0027s configuration format.  Note that very little customization"},{"line_number":79,"context_line":"is expected here -- more complex topologies should not be supported by"}],"source_content_type":"text/x-rst","patch_set":4,"id":"5f7c97a3_fd9f0462","line":76,"updated":"2018-07-09 15:00:46.000000000","message":"The two are not so different.  It is anticipated that nodepool may build container images as well, for the same reasons that we build VM images: to cache git repos and ensure OS packages are present.  See line 243.\n\nFor the container-as-machine case, I think we should maintain parity between the two systems -- the difference is not really meant to be user-visible.\n\nThis idea is a good one, however.  Perhaps we should discuss it further, but we should also discuss applying the same thing to cloud images (I can see no argument for this for container images that doesn\u0027t also apply to cloud images), as well as how to restrict it (it\u0027s certainly valid for an admin to want to restrict what container images are available to jobs).","commit_id":"1b2b1d0b8c34774df3012f7390311021c59b3642"},{"author":{"_account_id":2,"name":"Monty Taylor","email":"mordred@inaugust.com","username":"mordred"},"change_message_id":"08b4207bbc3aa5357feb96d57e540599acb8618f","unresolved":false,"context_lines":[{"line_number":73,"context_line":"single container is created.  Therefore, for this case, Nodepool will"},{"line_number":74,"context_line":"need to create a pod with a container.  Some aspects of the"},{"line_number":75,"context_line":"container\u0027s configuration (such as the image which is used) will need"},{"line_number":76,"context_line":"to be defined in the Nodepool configuration for the label.  These"},{"line_number":77,"context_line":"configuration values should be supplied in a manner typical of"},{"line_number":78,"context_line":"Nodepool\u0027s configuration format.  Note that very little customization"},{"line_number":79,"context_line":"is expected here -- more complex topologies should not be supported by"}],"source_content_type":"text/x-rst","patch_set":4,"id":"5f7c97a3_873d5f2d","line":76,"range":{"start_line":76,"start_character":0,"end_line":76,"end_character":57},"updated":"2018-07-09 13:59:16.000000000","message":"container images and vm images have a different costs and no consistent naming scheme. For vm images, we define how to build and upload them for the nodepool builders, then we define nodepool labels that abstract away the details of the actual cloud image name, allowing use to have a label like \u0027ubuntu-xenial\u0027 and an actual image on each cloud with a much more complex name.\n\nWith containers, it feels like requiring a nodepool admin to create (or review/approve) a label for a container image in the nodepool config is admin work that doesn\u0027t provide any value. It also doesn\u0027t allow for experimentation, since the nodepool config is not speculative, so in order to write a zuul job that uses the node:slim image, I\u0027d need to go make a label in nodepool.yaml, get it landed, then make a zuul job that uses it. Then, if it turns out I needed the node image or the node:alpine image - or the node:slim-8.5 image, I need to go make nodepool labels for each of them. (incidentally, for openstack, I\u0027d need to, I believe, make those labels for every cloud we have, assuming we\u0027re running a k8s in every cloud region) \n\nWhat about adding another field to the nodeset construct that is mutually exclusive with label that is something like:\n\n  - nodeset:\n      nodes:\n        - name: contoller\n          container: python:3.6\n\nthat can pull and boot any container image by reference?\n\nThen, in the nodepool config, for a given provider - or maybe a pool - we could add config information indicating that the provider/pool has the capabilities of providing containers, so when a request for container: python:3.6 comes in, the nodepool scheduler can check to see which provider can boot a container at all, then just boot the pod with the appropriate things.","commit_id":"1b2b1d0b8c34774df3012f7390311021c59b3642"},{"author":{"_account_id":2,"name":"Monty Taylor","email":"mordred@inaugust.com","username":"mordred"},"change_message_id":"a24d5f79f51b425ada4aa65dc4fff766717a6678","unresolved":false,"context_lines":[{"line_number":73,"context_line":"single container is created.  Therefore, for this case, Nodepool will"},{"line_number":74,"context_line":"need to create a pod with a container.  Some aspects of the"},{"line_number":75,"context_line":"container\u0027s configuration (such as the image which is used) will need"},{"line_number":76,"context_line":"to be defined in the Nodepool configuration for the label.  These"},{"line_number":77,"context_line":"configuration values should be supplied in a manner typical of"},{"line_number":78,"context_line":"Nodepool\u0027s configuration format.  Note that very little customization"},{"line_number":79,"context_line":"is expected here -- more complex topologies should not be supported by"}],"source_content_type":"text/x-rst","patch_set":4,"id":"5f7c97a3_7d6bf41d","line":76,"in_reply_to":"5f7c97a3_fd9f0462","updated":"2018-07-09 15:11:41.000000000","message":"I\u0027m going to ignore the cached resources part first for a sec. I\u0027ll come back to it, I promise.\n\nThe reason I think that container images are different from vm images is that there is currently a defacto global registry of  container names that can be used from anywhere that are not cloud-vendor specific. The newer technologies that aim to be more decentralized still allow image references to be urls - so a container image name can be specified by a user and the vendor of a cloud doesn\u0027t have an opportunity to mess things up.\n\nVM images, otoh, are different on every cloud. There is no \"ubuntu:xenial\" image that can be counted on to be ubuntu:xenial - so our vm image labels provide value in that they allow us to describe a generic concept and map it to specific image implementation names. This is both for hiding different names for pre-existing cloud provider images, as well as our serial number named images produced by nodepool-builder.\n\nBack to cached resources ...\n\nSince we\u0027re talking about always booting a pod-of-one-container, to deal with caching without needing to rebuild container images (or without having to build derivitive images) - we could alternately make a container image with the cached git repos in it (and other cached content) and have nodepool be configured to boot a pod with the requested image as well as the repo cache image with directories from the cache container bind-mounted in to the requested image as volumes. Since containers in pods can share a filesystem, this is a way we could provide the same functionality we provide to vm images.\n\nIt would also be much more efficient - since we\u0027d otherwise be laying the caching on TOP of the requested base image - which means if we made 5 different container images with added cache we\u0027d have 5 complete copies of the cache layer in the docker image cache. With a cache image that\u0027s just there to provide directories, the cache layer can be in the image layer cache once on each k8s node, and then the requested container image itself can be used over and over again unmodified.","commit_id":"1b2b1d0b8c34774df3012f7390311021c59b3642"},{"author":{"_account_id":9311,"name":"Tristan Cacqueray","email":"tdecacqu@redhat.com","username":"tristanC"},"change_message_id":"ee378cf35287d980359bdf101b21f32a216bbd46","unresolved":false,"context_lines":[{"line_number":75,"context_line":"container\u0027s configuration (such as the image which is used) will need"},{"line_number":76,"context_line":"to be defined in the Nodepool configuration for the label.  These"},{"line_number":77,"context_line":"configuration values should be supplied in a manner typical of"},{"line_number":78,"context_line":"Nodepool\u0027s configuration format.  Note that very little customization"},{"line_number":79,"context_line":"is expected here -- more complex topologies should not be supported by"},{"line_number":80,"context_line":"this mechanism and should be left instead for :ref:`container-native`."},{"line_number":81,"context_line":""},{"line_number":82,"context_line":"Containers in Kubernetes always run a single command, and when that"}],"source_content_type":"text/x-rst","patch_set":4,"id":"5f7c97a3_17e3eba1","line":79,"range":{"start_line":78,"start_character":34,"end_line":79,"end_character":16},"updated":"2018-07-05 22:48:34.000000000","message":"How are zuul-executors supposed to authenticate with the cluster?","commit_id":"1b2b1d0b8c34774df3012f7390311021c59b3642"},{"author":{"_account_id":1,"name":"James E. Blair","email":"jim@acmegating.com","username":"corvus"},"change_message_id":"37b33521aff18e454c9d55b9a37573bae06b67f2","unresolved":false,"context_lines":[{"line_number":76,"context_line":"to be defined in the Nodepool configuration for the label.  These"},{"line_number":77,"context_line":"configuration values should be supplied in a manner typical of"},{"line_number":78,"context_line":"Nodepool\u0027s configuration format.  Note that very little customization"},{"line_number":79,"context_line":"is expected here -- more complex topologies should not be supported by"},{"line_number":80,"context_line":"this mechanism and should be left instead for :ref:`container-native`."},{"line_number":81,"context_line":""},{"line_number":82,"context_line":"Containers in Kubernetes always run a single command, and when that"}],"source_content_type":"text/x-rst","patch_set":4,"id":"5f7c97a3_3a7c991e","line":79,"updated":"2018-07-09 20:56:37.000000000","message":"Yes, the executors will need some credentials.  Nodepool will provide the credentials to the executors via zookeeper (much in the way it provides other connection information now, such as the connection plugin and ip address (and if we were designing it from scratch now, we might even have it provide the ssh key; we still may in the future)).  I expect that the best way to do that would be for nodepool to create a service account and pass the resulting token to zuul via zookeeper.  I could add this to the spec if you like, or if you have another suggestion, I could incorporate that.  Though I\u0027m happy to leave some flexibility here for implementation -- if we communicate the idea that nodepool should provide whatever zuul needs to run kubectl, I\u0027m happy.\n\nBy \"very little customization\", I mean that the nodepool configuration language is not designed to replace or re-implement the k8s api.  If we find ourselves adding more than the minimum of information needed to bring up a usable image to run a pep8 job to the nodepool config, then we should rethink what we\u0027re doing.","commit_id":"1b2b1d0b8c34774df3012f7390311021c59b3642"},{"author":{"_account_id":2,"name":"Monty Taylor","email":"mordred@inaugust.com","username":"mordred"},"change_message_id":"08b4207bbc3aa5357feb96d57e540599acb8618f","unresolved":false,"context_lines":[{"line_number":75,"context_line":"container\u0027s configuration (such as the image which is used) will need"},{"line_number":76,"context_line":"to be defined in the Nodepool configuration for the label.  These"},{"line_number":77,"context_line":"configuration values should be supplied in a manner typical of"},{"line_number":78,"context_line":"Nodepool\u0027s configuration format.  Note that very little customization"},{"line_number":79,"context_line":"is expected here -- more complex topologies should not be supported by"},{"line_number":80,"context_line":"this mechanism and should be left instead for :ref:`container-native`."},{"line_number":81,"context_line":""},{"line_number":82,"context_line":"Containers in Kubernetes always run a single command, and when that"}],"source_content_type":"text/x-rst","patch_set":4,"id":"5f7c97a3_27d56b1b","line":79,"range":{"start_line":78,"start_character":34,"end_line":79,"end_character":16},"in_reply_to":"5f7c97a3_17e3eba1","updated":"2018-07-09 13:59:16.000000000","message":"I do not believe they are supposed to authenticate with the cluster in this context- I believe this is supposed to be nodepool doing the k8s/openshift api calls, creating the pod and handing it to the executors in the ansible inventory.","commit_id":"1b2b1d0b8c34774df3012f7390311021c59b3642"},{"author":{"_account_id":9311,"name":"Tristan Cacqueray","email":"tdecacqu@redhat.com","username":"tristanC"},"change_message_id":"866ebeec70c407f588d5f7710628b697814bd5c5","unresolved":false,"context_lines":[{"line_number":75,"context_line":"container\u0027s configuration (such as the image which is used) will need"},{"line_number":76,"context_line":"to be defined in the Nodepool configuration for the label.  These"},{"line_number":77,"context_line":"configuration values should be supplied in a manner typical of"},{"line_number":78,"context_line":"Nodepool\u0027s configuration format.  Note that very little customization"},{"line_number":79,"context_line":"is expected here -- more complex topologies should not be supported by"},{"line_number":80,"context_line":"this mechanism and should be left instead for :ref:`container-native`."},{"line_number":81,"context_line":""},{"line_number":82,"context_line":"Containers in Kubernetes always run a single command, and when that"}],"source_content_type":"text/x-rst","patch_set":4,"id":"5f7c97a3_7d021440","line":79,"range":{"start_line":78,"start_character":34,"end_line":79,"end_character":16},"in_reply_to":"5f7c97a3_27d56b1b","updated":"2018-07-09 15:16:17.000000000","message":"Could you define \"handling it to the executors\"? It seems like the ansible-playbook runners needs a .kube/config or kubectl_cert_file or  kubectl_token or *something* to be able to use kubectl exec and run tasks on any inventory pods. Would be nice to know more about this and how it qualifies as \"little customization expected here\" :)","commit_id":"1b2b1d0b8c34774df3012f7390311021c59b3642"},{"author":{"_account_id":9311,"name":"Tristan Cacqueray","email":"tdecacqu@redhat.com","username":"tristanC"},"change_message_id":"535ce60da991a7bd988a7978a9cf660f684d0664","unresolved":false,"context_lines":[{"line_number":76,"context_line":"to be defined in the Nodepool configuration for the label.  These"},{"line_number":77,"context_line":"configuration values should be supplied in a manner typical of"},{"line_number":78,"context_line":"Nodepool\u0027s configuration format.  Note that very little customization"},{"line_number":79,"context_line":"is expected here -- more complex topologies should not be supported by"},{"line_number":80,"context_line":"this mechanism and should be left instead for :ref:`container-native`."},{"line_number":81,"context_line":""},{"line_number":82,"context_line":"Containers in Kubernetes always run a single command, and when that"}],"source_content_type":"text/x-rst","patch_set":4,"id":"5f7c97a3_934b535d","line":79,"in_reply_to":"5f7c97a3_3a7c991e","updated":"2018-07-10 05:27:14.000000000","message":"Having nodepool create the zuul service account and pass the token through the Node object lgtm. I think it makes sense to mention it in this spec.\n\nThis doesn\u0027t seem to require any customization of the nodepool configuration, though we need to adapt the zuul-executor to decode the token from the node object and install it in the bubblewrap. Would be ideal to have this logic works for ssh keys too.","commit_id":"1b2b1d0b8c34774df3012f7390311021c59b3642"},{"author":{"_account_id":2,"name":"Monty Taylor","email":"mordred@inaugust.com","username":"mordred"},"change_message_id":"08b4207bbc3aa5357feb96d57e540599acb8618f","unresolved":false,"context_lines":[{"line_number":92,"context_line":"that a service be configured for the container along with ingress"},{"line_number":93,"context_line":"access to the cluster.  This is an additional complication that some"},{"line_number":94,"context_line":"users may not want to undertake, especially if the goal is to run a"},{"line_number":95,"context_line":"relatively simple job.  On the other hand, some environments may be"},{"line_number":96,"context_line":"more natually suited to using an SSH connection."},{"line_number":97,"context_line":""},{"line_number":98,"context_line":"We can address both cases, but they will be handled a bit differently."},{"line_number":99,"context_line":"First, the case where the container does run SSHD:"}],"source_content_type":"text/x-rst","patch_set":4,"id":"5f7c97a3_27c9eb23","line":96,"range":{"start_line":95,"start_character":24,"end_line":96,"end_character":48},"updated":"2018-07-09 13:59:16.000000000","message":"Related to my earlier comment, the \u0027this has ssh\u0027 vs. \u0027this uses kubectl\u0027 seems like it would need to be a config flag somewhere. (whether it\u0027s in a label definition, or in a nodeset definition)\n\nHowever, have we identitied environments where people would want to use k8s/containers as build resources that specifically want to ssh into the containers rather than using the inventory/kubectl approach you list below? As you mention, it\u0027s rather complicated - and I\u0027d hate to take on the complexity if it\u0027s not really a thing anyone wants.","commit_id":"1b2b1d0b8c34774df3012f7390311021c59b3642"},{"author":{"_account_id":1,"name":"James E. Blair","email":"jim@acmegating.com","username":"corvus"},"change_message_id":"dfeb05be7424e3031526774260eeea9fdd8998f2","unresolved":false,"context_lines":[{"line_number":93,"context_line":"access to the cluster.  This is an additional complication that some"},{"line_number":94,"context_line":"users may not want to undertake, especially if the goal is to run a"},{"line_number":95,"context_line":"relatively simple job.  On the other hand, some environments may be"},{"line_number":96,"context_line":"more natually suited to using an SSH connection."},{"line_number":97,"context_line":""},{"line_number":98,"context_line":"We can address both cases, but they will be handled a bit differently."},{"line_number":99,"context_line":"First, the case where the container does run SSHD:"}],"source_content_type":"text/x-rst","patch_set":4,"id":"5f7c97a3_1da3e0b2","line":96,"updated":"2018-07-09 15:00:46.000000000","message":"Yes, it would be a flag in nodepool (either a label or, if nodepool builds the image, an image flag).\nLines 109-111 say we\u0027re not going to do this right now.  This is just collecting the information we\u0027ve generated as part of this process for future work if someone wants to take it on, as well as providing contect for why we\u0027re doing the other thing (this was, after all, one of the most important questions we set out to answer about how to do this).","commit_id":"1b2b1d0b8c34774df3012f7390311021c59b3642"},{"author":{"_account_id":2,"name":"Monty Taylor","email":"mordred@inaugust.com","username":"mordred"},"change_message_id":"a24d5f79f51b425ada4aa65dc4fff766717a6678","unresolved":false,"context_lines":[{"line_number":93,"context_line":"access to the cluster.  This is an additional complication that some"},{"line_number":94,"context_line":"users may not want to undertake, especially if the goal is to run a"},{"line_number":95,"context_line":"relatively simple job.  On the other hand, some environments may be"},{"line_number":96,"context_line":"more natually suited to using an SSH connection."},{"line_number":97,"context_line":""},{"line_number":98,"context_line":"We can address both cases, but they will be handled a bit differently."},{"line_number":99,"context_line":"First, the case where the container does run SSHD:"}],"source_content_type":"text/x-rst","patch_set":4,"id":"5f7c97a3_9d4ff0bc","line":96,"in_reply_to":"5f7c97a3_1da3e0b2","updated":"2018-07-09 15:11:41.000000000","message":"++","commit_id":"1b2b1d0b8c34774df3012f7390311021c59b3642"},{"author":{"_account_id":2,"name":"Monty Taylor","email":"mordred@inaugust.com","username":"mordred"},"change_message_id":"08b4207bbc3aa5357feb96d57e540599acb8618f","unresolved":false,"context_lines":[{"line_number":106,"context_line":"the kubernetes cluster is external to Nodepool and Zuul, Nodepool will"},{"line_number":107,"context_line":"also need to establish an ingress resource in order to make it"},{"line_number":108,"context_line":"externally accessible.  Both of these will require additional Nodepool"},{"line_number":109,"context_line":"configuration and code to implement.  Due to the additional"},{"line_number":110,"context_line":"complexity, these should be implemented as follow-on changes after the"},{"line_number":111,"context_line":"simpler case where SSHD is not running in the container."},{"line_number":112,"context_line":""},{"line_number":113,"context_line":"In the case where the container does not run SSHD, and we interact"},{"line_number":114,"context_line":"with it via native commands, Nodepool will inform Zuul that the"}],"source_content_type":"text/x-rst","patch_set":4,"id":"5f7c97a3_e7ce932a","line":111,"range":{"start_line":109,"start_character":38,"end_line":111,"end_character":56},"updated":"2018-07-09 13:59:16.000000000","message":"++","commit_id":"1b2b1d0b8c34774df3012f7390311021c59b3642"},{"author":{"_account_id":27900,"name":"Artem Goncharov","email":"artem.goncharov@gmail.com","username":"gtema"},"change_message_id":"7ac6a28b2921435177d27aba5c6f5cad24604f33","unresolved":false,"context_lines":[{"line_number":157,"context_line":"based, and may have any number of containers which may be created and"},{"line_number":158,"context_line":"destroyed in the process of executing the job."},{"line_number":159,"context_line":""},{"line_number":160,"context_line":"It may use the `k8s_raw Ansible module`_ to interact directly with"},{"line_number":161,"context_line":"Kubernetes, creating and destroying pods for the job in much the same"},{"line_number":162,"context_line":"way that an existing job may use Ansible to orchestrate actions on a"},{"line_number":163,"context_line":"worker node."}],"source_content_type":"text/x-rst","patch_set":4,"id":"5f7c97a3_a65d22ed","line":160,"range":{"start_line":160,"start_character":16,"end_line":160,"end_character":38},"updated":"2018-07-06 07:27:25.000000000","message":"as before - k8s, instead of k8s_raw","commit_id":"1b2b1d0b8c34774df3012f7390311021c59b3642"},{"author":{"_account_id":27900,"name":"Artem Goncharov","email":"artem.goncharov@gmail.com","username":"gtema"},"change_message_id":"7ac6a28b2921435177d27aba5c6f5cad24604f33","unresolved":false,"context_lines":[{"line_number":178,"context_line":""},{"line_number":179,"context_line":"Within a single job, we could build images by requesting either a full"},{"line_number":180,"context_line":"machine or a :ref:`container-machine` from Nodepool and running the"},{"line_number":181,"context_line":"image build on that machine.  Or we could use the `k8s_raw Ansible"},{"line_number":182,"context_line":"module`_ to create that container from within the job.  We would use the"},{"line_number":183,"context_line":":ref:`git-repo-sync` process to get the appropriate source code onto"},{"line_number":184,"context_line":"the builder.  Regardless, once the image builds are complete, we can"}],"source_content_type":"text/x-rst","patch_set":4,"id":"5f7c97a3_066416b2","line":181,"range":{"start_line":181,"start_character":51,"end_line":181,"end_character":58},"updated":"2018-07-06 07:27:25.000000000","message":"k8s","commit_id":"1b2b1d0b8c34774df3012f7390311021c59b3642"},{"author":{"_account_id":27900,"name":"Artem Goncharov","email":"artem.goncharov@gmail.com","username":"gtema"},"change_message_id":"7ac6a28b2921435177d27aba5c6f5cad24604f33","unresolved":false,"context_lines":[{"line_number":210,"context_line":"``imagePullSecrets`` configured to use the private image repository [#f1]_."},{"line_number":211,"context_line":"We can have Zuul use one of the default service accouns, or have"},{"line_number":212,"context_line":"Nodepool create a new one specifically for Zuul, and then when using"},{"line_number":213,"context_line":"the `k8s_raw Ansible module`_, the image registry will automatically be"},{"line_number":214,"context_line":"used."},{"line_number":215,"context_line":""},{"line_number":216,"context_line":"While we may consider expanding the Nodepool API and configuration"}],"source_content_type":"text/x-rst","patch_set":4,"id":"5f7c97a3_461c4e12","line":213,"range":{"start_line":213,"start_character":5,"end_line":213,"end_character":12},"updated":"2018-07-06 07:27:25.000000000","message":"k8s","commit_id":"1b2b1d0b8c34774df3012f7390311021c59b3642"},{"author":{"_account_id":27900,"name":"Artem Goncharov","email":"artem.goncharov@gmail.com","username":"gtema"},"change_message_id":"7ac6a28b2921435177d27aba5c6f5cad24604f33","unresolved":false,"context_lines":[{"line_number":223,"context_line":""},{"line_number":224,"context_line":".. _OpenShift Project: https://docs.openshift.org/latest/dev_guide/projects.html"},{"line_number":225,"context_line":".. [#f1] https://docs.openshift.org/latest/dev_guide/managing_images.html#using-image-pull-secrets"},{"line_number":226,"context_line":".. _k8s_raw Ansible module: http://docs.ansible.com/ansible/2.5/modules/k8s_raw_module.html"},{"line_number":227,"context_line":""},{"line_number":228,"context_line":".. _git-repo-sync:"},{"line_number":229,"context_line":""}],"source_content_type":"text/x-rst","patch_set":4,"id":"5f7c97a3_c6be1ed3","line":226,"updated":"2018-07-06 07:27:25.000000000","message":"https://docs.ansible.com/ansible/latest/modules/k8s_module.html (note, k8s is present only from 2.6, while k8s_raw only in 2.5. https://github.com/openshift/openshift-restclient-python guys have deprecated k8s_raw. Instead `k8s` is able to handle k8s and openshift resources with one module","commit_id":"1b2b1d0b8c34774df3012f7390311021c59b3642"},{"author":{"_account_id":1,"name":"James E. Blair","email":"jim@acmegating.com","username":"corvus"},"change_message_id":"2760d9efff45b1ea4c161749b702af4e43c2291a","unresolved":false,"context_lines":[{"line_number":223,"context_line":""},{"line_number":224,"context_line":".. _OpenShift Project: https://docs.openshift.org/latest/dev_guide/projects.html"},{"line_number":225,"context_line":".. [#f1] https://docs.openshift.org/latest/dev_guide/managing_images.html#using-image-pull-secrets"},{"line_number":226,"context_line":".. _k8s_raw Ansible module: http://docs.ansible.com/ansible/2.5/modules/k8s_raw_module.html"},{"line_number":227,"context_line":""},{"line_number":228,"context_line":".. _git-repo-sync:"},{"line_number":229,"context_line":""}],"source_content_type":"text/x-rst","patch_set":4,"id":"5f7c97a3_18d413bc","line":226,"updated":"2018-07-23 15:34:24.000000000","message":"k8s is aliased to k8s_raw in 2.6; since we\u0027re still on 2.5, I\u0027m going to leave this as written.  But eventually we\u0027ll need to change our use.","commit_id":"1b2b1d0b8c34774df3012f7390311021c59b3642"},{"author":{"_account_id":1,"name":"James E. Blair","email":"jim@acmegating.com","username":"corvus"},"change_message_id":"dfeb05be7424e3031526774260eeea9fdd8998f2","unresolved":false,"context_lines":[{"line_number":223,"context_line":""},{"line_number":224,"context_line":".. _OpenShift Project: https://docs.openshift.org/latest/dev_guide/projects.html"},{"line_number":225,"context_line":".. [#f1] https://docs.openshift.org/latest/dev_guide/managing_images.html#using-image-pull-secrets"},{"line_number":226,"context_line":".. _k8s_raw Ansible module: http://docs.ansible.com/ansible/2.5/modules/k8s_raw_module.html"},{"line_number":227,"context_line":""},{"line_number":228,"context_line":".. _git-repo-sync:"},{"line_number":229,"context_line":""}],"source_content_type":"text/x-rst","patch_set":4,"id":"5f7c97a3_bda50cb6","line":226,"updated":"2018-07-09 15:00:46.000000000","message":"k8s_raw handles openshift and k8s as well.","commit_id":"1b2b1d0b8c34774df3012f7390311021c59b3642"},{"author":{"_account_id":27900,"name":"Artem Goncharov","email":"artem.goncharov@gmail.com","username":"gtema"},"change_message_id":"7bb045967a59b05bc2d2b281318a294f6c3bb796","unresolved":false,"context_lines":[{"line_number":223,"context_line":""},{"line_number":224,"context_line":".. _OpenShift Project: https://docs.openshift.org/latest/dev_guide/projects.html"},{"line_number":225,"context_line":".. [#f1] https://docs.openshift.org/latest/dev_guide/managing_images.html#using-image-pull-secrets"},{"line_number":226,"context_line":".. _k8s_raw Ansible module: http://docs.ansible.com/ansible/2.5/modules/k8s_raw_module.html"},{"line_number":227,"context_line":""},{"line_number":228,"context_line":".. _git-repo-sync:"},{"line_number":229,"context_line":""}],"source_content_type":"text/x-rst","patch_set":4,"id":"5f7c97a3_137503cc","line":226,"in_reply_to":"5f7c97a3_bda50cb6","updated":"2018-07-10 06:28:01.000000000","message":"and it is removed in ansible-2.6 in favor of `k8s`","commit_id":"1b2b1d0b8c34774df3012f7390311021c59b3642"},{"author":{"_account_id":1,"name":"James E. Blair","email":"jim@acmegating.com","username":"corvus"},"change_message_id":"2760d9efff45b1ea4c161749b702af4e43c2291a","unresolved":false,"context_lines":[{"line_number":307,"context_line":"to allow the user to tell Zuul that the resources used by a job (e.g.,"},{"line_number":308,"context_line":"the Kubernetes namespace, and any containers or other nodes) should"},{"line_number":309,"context_line":"continue running until the end of the buildset.  These resources would"},{"line_number":310,"context_line":"then be placed in the inventory of child jobs for their use.  In this"},{"line_number":311,"context_line":"way, the job we constructed earlier which built and image and uploaded"},{"line_number":312,"context_line":"it into a registry that it hosted could then be the root of a tree of"},{"line_number":313,"context_line":"child jobs which use that registry.  If the image-build-registry job"}],"source_content_type":"text/x-rst","patch_set":4,"id":"5f7c97a3_38d1cfcb","line":310,"updated":"2018-07-23 15:34:24.000000000","message":"I think it\u0027s implied sufficiently, especially for this hand-wavey thing we might possibly do in the future.","commit_id":"1b2b1d0b8c34774df3012f7390311021c59b3642"},{"author":{"_account_id":7069,"name":"Joshua Hesketh","email":"josh@nitrotech.org","username":"jhesketh"},"change_message_id":"50b90632be1508e4837a5f7cb766de4f6fff9f65","unresolved":false,"context_lines":[{"line_number":307,"context_line":"to allow the user to tell Zuul that the resources used by a job (e.g.,"},{"line_number":308,"context_line":"the Kubernetes namespace, and any containers or other nodes) should"},{"line_number":309,"context_line":"continue running until the end of the buildset.  These resources would"},{"line_number":310,"context_line":"then be placed in the inventory of child jobs for their use.  In this"},{"line_number":311,"context_line":"way, the job we constructed earlier which built and image and uploaded"},{"line_number":312,"context_line":"it into a registry that it hosted could then be the root of a tree of"},{"line_number":313,"context_line":"child jobs which use that registry.  If the image-build-registry job"}],"source_content_type":"text/x-rst","patch_set":4,"id":"5f7c97a3_f0d95df0","line":310,"range":{"start_line":310,"start_character":56,"end_line":310,"end_character":59},"updated":"2018-07-10 05:08:06.000000000","message":"I think it\u0027s possibly implied here, but for completeness I think it\u0027s worth pointing out that we\u0027d want a resource to be able to be placed in multiple [child] jobs at once. This would allow multiple jobs to reuse a built image at once.\n\nThis means nodepool will need to handle the cleanup only after each job using a resource is complete.","commit_id":"1b2b1d0b8c34774df3012f7390311021c59b3642"},{"author":{"_account_id":7069,"name":"Joshua Hesketh","email":"josh@nitrotech.org","username":"jhesketh"},"change_message_id":"50b90632be1508e4837a5f7cb766de4f6fff9f65","unresolved":false,"context_lines":[{"line_number":312,"context_line":"it into a registry that it hosted could then be the root of a tree of"},{"line_number":313,"context_line":"child jobs which use that registry.  If the image-build-registry job"},{"line_number":314,"context_line":"created a service token, that could be passed to the child jobs for"},{"line_number":315,"context_line":"their use when they start their own containers or pods."},{"line_number":316,"context_line":""},{"line_number":317,"context_line":"In order to support this, we may need to implement provider affinity"},{"line_number":318,"context_line":"for builds in a buildset in Nodepool so that we don\u0027t have to deal"}],"source_content_type":"text/x-rst","patch_set":4,"id":"5f7c97a3_b09685ee","line":315,"updated":"2018-07-10 05:08:06.000000000","message":"An alternate, or perhaps additional, approach is to consider how to pass arbitrary resources or files around between any jobs, regardless of their buildset.\n\nFor example, a job may build an image or some kind of artifact, which after gating has been completed needs to be uploaded to a central registry. A post job should be able to grab the same verified image.\n\nThis is likely a bit out of the scope of this spec though. I think having the pool resource (as described in this section) shared between child jobs is still necessary. Artifact persistence could be revisited later.","commit_id":"1b2b1d0b8c34774df3012f7390311021c59b3642"},{"author":{"_account_id":1,"name":"James E. Blair","email":"jim@acmegating.com","username":"corvus"},"change_message_id":"2760d9efff45b1ea4c161749b702af4e43c2291a","unresolved":false,"context_lines":[{"line_number":312,"context_line":"it into a registry that it hosted could then be the root of a tree of"},{"line_number":313,"context_line":"child jobs which use that registry.  If the image-build-registry job"},{"line_number":314,"context_line":"created a service token, that could be passed to the child jobs for"},{"line_number":315,"context_line":"their use when they start their own containers or pods."},{"line_number":316,"context_line":""},{"line_number":317,"context_line":"In order to support this, we may need to implement provider affinity"},{"line_number":318,"context_line":"for builds in a buildset in Nodepool so that we don\u0027t have to deal"}],"source_content_type":"text/x-rst","patch_set":4,"id":"5f7c97a3_d8dd3bdf","line":315,"updated":"2018-07-23 15:34:24.000000000","message":"Yes, that\u0027s a different problem space entirely.","commit_id":"1b2b1d0b8c34774df3012f7390311021c59b3642"},{"author":{"_account_id":4146,"name":"Clark Boylan","email":"cboylan@sapwetik.org","username":"cboylan"},"change_message_id":"bda0e619ed919fb2015de037720986c03ddac697","unresolved":false,"context_lines":[{"line_number":255,"context_line":"container image builds in Nodepool in the future, we can add support"},{"line_number":256,"context_line":"for that later."},{"line_number":257,"context_line":""},{"line_number":258,"context_line":"The first step in the process is to create a new pod based on either a"},{"line_number":259,"context_line":"base image.  Ensure it has ``git`` installed.  If the pod is going to"},{"line_number":260,"context_line":"be used to run a single command (i.e., :ref:`container-machine`, or"},{"line_number":261,"context_line":"will only be used to build images), then a single container is"}],"source_content_type":"text/x-rst","patch_set":5,"id":"5f7c97a3_d9e14599","line":258,"range":{"start_line":258,"start_character":62,"end_line":258,"end_character":68},"updated":"2018-07-27 23:06:37.000000000","message":"Slight nit. This either implies an alternative but I don\u0027t see one. Maybe it should just be \"a new pod based on a base image\" ?","commit_id":"3ad89dba93277282b9adcb7d0097557d6a243e85"}]}