)]}'
{"goals/ocata/split-tempest-plugins.rst":[{"author":{"_account_id":10239,"name":"Dmitry Tantsur","email":"dtantsur@protonmail.com","username":"dtantsur"},"change_message_id":"838bd809d5a5d98e7a2e782235c9a3ac6108724c","unresolved":false,"context_lines":[{"line_number":8,"context_line":"in any python project as long as it properly sets up the entrypoint. A common"},{"line_number":9,"context_line":"pattern has emerged of bundling these plugins inside a project repo. This is"},{"line_number":10,"context_line":"likely because it makes contributing tests at the same time as an API feature"},{"line_number":11,"context_line":"must simpler. However, this is really an antipattern and makes consuming the"},{"line_number":12,"context_line":"test plugins much more difficult, especially in all-in-one deployments. In"},{"line_number":13,"context_line":"order to make using plugins more consistent and easier for end users,"},{"line_number":14,"context_line":"packagers, and deployers this goal is to make sure we always use a separate"}],"source_content_type":"text/x-rst","patch_set":2,"id":"fa6399be_16780df8","line":11,"range":{"start_line":11,"start_character":14,"end_line":11,"end_character":72},"updated":"2016-10-18 12:01:19.000000000","message":"I can\u0027t disagree more with this idea. The goal of our CI is to test changes, period. Everything else is nice to have, of course, but not at expense of it. Landing tests with the code is the only correct way (what about moving unit tests to another repo? packagers put them to separate packages).\n\nWhat you\u0027re saying here is that we should sacrifice testing changes in favor of someone (who?) external consuming this.","commit_id":"d6c8dbfe757c9e63830238e8fd5364fdf4d8ce0f"},{"author":{"_account_id":2472,"name":"Doug Hellmann","email":"dhellmann@redhat.com","username":"doug-hellmann"},"change_message_id":"2283439f150b4473fa0437be5d4d34f3bc1afb30","unresolved":false,"context_lines":[{"line_number":8,"context_line":"in any python project as long as it properly sets up the entrypoint. A common"},{"line_number":9,"context_line":"pattern has emerged of bundling these plugins inside a project repo. This is"},{"line_number":10,"context_line":"likely because it makes contributing tests at the same time as an API feature"},{"line_number":11,"context_line":"must simpler. However, this is really an antipattern and makes consuming the"},{"line_number":12,"context_line":"test plugins much more difficult, especially in all-in-one deployments. In"},{"line_number":13,"context_line":"order to make using plugins more consistent and easier for end users,"},{"line_number":14,"context_line":"packagers, and deployers this goal is to make sure we always use a separate"}],"source_content_type":"text/x-rst","patch_set":2,"id":"7a8ec9b2_82f861c6","line":11,"updated":"2016-09-19 20:20:33.000000000","message":"s/must/much/","commit_id":"d6c8dbfe757c9e63830238e8fd5364fdf4d8ce0f"},{"author":{"_account_id":1955,"name":"Alan Pevec","email":"alan.pevec@redhat.com","username":"apevec"},"change_message_id":"6c19f984860774114db009079488b247b97f6b77","unresolved":false,"context_lines":[{"line_number":8,"context_line":"in any python project as long as it properly sets up the entrypoint. A common"},{"line_number":9,"context_line":"pattern has emerged of bundling these plugins inside a project repo. This is"},{"line_number":10,"context_line":"likely because it makes contributing tests at the same time as an API feature"},{"line_number":11,"context_line":"must simpler. However, this is really an antipattern and makes consuming the"},{"line_number":12,"context_line":"test plugins much more difficult, especially in all-in-one deployments. In"},{"line_number":13,"context_line":"order to make using plugins more consistent and easier for end users,"},{"line_number":14,"context_line":"packagers, and deployers this goal is to make sure we always use a separate"}],"source_content_type":"text/x-rst","patch_set":2,"id":"fa6399be_b59d855b","line":11,"range":{"start_line":11,"start_character":14,"end_line":11,"end_character":72},"in_reply_to":"fa6399be_16780df8","updated":"2016-10-18 18:43:24.000000000","message":"I\u0027m not in QA/Tempest team but I think idea is that you can have functional tests in-tree if you don\u0027t want to expose them, and keep in out-of-tree tempest plugin only tests for your service\u0027s public API.","commit_id":"d6c8dbfe757c9e63830238e8fd5364fdf4d8ce0f"},{"author":{"_account_id":5196,"name":"Matthew Treinish","email":"mtreinish@kortar.org","username":"treinish"},"change_message_id":"51aec99ca5d6e4adbe48f1d21a0d74f6620a6e2c","unresolved":false,"context_lines":[{"line_number":8,"context_line":"in any python project as long as it properly sets up the entrypoint. A common"},{"line_number":9,"context_line":"pattern has emerged of bundling these plugins inside a project repo. This is"},{"line_number":10,"context_line":"likely because it makes contributing tests at the same time as an API feature"},{"line_number":11,"context_line":"must simpler. However, this is really an antipattern and makes consuming the"},{"line_number":12,"context_line":"test plugins much more difficult, especially in all-in-one deployments. In"},{"line_number":13,"context_line":"order to make using plugins more consistent and easier for end users,"},{"line_number":14,"context_line":"packagers, and deployers this goal is to make sure we always use a separate"}],"source_content_type":"text/x-rst","patch_set":2,"id":"fa6399be_101d5f66","line":11,"range":{"start_line":11,"start_character":14,"end_line":11,"end_character":72},"in_reply_to":"fa6399be_b59d855b","updated":"2016-10-18 19:00:05.000000000","message":"Right, Alan you\u0027ve got the basic idea. If you don\u0027t want to expose tests to the outside world and just use them for the gate you can easily just test things with an in-tree test suite and not use the plugin interface. The plugin interface is about making things easy to consume both inside and outside the gate.\n\nThe other aspect here is that doing seperate repos/projects for plugins actually fits the model of tempest. The current mismatch is where most of the issues with external consumption issues come from. This also effects the gate and testing the api effectively across release boundaries (see my earlier comment) but we\u0027ve down the groundwork to hack around that, a lot of people don\u0027t even realize that.","commit_id":"d6c8dbfe757c9e63830238e8fd5364fdf4d8ce0f"},{"author":{"_account_id":6159,"name":"Flavio Percoco Premoli","display_name":"flaper87","email":"flavio.percoco@flyrlabs.com","username":"flaper87"},"change_message_id":"c5fd14af7382f5a058cfc8ac2e022cda5b2439cd","unresolved":false,"context_lines":[{"line_number":9,"context_line":"pattern has emerged of bundling these plugins inside a project repo. This is"},{"line_number":10,"context_line":"likely because it makes contributing tests at the same time as an API feature"},{"line_number":11,"context_line":"must simpler. However, this is really an antipattern and makes consuming the"},{"line_number":12,"context_line":"test plugins much more difficult, especially in all-in-one deployments. In"},{"line_number":13,"context_line":"order to make using plugins more consistent and easier for end users,"},{"line_number":14,"context_line":"packagers, and deployers this goal is to make sure we always use a separate"},{"line_number":15,"context_line":"python project for each plugin."}],"source_content_type":"text/x-rst","patch_set":2,"id":"7a8ec9b2_640330b9","line":12,"updated":"2016-09-20 14:54:47.000000000","message":"How does this compare with devstack, which has a similar (in-tree) model for plugins?\n\nI know it\u0027s a different language, different purpose and different way to consume things but, why is it more difficult for tempest to consume plugins this way than it is for devstack? It would be good to elaborate on this here to help folks understand the purpose of this change better.\n\nWe need to evaluate the trade-offs since this will significantly change the way tests are contributed in each project.","commit_id":"d6c8dbfe757c9e63830238e8fd5364fdf4d8ce0f"},{"author":{"_account_id":9664,"name":"amrith","email":"amrith.kumar@gmail.com","username":"amrith"},"change_message_id":"1bcf82bd7cbf922de4ead65fb45f961b5481371f","unresolved":false,"context_lines":[{"line_number":9,"context_line":"pattern has emerged of bundling these plugins inside a project repo. This is"},{"line_number":10,"context_line":"likely because it makes contributing tests at the same time as an API feature"},{"line_number":11,"context_line":"must simpler. However, this is really an antipattern and makes consuming the"},{"line_number":12,"context_line":"test plugins much more difficult, especially in all-in-one deployments. In"},{"line_number":13,"context_line":"order to make using plugins more consistent and easier for end users,"},{"line_number":14,"context_line":"packagers, and deployers this goal is to make sure we always use a separate"},{"line_number":15,"context_line":"python project for each plugin."}],"source_content_type":"text/x-rst","patch_set":2,"id":"7a8ec9b2_8453dc38","line":12,"in_reply_to":"7a8ec9b2_640330b9","updated":"2016-09-20 15:03:59.000000000","message":"Are you asking me? I hope not because I bring up the same exact issue in my earlier comment.","commit_id":"d6c8dbfe757c9e63830238e8fd5364fdf4d8ce0f"},{"author":{"_account_id":10239,"name":"Dmitry Tantsur","email":"dtantsur@protonmail.com","username":"dtantsur"},"change_message_id":"838bd809d5a5d98e7a2e782235c9a3ac6108724c","unresolved":false,"context_lines":[{"line_number":10,"context_line":"likely because it makes contributing tests at the same time as an API feature"},{"line_number":11,"context_line":"must simpler. However, this is really an antipattern and makes consuming the"},{"line_number":12,"context_line":"test plugins much more difficult, especially in all-in-one deployments. In"},{"line_number":13,"context_line":"order to make using plugins more consistent and easier for end users,"},{"line_number":14,"context_line":"packagers, and deployers this goal is to make sure we always use a separate"},{"line_number":15,"context_line":"python project for each plugin."},{"line_number":16,"context_line":""}],"source_content_type":"text/x-rst","patch_set":2,"id":"fa6399be_f68f19ca","line":13,"range":{"start_line":13,"start_character":14,"end_line":13,"end_character":68},"updated":"2016-10-18 12:01:19.000000000","message":"Who are end users here? I imply a person with openstack credentials sitting in front of CLI and/or Horizon. Are we doing to have Horizon support for Tempest? I think this is confusing.","commit_id":"d6c8dbfe757c9e63830238e8fd5364fdf4d8ce0f"},{"author":{"_account_id":5196,"name":"Matthew Treinish","email":"mtreinish@kortar.org","username":"treinish"},"change_message_id":"51aec99ca5d6e4adbe48f1d21a0d74f6620a6e2c","unresolved":false,"context_lines":[{"line_number":10,"context_line":"likely because it makes contributing tests at the same time as an API feature"},{"line_number":11,"context_line":"must simpler. However, this is really an antipattern and makes consuming the"},{"line_number":12,"context_line":"test plugins much more difficult, especially in all-in-one deployments. In"},{"line_number":13,"context_line":"order to make using plugins more consistent and easier for end users,"},{"line_number":14,"context_line":"packagers, and deployers this goal is to make sure we always use a separate"},{"line_number":15,"context_line":"python project for each plugin."},{"line_number":16,"context_line":""}],"source_content_type":"text/x-rst","patch_set":2,"id":"fa6399be_d5af5929","line":13,"range":{"start_line":13,"start_character":14,"end_line":13,"end_character":68},"in_reply_to":"fa6399be_35eed5ec","updated":"2016-10-18 19:00:05.000000000","message":"Yeah, that\u0027s a good example. Another, prominent example is anyone who wants a defcore certifcation. Tempest is designed to work against any cloud, not just in the gate (it\u0027s the first design tenant in the README) and it\u0027s commonly used to test clouds outside the gate.\n\nDmitry, please take a look at tempest\u0027s documentation, it explains some use cases and how to run it outside the gate and hopefully this should clear up some of your confusion.","commit_id":"d6c8dbfe757c9e63830238e8fd5364fdf4d8ce0f"},{"author":{"_account_id":1955,"name":"Alan Pevec","email":"alan.pevec@redhat.com","username":"apevec"},"change_message_id":"6c19f984860774114db009079488b247b97f6b77","unresolved":false,"context_lines":[{"line_number":10,"context_line":"likely because it makes contributing tests at the same time as an API feature"},{"line_number":11,"context_line":"must simpler. However, this is really an antipattern and makes consuming the"},{"line_number":12,"context_line":"test plugins much more difficult, especially in all-in-one deployments. In"},{"line_number":13,"context_line":"order to make using plugins more consistent and easier for end users,"},{"line_number":14,"context_line":"packagers, and deployers this goal is to make sure we always use a separate"},{"line_number":15,"context_line":"python project for each plugin."},{"line_number":16,"context_line":""}],"source_content_type":"text/x-rst","patch_set":2,"id":"fa6399be_35eed5ec","line":13,"range":{"start_line":13,"start_character":14,"end_line":13,"end_character":68},"in_reply_to":"fa6399be_f68f19ca","updated":"2016-10-18 18:43:24.000000000","message":"Example end-user would be an administrator using Tempest to validate his fresh openstack installation.","commit_id":"d6c8dbfe757c9e63830238e8fd5364fdf4d8ce0f"},{"author":{"_account_id":9664,"name":"amrith","email":"amrith.kumar@gmail.com","username":"amrith"},"change_message_id":"08da78e145d345b5b1ecbd48799599a6bb84919f","unresolved":false,"context_lines":[{"line_number":12,"context_line":"test plugins much more difficult, especially in all-in-one deployments. In"},{"line_number":13,"context_line":"order to make using plugins more consistent and easier for end users,"},{"line_number":14,"context_line":"packagers, and deployers this goal is to make sure we always use a separate"},{"line_number":15,"context_line":"python project for each plugin."},{"line_number":16,"context_line":""},{"line_number":17,"context_line":""},{"line_number":18,"context_line":"Gerrit Topic"}],"source_content_type":"text/x-rst","patch_set":2,"id":"7a8ec9b2_6ca87cdc","line":15,"updated":"2016-09-20 14:36:53.000000000","message":"if I understand this correctly, the gist is that it is easier to develop features and tests if the features and the tests are in the same project. But it is harder to consume the tests.\n\nSo, we\u0027ll make it easier to consume the tests, knowing that it may become harder to develop features and tests together.\n\nThis strikes me as counter intuitive.\n\nIn the case of the devstack plugin for example, making the plugin part of the projects is a benefit for the projects because we can do in the plugin the things we need for the feature that is being released.\n\nIt appears that this proposal is going towards more projects and more repositories at the expense of development.\n\nI may be misunderstanding this, but that\u0027s what I understand of the stated end-state.\n\nAs stated in the tempest plugin file (referenced later in references)\n\n\"The sole advantage to integrating a plugin into an existing python project is that it enables you to land code changes at the same time you land test changes in the plugin. This reduces some of the burden on contributors by not having to land 2 changes to add a new API feature and then test it and doing it as a single combined commit.\"\n\nI think that \"sole advantage\" is a pretty compelling reason to do things this way.","commit_id":"d6c8dbfe757c9e63830238e8fd5364fdf4d8ce0f"},{"author":{"_account_id":2417,"name":"Ben Swartzlander","email":"ben@swartzlander.org","username":"bswartz"},"change_message_id":"ea31d794ddb5fd996851f35bdbdcafb6461425c5","unresolved":false,"context_lines":[{"line_number":12,"context_line":"test plugins much more difficult, especially in all-in-one deployments. In"},{"line_number":13,"context_line":"order to make using plugins more consistent and easier for end users,"},{"line_number":14,"context_line":"packagers, and deployers this goal is to make sure we always use a separate"},{"line_number":15,"context_line":"python project for each plugin."},{"line_number":16,"context_line":""},{"line_number":17,"context_line":""},{"line_number":18,"context_line":"Gerrit Topic"}],"source_content_type":"text/x-rst","patch_set":2,"id":"fa6399be_459bb0c9","line":15,"in_reply_to":"7a8ec9b2_303f9726","updated":"2016-10-11 13:47:33.000000000","message":"I hate the 2 repo idea. It\u0027s very important to be able to push a single commit that changes both the code and the tests at the same time. If the problem being solved is that installing tests is difficult then I\u0027d propose that projects generate \"test tarballs\" with each commit and that tempest installs from those tarballs instead of cloning git repos. Tarballs are lightweight (compared to git cloning) and can be trimmed to contain exactly what tempest needs.","commit_id":"d6c8dbfe757c9e63830238e8fd5364fdf4d8ce0f"},{"author":{"_account_id":5196,"name":"Matthew Treinish","email":"mtreinish@kortar.org","username":"treinish"},"change_message_id":"96083f2e93bf36c7f663a971d8db323fcbc51585","unresolved":false,"context_lines":[{"line_number":12,"context_line":"test plugins much more difficult, especially in all-in-one deployments. In"},{"line_number":13,"context_line":"order to make using plugins more consistent and easier for end users,"},{"line_number":14,"context_line":"packagers, and deployers this goal is to make sure we always use a separate"},{"line_number":15,"context_line":"python project for each plugin."},{"line_number":16,"context_line":""},{"line_number":17,"context_line":""},{"line_number":18,"context_line":"Gerrit Topic"}],"source_content_type":"text/x-rst","patch_set":2,"id":"7a8ec9b2_67c89275","line":15,"in_reply_to":"7a8ec9b2_426e27ca","updated":"2016-09-20 15:12:25.000000000","message":"This ignores the entire user story around tempest plugins. You\u0027re optimizing for the wrong problem when you bundle it in repo, you\u0027re putting developer ease ahead of our users, packagers, and operators. I don\u0027t think that\u0027s the right way to do things. Tempest is used for far more than just the gate and saying we should make external users lives more difficult so we don\u0027t have to \n\nThere are several issues with the current situation: (or at least the ones I can remember right now, there might be others)\n\n1. You deploy an all in one system with 20 services that have bundled tempest plugins and then run tempest outside of a venv. When you run tempest you see the tempest tests for everything you installed whether you intended to or not. (It is also a fairly common occurence for a project to ship a broken plugin that breaks at import time preventing  discovery from working on any project which coinstalls in devstack)\n\n2. A packager tries to use a stable version of your project with a newer tempest. This results in conflicting requirements because tempest is branchless and follows master requirements and makes using stable release plugins with master tempest impossible.\n\n3. Most projects don\u0027t actually use the same tests across release boundaries (or just don\u0027t use plugins on stable branches), which means breaking api changes can land. There are 2 exceptions to this trove and ironic which we hacked together a mechanism to install the project repo from master in the tempest venv. \n\n4. Plugin requirements aren\u0027t exposed at install time. Most projects use test-requirements to set tempest plugin requirements (since it is for tests) which do not get installed when you pip install a project. However the entrypoint gets exposed and tempest will pick it up regardless of whether all the requirements are installed.\n\nHaving tempest plugins in a separate repo/project solves all of these issues and makes using a plugin opt in all cases. (ie you have to pip install plugin to use it and won\u0027t show up as an unexpected side effect) It also means for packagers that you don\u0027t have to install all of a project repo to ship a plugin package.\n\n@Amrith Also, your comparison to devstack plugins is invalid, Devstack plugins are designed to live in a project repo and a hard coupled to the release and stable branch of the project. They also aren\u0027t registered and python install time and have to be explicitly added to your localrc/local.conf. Tempest plugins are not the same thing and are designed to test APIs which are not coupled to a particular release.","commit_id":"d6c8dbfe757c9e63830238e8fd5364fdf4d8ce0f"},{"author":{"_account_id":10980,"name":"Doug Wiegley","email":"dougwig@parkside.io","username":"dougw"},"change_message_id":"377522efd132575a0b6c8d70424ebbac04c3486a","unresolved":false,"context_lines":[{"line_number":12,"context_line":"test plugins much more difficult, especially in all-in-one deployments. In"},{"line_number":13,"context_line":"order to make using plugins more consistent and easier for end users,"},{"line_number":14,"context_line":"packagers, and deployers this goal is to make sure we always use a separate"},{"line_number":15,"context_line":"python project for each plugin."},{"line_number":16,"context_line":""},{"line_number":17,"context_line":""},{"line_number":18,"context_line":"Gerrit Topic"}],"source_content_type":"text/x-rst","patch_set":2,"id":"7a8ec9b2_303f9726","line":15,"in_reply_to":"7a8ec9b2_67c89275","updated":"2016-09-20 20:06:34.000000000","message":"This really boils down to ease of tester vs ease of dev.\n\n\u003e\u003e \"The sole advantage to integrating a plugin into an existing python project is that it enables you to land code changes at the same time you land test changes in the plugin. This reduces some of the burden on contributors by not having to land 2 changes to add a new API feature and then test it and doing it as a single combined commit.\"\n\u003e I think that \"sole advantage\" is a pretty compelling reason to do things this way.\n\nThe two repo dance is pretty miserable, and there is literally no enforcement for the second commit actually ever happening. And in many projects with more than one project repo, it is MORE than a two repo dance. It\u0027s already darn near untenable. We\u0027ll just end up back in the space where no one contributes tests.","commit_id":"d6c8dbfe757c9e63830238e8fd5364fdf4d8ce0f"},{"author":{"_account_id":782,"name":"John Garbutt","email":"john@johngarbutt.com","username":"johngarbutt"},"change_message_id":"1ef02b42447a3e210514d0314b347e33e31d30b4","unresolved":false,"context_lines":[{"line_number":12,"context_line":"test plugins much more difficult, especially in all-in-one deployments. In"},{"line_number":13,"context_line":"order to make using plugins more consistent and easier for end users,"},{"line_number":14,"context_line":"packagers, and deployers this goal is to make sure we always use a separate"},{"line_number":15,"context_line":"python project for each plugin."},{"line_number":16,"context_line":""},{"line_number":17,"context_line":""},{"line_number":18,"context_line":"Gerrit Topic"}],"source_content_type":"text/x-rst","patch_set":2,"id":"7a8ec9b2_1e7717d5","line":15,"in_reply_to":"7a8ec9b2_67c89275","updated":"2016-09-20 16:43:42.000000000","message":"mtrenish: love the detail in your comment. I think its worth stating the problem like that in this goal, so its clear why we should do this thing.\n\ndepends-on really helps now, making it easy to review/dev/merge code and tests at the same time.","commit_id":"d6c8dbfe757c9e63830238e8fd5364fdf4d8ce0f"},{"author":{"_account_id":9656,"name":"Ihar Hrachyshka","email":"ihrachys@redhat.com","username":"ihrachys","status":"Red Hat Networking Systems Engineer"},"change_message_id":"62113addb314eedf59e8359226bfa2a62b24a42f","unresolved":false,"context_lines":[{"line_number":12,"context_line":"test plugins much more difficult, especially in all-in-one deployments. In"},{"line_number":13,"context_line":"order to make using plugins more consistent and easier for end users,"},{"line_number":14,"context_line":"packagers, and deployers this goal is to make sure we always use a separate"},{"line_number":15,"context_line":"python project for each plugin."},{"line_number":16,"context_line":""},{"line_number":17,"context_line":""},{"line_number":18,"context_line":"Gerrit Topic"}],"source_content_type":"text/x-rst","patch_set":2,"id":"7a8ec9b2_426e27ca","line":15,"in_reply_to":"7a8ec9b2_6ca87cdc","updated":"2016-09-20 14:39:44.000000000","message":"Agreed. As far as I see it, the only problem consumers have is that the entry point is installed for them when base project code is installed. If that\u0027s the deal, then we can move the entry point in a separate repo, but still keep the tests close to the project code.","commit_id":"d6c8dbfe757c9e63830238e8fd5364fdf4d8ce0f"},{"author":{"_account_id":964,"name":"Anne Gentle","email":"annegentle@justwriteclick.com","username":"annegentle"},"change_message_id":"9ae71e02057b4dab119b3c7f9e41b79d15321113","unresolved":false,"context_lines":[{"line_number":13,"context_line":"order to make using plugins more consistent and easier for end users,"},{"line_number":14,"context_line":"packagers, and deployers this goal is to make sure we always use a separate"},{"line_number":15,"context_line":"python project for each plugin."},{"line_number":16,"context_line":""},{"line_number":17,"context_line":""},{"line_number":18,"context_line":"Gerrit Topic"},{"line_number":19,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"}],"source_content_type":"text/x-rst","patch_set":2,"id":"7a8ec9b2_508423b0","line":16,"updated":"2016-09-20 20:12:00.000000000","message":"I think it\u0027s a good idea to add the benefits listing from your comment directly to the doc.","commit_id":"d6c8dbfe757c9e63830238e8fd5364fdf4d8ce0f"},{"author":{"_account_id":2472,"name":"Doug Hellmann","email":"dhellmann@redhat.com","username":"doug-hellmann"},"change_message_id":"2283439f150b4473fa0437be5d4d34f3bc1afb30","unresolved":false,"context_lines":[{"line_number":28,"context_line":""},{"line_number":29,"context_line":"For all projects with a bundled tempest plugin:"},{"line_number":30,"context_line":""},{"line_number":31,"context_line":"#. Create a new seperate repo for the tempest plugin"},{"line_number":32,"context_line":"#. Migrate all the functionality from the bundled plugin to the new repo"},{"line_number":33,"context_line":"#. Switch gating jobs to use the new plugin"},{"line_number":34,"context_line":"#. Delete the bundled tempest plugin from the project repo"}],"source_content_type":"text/x-rst","patch_set":2,"id":"7a8ec9b2_a291dd54","line":31,"updated":"2016-09-19 20:20:33.000000000","message":"If this is approved, it would be useful to document how to migrate the existing files while retaining their git history. We have tools for that floating around somewhere in an oslo repository.","commit_id":"d6c8dbfe757c9e63830238e8fd5364fdf4d8ce0f"},{"author":{"_account_id":2472,"name":"Doug Hellmann","email":"dhellmann@redhat.com","username":"doug-hellmann"},"change_message_id":"6ce65f5fb007944ea6cd53daf51b691f66216bea","unresolved":false,"context_lines":[{"line_number":28,"context_line":""},{"line_number":29,"context_line":"For all projects with a bundled tempest plugin:"},{"line_number":30,"context_line":""},{"line_number":31,"context_line":"#. Create a new seperate repo for the tempest plugin"},{"line_number":32,"context_line":"#. Migrate all the functionality from the bundled plugin to the new repo"},{"line_number":33,"context_line":"#. Switch gating jobs to use the new plugin"},{"line_number":34,"context_line":"#. Delete the bundled tempest plugin from the project repo"}],"source_content_type":"text/x-rst","patch_set":2,"id":"7a8ec9b2_f0252f5c","line":31,"in_reply_to":"7a8ec9b2_627083dd","updated":"2016-09-20 19:59:23.000000000","message":"That tool is http://git.openstack.org/cgit/openstack/oslo.tools/tree/filter_git_history.sh","commit_id":"d6c8dbfe757c9e63830238e8fd5364fdf4d8ce0f"},{"author":{"_account_id":9664,"name":"amrith","email":"amrith.kumar@gmail.com","username":"amrith"},"change_message_id":"08da78e145d345b5b1ecbd48799599a6bb84919f","unresolved":false,"context_lines":[{"line_number":28,"context_line":""},{"line_number":29,"context_line":"For all projects with a bundled tempest plugin:"},{"line_number":30,"context_line":""},{"line_number":31,"context_line":"#. Create a new seperate repo for the tempest plugin"},{"line_number":32,"context_line":"#. Migrate all the functionality from the bundled plugin to the new repo"},{"line_number":33,"context_line":"#. Switch gating jobs to use the new plugin"},{"line_number":34,"context_line":"#. Delete the bundled tempest plugin from the project repo"}],"source_content_type":"text/x-rst","patch_set":2,"id":"7a8ec9b2_ec8bac66","line":31,"in_reply_to":"7a8ec9b2_a291dd54","updated":"2016-09-20 14:36:53.000000000","message":"++","commit_id":"d6c8dbfe757c9e63830238e8fd5364fdf4d8ce0f"},{"author":{"_account_id":5196,"name":"Matthew Treinish","email":"mtreinish@kortar.org","username":"treinish"},"change_message_id":"96083f2e93bf36c7f663a971d8db323fcbc51585","unresolved":false,"context_lines":[{"line_number":28,"context_line":""},{"line_number":29,"context_line":"For all projects with a bundled tempest plugin:"},{"line_number":30,"context_line":""},{"line_number":31,"context_line":"#. Create a new seperate repo for the tempest plugin"},{"line_number":32,"context_line":"#. Migrate all the functionality from the bundled plugin to the new repo"},{"line_number":33,"context_line":"#. Switch gating jobs to use the new plugin"},{"line_number":34,"context_line":"#. Delete the bundled tempest plugin from the project repo"}],"source_content_type":"text/x-rst","patch_set":2,"id":"7a8ec9b2_627083dd","line":31,"in_reply_to":"7a8ec9b2_ec8bac66","updated":"2016-09-20 15:12:25.000000000","message":"Sure, I can do that. I think this is a good idea","commit_id":"d6c8dbfe757c9e63830238e8fd5364fdf4d8ce0f"},{"author":{"_account_id":9664,"name":"amrith","email":"amrith.kumar@gmail.com","username":"amrith"},"change_message_id":"08da78e145d345b5b1ecbd48799599a6bb84919f","unresolved":false,"context_lines":[{"line_number":31,"context_line":"#. Create a new seperate repo for the tempest plugin"},{"line_number":32,"context_line":"#. Migrate all the functionality from the bundled plugin to the new repo"},{"line_number":33,"context_line":"#. Switch gating jobs to use the new plugin"},{"line_number":34,"context_line":"#. Delete the bundled tempest plugin from the project repo"},{"line_number":35,"context_line":""},{"line_number":36,"context_line":"References"},{"line_number":37,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"}],"source_content_type":"text/x-rst","patch_set":2,"id":"7a8ec9b2_6ccddcb6","line":34,"updated":"2016-09-20 14:36:53.000000000","message":"is the new repository versioned and released?\n\nOn what cadence?\n\nHow do we handle the tempest plugin version to project version mapping?","commit_id":"d6c8dbfe757c9e63830238e8fd5364fdf4d8ce0f"},{"author":{"_account_id":14885,"name":"Daniel Mellado","email":"dmellado@redhat.com","username":"daniel-mellado"},"change_message_id":"fbc97edf7315c66a9d83fd3b57c7c0602513dfc8","unresolved":false,"context_lines":[{"line_number":31,"context_line":"#. Create a new seperate repo for the tempest plugin"},{"line_number":32,"context_line":"#. Migrate all the functionality from the bundled plugin to the new repo"},{"line_number":33,"context_line":"#. Switch gating jobs to use the new plugin"},{"line_number":34,"context_line":"#. Delete the bundled tempest plugin from the project repo"},{"line_number":35,"context_line":""},{"line_number":36,"context_line":"References"},{"line_number":37,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"}],"source_content_type":"text/x-rst","patch_set":2,"id":"7a8ec9b2_4720f6b0","line":34,"in_reply_to":"7a8ec9b2_4477143b","updated":"2016-09-20 15:18:43.000000000","message":"In any case, I assume there would be a tag associated to them when there\u0027s a new release, isn\u0027t it? As you were commenting, what would happen if there\u0027s API breakage between releases? If that\u0027s the case, maybe it\u0027d be for the better than those repos would have branches matching the project release.","commit_id":"d6c8dbfe757c9e63830238e8fd5364fdf4d8ce0f"},{"author":{"_account_id":5196,"name":"Matthew Treinish","email":"mtreinish@kortar.org","username":"treinish"},"change_message_id":"a276fccec3454fbd08a7b19d16831f83ffe44f91","unresolved":false,"context_lines":[{"line_number":31,"context_line":"#. Create a new seperate repo for the tempest plugin"},{"line_number":32,"context_line":"#. Migrate all the functionality from the bundled plugin to the new repo"},{"line_number":33,"context_line":"#. Switch gating jobs to use the new plugin"},{"line_number":34,"context_line":"#. Delete the bundled tempest plugin from the project repo"},{"line_number":35,"context_line":""},{"line_number":36,"context_line":"References"},{"line_number":37,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"}],"source_content_type":"text/x-rst","patch_set":2,"id":"7a8ec9b2_aa30a108","line":34,"in_reply_to":"7a8ec9b2_4720f6b0","updated":"2016-09-20 15:31:35.000000000","message":"@Daniel We should never be allowing an API change to occur across a release boundary. It breaks our end user contract on backwards compatibility. If an api change is required it has to be versioned and an opt in thing. Otherwise any one using the api would be subject to random breakages across multiple clouds or when they upgrade a deployment. This is the exact reason why tempest is branchless and decoupled from the project release cycle and that plugins should be doing the same thing.","commit_id":"d6c8dbfe757c9e63830238e8fd5364fdf4d8ce0f"},{"author":{"_account_id":5196,"name":"Matthew Treinish","email":"mtreinish@kortar.org","username":"treinish"},"change_message_id":"96083f2e93bf36c7f663a971d8db323fcbc51585","unresolved":false,"context_lines":[{"line_number":31,"context_line":"#. Create a new seperate repo for the tempest plugin"},{"line_number":32,"context_line":"#. Migrate all the functionality from the bundled plugin to the new repo"},{"line_number":33,"context_line":"#. Switch gating jobs to use the new plugin"},{"line_number":34,"context_line":"#. Delete the bundled tempest plugin from the project repo"},{"line_number":35,"context_line":""},{"line_number":36,"context_line":"References"},{"line_number":37,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"}],"source_content_type":"text/x-rst","patch_set":2,"id":"7a8ec9b2_4477143b","line":34,"in_reply_to":"7a8ec9b2_6ccddcb6","updated":"2016-09-20 15:12:25.000000000","message":"Ideally yes they\u0027d be versioned and released, but it wouldn\u0027t be a requirement. (although it makes the setup much easier, ie you do pip install foo-tempest-plugin \u0026\u0026 tempest run) \n\nTempest tests are about the APIs not released versions. There should never be a hard coupling between the 2, if there is that means you don\u0027t have backwards compatible api. The release cadence for plugins is completely decoupled from the project under test. That\u0027s a big part of the problem with bundling them.","commit_id":"d6c8dbfe757c9e63830238e8fd5364fdf4d8ce0f"}],"goals/pike/split-tempest-plugins.rst":[{"author":{"_account_id":10239,"name":"Dmitry Tantsur","email":"dtantsur@protonmail.com","username":"dtantsur"},"change_message_id":"aa8e5386a4441dd3abcc6eea63ce165c724b3c31","unresolved":false,"context_lines":[{"line_number":17,"context_line":"There are several issues with the current situation including:"},{"line_number":18,"context_line":""},{"line_number":19,"context_line":"1. You deploy an all in one system with 20 services that have bundled tempest"},{"line_number":20,"context_line":"plugins and then run tempest outside of a venv. When you run tempest you see"},{"line_number":21,"context_line":"the tempest tests for everything you installed whether you intended to or not."},{"line_number":22,"context_line":"(It is also a fairly common occurrence for a project to ship a broken plugin"},{"line_number":23,"context_line":"that breaks at import time preventing discovery from working on any project"},{"line_number":24,"context_line":"which co-installs in devstack)"}],"source_content_type":"text/x-rst","patch_set":3,"id":"ba5201f7_f213925a","line":21,"range":{"start_line":20,"start_character":48,"end_line":21,"end_character":78},"updated":"2017-01-02 17:34:32.000000000","message":"I\u0027d say it\u0027s more of a flow in how we run tests. I don\u0027t think we should automatically import python code from all present entrypoints, unless something like --yes-really-all-of-them flag is passed to the tempest runner.\n\nAnd splitting plugins away only hides this problem, not really solves it. Installing a problematic plugin will still break everything.","commit_id":"7cb8f93e9294e2f5b56ad4bf43e3142f1cd85829"},{"author":{"_account_id":10239,"name":"Dmitry Tantsur","email":"dtantsur@protonmail.com","username":"dtantsur"},"change_message_id":"bb54351ae1003935ed2dd87497d83bbebd423a2f","unresolved":false,"context_lines":[{"line_number":17,"context_line":"There are several issues with the current situation including:"},{"line_number":18,"context_line":""},{"line_number":19,"context_line":"1. You deploy an all in one system with 20 services that have bundled tempest"},{"line_number":20,"context_line":"plugins and then run tempest outside of a venv. When you run tempest you see"},{"line_number":21,"context_line":"the tempest tests for everything you installed whether you intended to or not."},{"line_number":22,"context_line":"(It is also a fairly common occurrence for a project to ship a broken plugin"},{"line_number":23,"context_line":"that breaks at import time preventing discovery from working on any project"},{"line_number":24,"context_line":"which co-installs in devstack)"}],"source_content_type":"text/x-rst","patch_set":3,"id":"ba5201f7_a3fe0936","line":21,"range":{"start_line":20,"start_character":48,"end_line":21,"end_character":78},"in_reply_to":"ba5201f7_25b141e2","updated":"2017-01-03 16:22:45.000000000","message":"\u003e tempest owns the test runner, which it doesn\u0027t\n\nWe can start owning it, why not? If the current approach gives us problems. Your proposal does not solve this issue: as soon as the packages with split out plugins are installed, they have a potential of breaking tempest.","commit_id":"7cb8f93e9294e2f5b56ad4bf43e3142f1cd85829"},{"author":{"_account_id":5196,"name":"Matthew Treinish","email":"mtreinish@kortar.org","username":"treinish"},"change_message_id":"efd28608c9421378fb4dde4e376e61e1d670a0e1","unresolved":false,"context_lines":[{"line_number":17,"context_line":"There are several issues with the current situation including:"},{"line_number":18,"context_line":""},{"line_number":19,"context_line":"1. You deploy an all in one system with 20 services that have bundled tempest"},{"line_number":20,"context_line":"plugins and then run tempest outside of a venv. When you run tempest you see"},{"line_number":21,"context_line":"the tempest tests for everything you installed whether you intended to or not."},{"line_number":22,"context_line":"(It is also a fairly common occurrence for a project to ship a broken plugin"},{"line_number":23,"context_line":"that breaks at import time preventing discovery from working on any project"},{"line_number":24,"context_line":"which co-installs in devstack)"}],"source_content_type":"text/x-rst","patch_set":3,"id":"ba5201f7_390a4822","line":21,"range":{"start_line":20,"start_character":48,"end_line":21,"end_character":78},"in_reply_to":"ba5201f7_a3fe0936","updated":"2017-01-03 16:52:09.000000000","message":"Because, that\u0027s not how unittest works. Tempest is not going to implement a unittest runner, it\u0027s way outside the scope of the project. Tempest also have to maintain backwards compatibility and using the standard python unittest APIs and exposing tempest as a test suite isn\u0027t going to change.\n\nI\u0027m not sure I understand what you\u0027re getting at. The point of splitting it out is to make the install explicit. In today\u0027s world you can install a service and inadvertently get a plugin installed. But, by having a separate project for each plugin it makes the install explicit. You don\u0027t have any plugins that you didn\u0027t explicitly. This also works in reverse, which is where the real trouble with bundling lies. If you don\u0027t want the plugin you uninstall it. If you install a broken plugin you can just uninstall it. Today with bundled plugins this means uninstalling a service, which is something you often can\u0027t do.","commit_id":"7cb8f93e9294e2f5b56ad4bf43e3142f1cd85829"},{"author":{"_account_id":5196,"name":"Matthew Treinish","email":"mtreinish@kortar.org","username":"treinish"},"change_message_id":"4bac1609322d87669c1c489c14bdc274a1f4b094","unresolved":false,"context_lines":[{"line_number":17,"context_line":"There are several issues with the current situation including:"},{"line_number":18,"context_line":""},{"line_number":19,"context_line":"1. You deploy an all in one system with 20 services that have bundled tempest"},{"line_number":20,"context_line":"plugins and then run tempest outside of a venv. When you run tempest you see"},{"line_number":21,"context_line":"the tempest tests for everything you installed whether you intended to or not."},{"line_number":22,"context_line":"(It is also a fairly common occurrence for a project to ship a broken plugin"},{"line_number":23,"context_line":"that breaks at import time preventing discovery from working on any project"},{"line_number":24,"context_line":"which co-installs in devstack)"}],"source_content_type":"text/x-rst","patch_set":3,"id":"ba5201f7_25b141e2","line":21,"range":{"start_line":20,"start_character":48,"end_line":21,"end_character":78},"in_reply_to":"ba5201f7_f213925a","updated":"2017-01-03 15:59:02.000000000","message":"Except you\u0027re making an assumption that tempest owns the test runner, which it doesn\u0027t. We can add a CLI argument to the tempest run command. But that\u0027s not the only way people execute tempest tests. (and is likely the smaller number of users)","commit_id":"7cb8f93e9294e2f5b56ad4bf43e3142f1cd85829"},{"author":{"_account_id":2472,"name":"Doug Hellmann","email":"dhellmann@redhat.com","username":"doug-hellmann"},"change_message_id":"2f1179b4d4ed966ebd975daab6ead8edf24e4b46","unresolved":false,"context_lines":[{"line_number":21,"context_line":"the tempest tests for everything you installed whether you intended to or not."},{"line_number":22,"context_line":"(It is also a fairly common occurrence for a project to ship a broken plugin"},{"line_number":23,"context_line":"that breaks at import time preventing discovery from working on any project"},{"line_number":24,"context_line":"which co-installs in devstack)"},{"line_number":25,"context_line":""},{"line_number":26,"context_line":"2. A packager tries to use a stable version of your project with a newer"},{"line_number":27,"context_line":"tempest. This results in conflicting requirements because tempest is branchless"}],"source_content_type":"text/x-rst","patch_set":3,"id":"ba5201f7_a1529e11","line":24,"updated":"2017-01-02 15:17:55.000000000","message":"We can fix tempest so import errors are reported as a test failure instead of causing the entire run to fail.","commit_id":"7cb8f93e9294e2f5b56ad4bf43e3142f1cd85829"},{"author":{"_account_id":2472,"name":"Doug Hellmann","email":"dhellmann@redhat.com","username":"doug-hellmann"},"change_message_id":"04a7808b1e1eec159b7b19209b43a601742aada4","unresolved":false,"context_lines":[{"line_number":21,"context_line":"the tempest tests for everything you installed whether you intended to or not."},{"line_number":22,"context_line":"(It is also a fairly common occurrence for a project to ship a broken plugin"},{"line_number":23,"context_line":"that breaks at import time preventing discovery from working on any project"},{"line_number":24,"context_line":"which co-installs in devstack)"},{"line_number":25,"context_line":""},{"line_number":26,"context_line":"2. A packager tries to use a stable version of your project with a newer"},{"line_number":27,"context_line":"tempest. This results in conflicting requirements because tempest is branchless"}],"source_content_type":"text/x-rst","patch_set":3,"id":"ba5201f7_3eb78e87","line":24,"in_reply_to":"ba5201f7_451005a9","updated":"2017-01-03 16:43:07.000000000","message":"It sounds like I made the same assumption as Dmitry did. Fair enough.","commit_id":"7cb8f93e9294e2f5b56ad4bf43e3142f1cd85829"},{"author":{"_account_id":5196,"name":"Matthew Treinish","email":"mtreinish@kortar.org","username":"treinish"},"change_message_id":"4bac1609322d87669c1c489c14bdc274a1f4b094","unresolved":false,"context_lines":[{"line_number":21,"context_line":"the tempest tests for everything you installed whether you intended to or not."},{"line_number":22,"context_line":"(It is also a fairly common occurrence for a project to ship a broken plugin"},{"line_number":23,"context_line":"that breaks at import time preventing discovery from working on any project"},{"line_number":24,"context_line":"which co-installs in devstack)"},{"line_number":25,"context_line":""},{"line_number":26,"context_line":"2. A packager tries to use a stable version of your project with a newer"},{"line_number":27,"context_line":"tempest. This results in conflicting requirements because tempest is branchless"}],"source_content_type":"text/x-rst","patch_set":3,"id":"ba5201f7_451005a9","line":24,"in_reply_to":"ba5201f7_a1529e11","updated":"2017-01-03 15:59:02.000000000","message":"Actually that\u0027s not as straightforward as it would seem. The import errors are hit by unittest discovery and aren\u0027t being caused by tempest directly. Tempest only tells the test runner (using the unittest load_tests api) to also use plugin tests. The actual errors are not being raised anywhere in tempest\u0027s code, it happens in a layer above and outside of tempest. (it\u0027s the same thing that kills any test run with testr if there is an import error in a project)\n\nIf it was a simple fix, this wouldn\u0027t be on the list.","commit_id":"7cb8f93e9294e2f5b56ad4bf43e3142f1cd85829"},{"author":{"_account_id":10239,"name":"Dmitry Tantsur","email":"dtantsur@protonmail.com","username":"dtantsur"},"change_message_id":"aa8e5386a4441dd3abcc6eea63ce165c724b3c31","unresolved":false,"context_lines":[{"line_number":24,"context_line":"which co-installs in devstack)"},{"line_number":25,"context_line":""},{"line_number":26,"context_line":"2. A packager tries to use a stable version of your project with a newer"},{"line_number":27,"context_line":"tempest. This results in conflicting requirements because tempest is branchless"},{"line_number":28,"context_line":"and follows master requirements and makes using stable release plugins with"},{"line_number":29,"context_line":"master tempest impossible."},{"line_number":30,"context_line":""},{"line_number":31,"context_line":"3. Most projects don\u0027t actually use the same tests across release boundaries"}],"source_content_type":"text/x-rst","patch_set":3,"id":"ba5201f7_521b7e72","line":28,"range":{"start_line":27,"start_character":59,"end_line":28,"end_character":31},"updated":"2017-01-02 17:34:32.000000000","message":"This problem actually prevents (some?) packagers from shipping tempest master. It\u0027s not only about plugins, we can\u0027t e.g. have conflicting versions of oslo.utils or whatever common dependencies tempest has with other projects.","commit_id":"7cb8f93e9294e2f5b56ad4bf43e3142f1cd85829"},{"author":{"_account_id":2472,"name":"Doug Hellmann","email":"dhellmann@redhat.com","username":"doug-hellmann"},"change_message_id":"2f1179b4d4ed966ebd975daab6ead8edf24e4b46","unresolved":false,"context_lines":[{"line_number":32,"context_line":"(or just don\u0027t use plugins on stable branches), which means breaking api"},{"line_number":33,"context_line":"changes can land. There are 2 exceptions to this trove and ironic which we"},{"line_number":34,"context_line":"hacked together a mechanism to install the project repo from master in the"},{"line_number":35,"context_line":"tempest venv."},{"line_number":36,"context_line":""},{"line_number":37,"context_line":"4. Plugin requirements aren\u0027t exposed at install time. Most projects use"},{"line_number":38,"context_line":"test-requirements to set tempest plugin requirements (since it is for tests)"}],"source_content_type":"text/x-rst","patch_set":3,"id":"ba5201f7_6149965f","line":35,"updated":"2017-01-02 15:17:55.000000000","message":"This seems like the key motivation from a community perspective: Ensuring API compatibility across releases by using branchless tempest plugins.","commit_id":"7cb8f93e9294e2f5b56ad4bf43e3142f1cd85829"},{"author":{"_account_id":10239,"name":"Dmitry Tantsur","email":"dtantsur@protonmail.com","username":"dtantsur"},"change_message_id":"aa8e5386a4441dd3abcc6eea63ce165c724b3c31","unresolved":false,"context_lines":[{"line_number":32,"context_line":"(or just don\u0027t use plugins on stable branches), which means breaking api"},{"line_number":33,"context_line":"changes can land. There are 2 exceptions to this trove and ironic which we"},{"line_number":34,"context_line":"hacked together a mechanism to install the project repo from master in the"},{"line_number":35,"context_line":"tempest venv."},{"line_number":36,"context_line":""},{"line_number":37,"context_line":"4. Plugin requirements aren\u0027t exposed at install time. Most projects use"},{"line_number":38,"context_line":"test-requirements to set tempest plugin requirements (since it is for tests)"}],"source_content_type":"text/x-rst","patch_set":3,"id":"ba5201f7_f23ab2cc","line":35,"in_reply_to":"ba5201f7_6149965f","updated":"2017-01-02 17:34:32.000000000","message":"\u003e Ensuring API compatibility across releases by using branchless tempest plugins.\n\n+1 to this way to put it","commit_id":"7cb8f93e9294e2f5b56ad4bf43e3142f1cd85829"},{"author":{"_account_id":2472,"name":"Doug Hellmann","email":"dhellmann@redhat.com","username":"doug-hellmann"},"change_message_id":"2f1179b4d4ed966ebd975daab6ead8edf24e4b46","unresolved":false,"context_lines":[{"line_number":38,"context_line":"test-requirements to set tempest plugin requirements (since it is for tests)"},{"line_number":39,"context_line":"which do not get installed when you pip install a project. However the"},{"line_number":40,"context_line":"entrypoint gets exposed and tempest will pick it up regardless of whether all"},{"line_number":41,"context_line":"the requirements are installed."},{"line_number":42,"context_line":""},{"line_number":43,"context_line":"Having tempest plugins in a separate repo/project solves all of these issues"},{"line_number":44,"context_line":"and makes using a plugin opt in all cases. (ie you have to pip install plugin"}],"source_content_type":"text/x-rst","patch_set":3,"id":"ba5201f7_8174ba9c","line":41,"updated":"2017-01-02 15:17:55.000000000","message":"This sounds like the cause of point 1?","commit_id":"7cb8f93e9294e2f5b56ad4bf43e3142f1cd85829"},{"author":{"_account_id":5196,"name":"Matthew Treinish","email":"mtreinish@kortar.org","username":"treinish"},"change_message_id":"4bac1609322d87669c1c489c14bdc274a1f4b094","unresolved":false,"context_lines":[{"line_number":38,"context_line":"test-requirements to set tempest plugin requirements (since it is for tests)"},{"line_number":39,"context_line":"which do not get installed when you pip install a project. However the"},{"line_number":40,"context_line":"entrypoint gets exposed and tempest will pick it up regardless of whether all"},{"line_number":41,"context_line":"the requirements are installed."},{"line_number":42,"context_line":""},{"line_number":43,"context_line":"Having tempest plugins in a separate repo/project solves all of these issues"},{"line_number":44,"context_line":"and makes using a plugin opt in all cases. (ie you have to pip install plugin"}],"source_content_type":"text/x-rst","patch_set":3,"id":"ba5201f7_2545c16f","line":41,"in_reply_to":"ba5201f7_8174ba9c","updated":"2017-01-03 15:59:02.000000000","message":"It\u0027s not always the cause of point 1, but it definitely can be, and commonly is.","commit_id":"7cb8f93e9294e2f5b56ad4bf43e3142f1cd85829"},{"author":{"_account_id":2472,"name":"Doug Hellmann","email":"dhellmann@redhat.com","username":"doug-hellmann"},"change_message_id":"2f1179b4d4ed966ebd975daab6ead8edf24e4b46","unresolved":false,"context_lines":[{"line_number":41,"context_line":"the requirements are installed."},{"line_number":42,"context_line":""},{"line_number":43,"context_line":"Having tempest plugins in a separate repo/project solves all of these issues"},{"line_number":44,"context_line":"and makes using a plugin opt in all cases. (ie you have to pip install plugin"},{"line_number":45,"context_line":"to use it and won\u0027t show up as an unexpected side effect) It also means for"},{"line_number":46,"context_line":"packagers that you don\u0027t have to install all of a project repo to ship a plugin"},{"line_number":47,"context_line":"package."}],"source_content_type":"text/x-rst","patch_set":3,"id":"ba5201f7_c1a5e204","line":44,"range":{"start_line":44,"start_character":25,"end_line":44,"end_character":28},"updated":"2017-01-02 15:17:55.000000000","message":"\"optional\"?","commit_id":"7cb8f93e9294e2f5b56ad4bf43e3142f1cd85829"},{"author":{"_account_id":5196,"name":"Matthew Treinish","email":"mtreinish@kortar.org","username":"treinish"},"change_message_id":"4bac1609322d87669c1c489c14bdc274a1f4b094","unresolved":false,"context_lines":[{"line_number":41,"context_line":"the requirements are installed."},{"line_number":42,"context_line":""},{"line_number":43,"context_line":"Having tempest plugins in a separate repo/project solves all of these issues"},{"line_number":44,"context_line":"and makes using a plugin opt in all cases. (ie you have to pip install plugin"},{"line_number":45,"context_line":"to use it and won\u0027t show up as an unexpected side effect) It also means for"},{"line_number":46,"context_line":"packagers that you don\u0027t have to install all of a project repo to ship a plugin"},{"line_number":47,"context_line":"package."}],"source_content_type":"text/x-rst","patch_set":3,"id":"ba5201f7_c5c4f560","line":44,"range":{"start_line":44,"start_character":25,"end_line":44,"end_character":28},"in_reply_to":"ba5201f7_c1a5e204","updated":"2017-01-03 15:59:02.000000000","message":"No, this is what I meant to write. See:\nhttps://en.wiktionary.org/wiki/opt_in#English\n\nalthough it might normally be a hyphenate(I\u0027m not sure), I can change that if you want.","commit_id":"7cb8f93e9294e2f5b56ad4bf43e3142f1cd85829"},{"author":{"_account_id":2472,"name":"Doug Hellmann","email":"dhellmann@redhat.com","username":"doug-hellmann"},"change_message_id":"04a7808b1e1eec159b7b19209b43a601742aada4","unresolved":false,"context_lines":[{"line_number":41,"context_line":"the requirements are installed."},{"line_number":42,"context_line":""},{"line_number":43,"context_line":"Having tempest plugins in a separate repo/project solves all of these issues"},{"line_number":44,"context_line":"and makes using a plugin opt in all cases. (ie you have to pip install plugin"},{"line_number":45,"context_line":"to use it and won\u0027t show up as an unexpected side effect) It also means for"},{"line_number":46,"context_line":"packagers that you don\u0027t have to install all of a project repo to ship a plugin"},{"line_number":47,"context_line":"package."}],"source_content_type":"text/x-rst","patch_set":3,"id":"ba5201f7_5e71b2df","line":44,"range":{"start_line":44,"start_character":25,"end_line":44,"end_character":28},"in_reply_to":"ba5201f7_c5c4f560","updated":"2017-01-03 16:43:07.000000000","message":"I read it as \"optional in all cases\" but I think you want \"opt-in in all cases\".","commit_id":"7cb8f93e9294e2f5b56ad4bf43e3142f1cd85829"},{"author":{"_account_id":2472,"name":"Doug Hellmann","email":"dhellmann@redhat.com","username":"doug-hellmann"},"change_message_id":"2f1179b4d4ed966ebd975daab6ead8edf24e4b46","unresolved":false,"context_lines":[{"line_number":53,"context_line":"To facilitate tracking, commits related to this goal should use the"},{"line_number":54,"context_line":"gerrit topic::"},{"line_number":55,"context_line":""},{"line_number":56,"context_line":"  split-tempest-plugins"},{"line_number":57,"context_line":""},{"line_number":58,"context_line":"Completion Criteria"},{"line_number":59,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"}],"source_content_type":"text/x-rst","patch_set":3,"id":"ba5201f7_21386ed3","line":56,"updated":"2017-01-02 15:17:55.000000000","message":"I\u0027d like us to be consistent in putting \"goal\" in the topics to emphasize that these are in fact goals and not just another project effort. So maybe \"goal-split-tempest-plugins\"?","commit_id":"7cb8f93e9294e2f5b56ad4bf43e3142f1cd85829"},{"author":{"_account_id":5196,"name":"Matthew Treinish","email":"mtreinish@kortar.org","username":"treinish"},"change_message_id":"4bac1609322d87669c1c489c14bdc274a1f4b094","unresolved":false,"context_lines":[{"line_number":53,"context_line":"To facilitate tracking, commits related to this goal should use the"},{"line_number":54,"context_line":"gerrit topic::"},{"line_number":55,"context_line":""},{"line_number":56,"context_line":"  split-tempest-plugins"},{"line_number":57,"context_line":""},{"line_number":58,"context_line":"Completion Criteria"},{"line_number":59,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"}],"source_content_type":"text/x-rst","patch_set":3,"id":"ba5201f7_e52c994c","line":56,"in_reply_to":"ba5201f7_21386ed3","updated":"2017-01-03 15:59:02.000000000","message":"Done","commit_id":"7cb8f93e9294e2f5b56ad4bf43e3142f1cd85829"},{"author":{"_account_id":10239,"name":"Dmitry Tantsur","email":"dtantsur@protonmail.com","username":"dtantsur"},"change_message_id":"aa8e5386a4441dd3abcc6eea63ce165c724b3c31","unresolved":false,"context_lines":[{"line_number":78,"context_line":"http://docs.openstack.org/developer/tempest/plugin.html#standalone-plugin-vs-in-repo-plugin"},{"line_number":79,"context_line":""},{"line_number":80,"context_line":""},{"line_number":81,"context_line":"Current State / Anticipated Impact"},{"line_number":82,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"},{"line_number":83,"context_line":""},{"line_number":84,"context_line":"The following script was used to find all the openstack projects with tempest"}],"source_content_type":"text/x-rst","patch_set":3,"id":"ba5201f7_d25cce27","line":81,"range":{"start_line":81,"start_character":16,"end_line":81,"end_character":34},"updated":"2017-01-02 17:34:32.000000000","message":"I don\u0027t think the anticipated impact is really covered in this document. Please discuss the trade-offs coming with this decision.\n\nI also wonder how well branchless tempest plugins are going to play with branched devstack and grenade plugins. Do we need some configuration bits to also migrate from devstack to tempest plugins?","commit_id":"7cb8f93e9294e2f5b56ad4bf43e3142f1cd85829"},{"author":{"_account_id":5196,"name":"Matthew Treinish","email":"mtreinish@kortar.org","username":"treinish"},"change_message_id":"efd28608c9421378fb4dde4e376e61e1d670a0e1","unresolved":false,"context_lines":[{"line_number":78,"context_line":"http://docs.openstack.org/developer/tempest/plugin.html#standalone-plugin-vs-in-repo-plugin"},{"line_number":79,"context_line":""},{"line_number":80,"context_line":""},{"line_number":81,"context_line":"Current State / Anticipated Impact"},{"line_number":82,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"},{"line_number":83,"context_line":""},{"line_number":84,"context_line":"The following script was used to find all the openstack projects with tempest"}],"source_content_type":"text/x-rst","patch_set":3,"id":"ba5201f7_59ce2cb5","line":81,"range":{"start_line":81,"start_character":16,"end_line":81,"end_character":34},"in_reply_to":"ba5201f7_433c7de2","updated":"2017-01-03 16:52:09.000000000","message":"I really don\u0027t see how having a separate tempest plugin repo would effect that block of code in that or any configuration at all. All that is required is you pip install plugin-foo in devstack if you want to ensure the plugin is present. (which devstack also already has an api for see: http://git.openstack.org/cgit/openstack-dev/devstack/tree/lib/tempest#n632 ) If you want the devstack plugin to always install and configure the plugin you just do that.","commit_id":"7cb8f93e9294e2f5b56ad4bf43e3142f1cd85829"},{"author":{"_account_id":10239,"name":"Dmitry Tantsur","email":"dtantsur@protonmail.com","username":"dtantsur"},"change_message_id":"e74ea10025aa19e96f0278f3cbe4be00ac7b280f","unresolved":false,"context_lines":[{"line_number":78,"context_line":"http://docs.openstack.org/developer/tempest/plugin.html#standalone-plugin-vs-in-repo-plugin"},{"line_number":79,"context_line":""},{"line_number":80,"context_line":""},{"line_number":81,"context_line":"Current State / Anticipated Impact"},{"line_number":82,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"},{"line_number":83,"context_line":""},{"line_number":84,"context_line":"The following script was used to find all the openstack projects with tempest"}],"source_content_type":"text/x-rst","patch_set":3,"id":"ba5201f7_7996904d","line":81,"range":{"start_line":81,"start_character":16,"end_line":81,"end_character":34},"in_reply_to":"ba5201f7_59ce2cb5","updated":"2017-01-03 16:59:01.000000000","message":"Yeah, I think it boils down to configuration update taking more steps than before, which is acceptable.\n\nAnother thing interesting to cover here (but may be too detailed): do we expect people to run tempest jobs from all stable branches on these new repositories?","commit_id":"7cb8f93e9294e2f5b56ad4bf43e3142f1cd85829"},{"author":{"_account_id":5196,"name":"Matthew Treinish","email":"mtreinish@kortar.org","username":"treinish"},"change_message_id":"4bac1609322d87669c1c489c14bdc274a1f4b094","unresolved":false,"context_lines":[{"line_number":78,"context_line":"http://docs.openstack.org/developer/tempest/plugin.html#standalone-plugin-vs-in-repo-plugin"},{"line_number":79,"context_line":""},{"line_number":80,"context_line":""},{"line_number":81,"context_line":"Current State / Anticipated Impact"},{"line_number":82,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"},{"line_number":83,"context_line":""},{"line_number":84,"context_line":"The following script was used to find all the openstack projects with tempest"}],"source_content_type":"text/x-rst","patch_set":3,"id":"ba5201f7_e84d5073","line":81,"range":{"start_line":81,"start_character":16,"end_line":81,"end_character":34},"in_reply_to":"ba5201f7_d25cce27","updated":"2017-01-03 15:59:02.000000000","message":"This is a header from the goals template:\nhttp://git.openstack.org/cgit/openstack/governance/tree/goals/template.txt#n34\n\n\nI do not understand what your asking in the second half? Devstack and grenade plugins are completely independent from tempest plugins.","commit_id":"7cb8f93e9294e2f5b56ad4bf43e3142f1cd85829"},{"author":{"_account_id":10239,"name":"Dmitry Tantsur","email":"dtantsur@protonmail.com","username":"dtantsur"},"change_message_id":"bb54351ae1003935ed2dd87497d83bbebd423a2f","unresolved":false,"context_lines":[{"line_number":78,"context_line":"http://docs.openstack.org/developer/tempest/plugin.html#standalone-plugin-vs-in-repo-plugin"},{"line_number":79,"context_line":""},{"line_number":80,"context_line":""},{"line_number":81,"context_line":"Current State / Anticipated Impact"},{"line_number":82,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"},{"line_number":83,"context_line":""},{"line_number":84,"context_line":"The following script was used to find all the openstack projects with tempest"}],"source_content_type":"text/x-rst","patch_set":3,"id":"ba5201f7_433c7de2","line":81,"range":{"start_line":81,"start_character":16,"end_line":81,"end_character":34},"in_reply_to":"ba5201f7_e84d5073","updated":"2017-01-03 16:22:45.000000000","message":"\u003e Devstack and grenade plugins are completely independent from tempest plugins.\n\nYes, except for the fact that tempest requires quite some configuration to work (especially with non-standard things like ironic), and it\u0027s all done in devstack plugins. Like this: https://github.com/openstack/ironic/blob/master/devstack/lib/ironic#L1695-L1723. Currently we can update this in lockstep with tempest plugins, with your proposal we won\u0027t. I wonder if we need these bits moved to tempest plugins as well.","commit_id":"7cb8f93e9294e2f5b56ad4bf43e3142f1cd85829"},{"author":{"_account_id":2472,"name":"Doug Hellmann","email":"dhellmann@redhat.com","username":"doug-hellmann"},"change_message_id":"2f1179b4d4ed966ebd975daab6ead8edf24e4b46","unresolved":false,"context_lines":[{"line_number":147,"context_line":"  for project in found_plugins:"},{"line_number":148,"context_line":"      print(project)"},{"line_number":149,"context_line":""},{"line_number":150,"context_line":"On 13 Sept 2016 this was run and found all the projects with bundled tempest plugins. 3 matches were found that are standalone plugins: tempest-horizon,"},{"line_number":151,"context_line":"designate-tempest-plugin, and sahara-tests (this is a multitest repo but it\u0027s"},{"line_number":152,"context_line":"not bundled with the project so it\u0027s not applicable here either) Also all the"},{"line_number":153,"context_line":"deb packaging repos were removed as they are duplicates. This list is:"}],"source_content_type":"text/x-rst","patch_set":3,"id":"ba5201f7_c120829e","line":150,"updated":"2017-01-02 15:17:55.000000000","message":"The project list has changed since September. Is the output today the same?","commit_id":"7cb8f93e9294e2f5b56ad4bf43e3142f1cd85829"},{"author":{"_account_id":5196,"name":"Matthew Treinish","email":"mtreinish@kortar.org","username":"treinish"},"change_message_id":"4bac1609322d87669c1c489c14bdc274a1f4b094","unresolved":false,"context_lines":[{"line_number":147,"context_line":"  for project in found_plugins:"},{"line_number":148,"context_line":"      print(project)"},{"line_number":149,"context_line":""},{"line_number":150,"context_line":"On 13 Sept 2016 this was run and found all the projects with bundled tempest plugins. 3 matches were found that are standalone plugins: tempest-horizon,"},{"line_number":151,"context_line":"designate-tempest-plugin, and sahara-tests (this is a multitest repo but it\u0027s"},{"line_number":152,"context_line":"not bundled with the project so it\u0027s not applicable here either) Also all the"},{"line_number":153,"context_line":"deb packaging repos were removed as they are duplicates. This list is:"}],"source_content_type":"text/x-rst","patch_set":3,"id":"ba5201f7_45ec858f","line":150,"in_reply_to":"ba5201f7_c120829e","updated":"2017-01-03 15:59:02.000000000","message":"I\u0027ll rerun it and update the output","commit_id":"7cb8f93e9294e2f5b56ad4bf43e3142f1cd85829"},{"author":{"_account_id":782,"name":"John Garbutt","email":"john@johngarbutt.com","username":"johngarbutt"},"change_message_id":"256e98462e8b4981016e7cf6ad3dbb2bc3df816d","unresolved":false,"context_lines":[{"line_number":189,"context_line":"  openstack/watcher"},{"line_number":190,"context_line":"  openstack/zaqar"},{"line_number":191,"context_line":"  openstack/zun"},{"line_number":192,"context_line":""},{"line_number":193,"context_line":"Project Teams"},{"line_number":194,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"},{"line_number":195,"context_line":""}],"source_content_type":"text/x-rst","patch_set":4,"id":"ba5201f7_5a7a7ef2","line":192,"updated":"2017-01-04 10:37:34.000000000","message":"I think this is the same comment steve made, but I guess we could do with a list of the steps the above projects need to take to transition from the test being in the repo.\n\nI assume its a case of:\n* create new repo, add in tests\n* get new tests running in gates, etc\n* drop old tests from the tree, with a note on where to find the new ones\n\nNot sure if that hits our usual deprecation system, I guess its not really and end user facing feature, so it shouldn\u0027t hit that.","commit_id":"5a49a4c02c16d680f0c53446ce1276bbe1e6ee5e"},{"author":{"_account_id":5196,"name":"Matthew Treinish","email":"mtreinish@kortar.org","username":"treinish"},"change_message_id":"43ad79993379eea221694ec3344be973da393f7c","unresolved":false,"context_lines":[{"line_number":189,"context_line":"  openstack/watcher"},{"line_number":190,"context_line":"  openstack/zaqar"},{"line_number":191,"context_line":"  openstack/zun"},{"line_number":192,"context_line":""},{"line_number":193,"context_line":"Project Teams"},{"line_number":194,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"},{"line_number":195,"context_line":""}],"source_content_type":"text/x-rst","patch_set":4,"id":"ba5201f7_f4d5941b","line":192,"in_reply_to":"ba5201f7_5a7a7ef2","updated":"2017-01-06 22:51:21.000000000","message":"Well isn\u0027t that what I put in the completion criteria section?\n\nAs for the deprecation system I hadn\u0027t thought of the implications there. The plugins are an end user thing, but for tempest, not really for end users of the service. (that\u0027s where a lot of the issues stem from is that those 2 things aren\u0027t the same)\n\nLooking at:\n\nhttps://governance.openstack.org/tc/reference/tags/assert_follows-standard-deprecation.html#requirements\n\nit only mentions the things in the service itself so I think we\u0027re probably fine.","commit_id":"5a49a4c02c16d680f0c53446ce1276bbe1e6ee5e"},{"author":{"_account_id":7822,"name":"Chris Hoge (not involved in upstream development)","email":"chris@hogepodge.com","username":"hogepodge"},"change_message_id":"88b522a73b6102dc0d053211f97e4a3874a9f38f","unresolved":false,"context_lines":[{"line_number":8,"context_line":"in any python project as long as it properly sets up the entrypoint. A common"},{"line_number":9,"context_line":"pattern has emerged of bundling these plugins inside a project repo. This is"},{"line_number":10,"context_line":"likely because it makes contributing tests at the same time as an API feature"},{"line_number":11,"context_line":"must simpler. However, this is really an antipattern and makes consuming the"},{"line_number":12,"context_line":"test plugins much more difficult, especially in all-in-one deployments. In"},{"line_number":13,"context_line":"order to make using plugins more consistent and easier for end users,"},{"line_number":14,"context_line":"packagers, and deployers this goal is to make sure we always use a separate"}],"source_content_type":"text/x-rst","patch_set":5,"id":"ba5201f7_5af450ce","line":11,"range":{"start_line":11,"start_character":0,"end_line":11,"end_character":4},"updated":"2017-01-10 22:02:19.000000000","message":"typo \u0027much\u0027","commit_id":"fa140f6e54c11a91fe24812508291a2e6929a424"},{"author":{"_account_id":2417,"name":"Ben Swartzlander","email":"ben@swartzlander.org","username":"bswartz"},"change_message_id":"97ad118463320e1c22d04383bf46786f7638f85c","unresolved":false,"context_lines":[{"line_number":16,"context_line":""},{"line_number":17,"context_line":"There are several issues with the current situation including:"},{"line_number":18,"context_line":""},{"line_number":19,"context_line":"1. When you deploy an all in one system with multiple services that have"},{"line_number":20,"context_line":"bundled tempest plugins and then run tempest outside of a venv tempest will see"},{"line_number":21,"context_line":"the tests for all the plugins whether you intended to use them or not."},{"line_number":22,"context_line":"It is also a fairly common occurrence for a project to ship a broken plugin"},{"line_number":23,"context_line":"that breaks at import time. This prevents unittest discovery from working which"},{"line_number":24,"context_line":"will block any tests from running."},{"line_number":25,"context_line":""},{"line_number":26,"context_line":"2. A packager tries to use a stable version of your project with a newer"},{"line_number":27,"context_line":"tempest. This results in conflicting requirements because tempest is branchless"}],"source_content_type":"text/x-rst","patch_set":5,"id":"7a3c09a3_8140af35","line":24,"range":{"start_line":19,"start_character":0,"end_line":24,"end_character":34},"updated":"2017-01-17 19:05:21.000000000","message":"This seems more like a problem with how tempest discovers tests than where the source code for the tests lives. Splitting the tests into other repos does not solve the tempest problem -- it just creates a workaround.","commit_id":"fa140f6e54c11a91fe24812508291a2e6929a424"},{"author":{"_account_id":8851,"name":"Valeriy Ponomaryov","email":"kiparis.kh@gmail.com","username":"vponomaryov"},"change_message_id":"6aa2ff50537cea093ab19ad44338fb4c43d28dfa","unresolved":false,"context_lines":[{"line_number":16,"context_line":""},{"line_number":17,"context_line":"There are several issues with the current situation including:"},{"line_number":18,"context_line":""},{"line_number":19,"context_line":"1. When you deploy an all in one system with multiple services that have"},{"line_number":20,"context_line":"bundled tempest plugins and then run tempest outside of a venv tempest will see"},{"line_number":21,"context_line":"the tests for all the plugins whether you intended to use them or not."},{"line_number":22,"context_line":"It is also a fairly common occurrence for a project to ship a broken plugin"},{"line_number":23,"context_line":"that breaks at import time. This prevents unittest discovery from working which"},{"line_number":24,"context_line":"will block any tests from running."},{"line_number":25,"context_line":""},{"line_number":26,"context_line":"2. A packager tries to use a stable version of your project with a newer"},{"line_number":27,"context_line":"tempest. This results in conflicting requirements because tempest is branchless"}],"source_content_type":"text/x-rst","patch_set":5,"id":"da36d5c6_67e1a5fe","line":24,"range":{"start_line":19,"start_character":0,"end_line":24,"end_character":34},"in_reply_to":"7a3c09a3_8140af35","updated":"2017-02-28 11:49:30.000000000","message":"I need to mention, that it is possible to update tempest so it will filter plugins it finds, before attempting to load their tests. So, it cannot be valid argument as it is solvable by tempest itself. And yes, I agree with Ben that it is \"workaround\" for problem, not its solving. We should be able not only see list of existing plugins but also enable/disable them to avoid mentioned \"import errors\". This problem will exist even having plugins in separate repos.\nI am, as QA/developer, should be able to disable broken plugin temporarily and use tempest in parallel fixing broken plugin.","commit_id":"fa140f6e54c11a91fe24812508291a2e6929a424"},{"author":{"_account_id":2417,"name":"Ben Swartzlander","email":"ben@swartzlander.org","username":"bswartz"},"change_message_id":"97ad118463320e1c22d04383bf46786f7638f85c","unresolved":false,"context_lines":[{"line_number":23,"context_line":"that breaks at import time. This prevents unittest discovery from working which"},{"line_number":24,"context_line":"will block any tests from running."},{"line_number":25,"context_line":""},{"line_number":26,"context_line":"2. A packager tries to use a stable version of your project with a newer"},{"line_number":27,"context_line":"tempest. This results in conflicting requirements because tempest is branchless"},{"line_number":28,"context_line":"and follows master requirements and makes using stable release plugins with"},{"line_number":29,"context_line":"master tempest impossible."},{"line_number":30,"context_line":""},{"line_number":31,"context_line":"3. Most projects don\u0027t actually use the same tests across release boundaries"},{"line_number":32,"context_line":"(or just don\u0027t use plugins on stable branches), which means breaking api"}],"source_content_type":"text/x-rst","patch_set":5,"id":"7a3c09a3_a1bccb06","line":29,"range":{"start_line":26,"start_character":0,"end_line":29,"end_character":26},"updated":"2017-01-17 19:05:21.000000000","message":"This problem only exists because tempest is branchless. If tempest was properly versioned like everything else and had stable branches, this would be a non-issue.","commit_id":"fa140f6e54c11a91fe24812508291a2e6929a424"},{"author":{"_account_id":2750,"name":"Sean Dague","email":"sean@dague.net","username":"sdague"},"change_message_id":"260441ce92a609ef790bce084047d3005f1a352e","unresolved":false,"context_lines":[{"line_number":23,"context_line":"that breaks at import time. This prevents unittest discovery from working which"},{"line_number":24,"context_line":"will block any tests from running."},{"line_number":25,"context_line":""},{"line_number":26,"context_line":"2. A packager tries to use a stable version of your project with a newer"},{"line_number":27,"context_line":"tempest. This results in conflicting requirements because tempest is branchless"},{"line_number":28,"context_line":"and follows master requirements and makes using stable release plugins with"},{"line_number":29,"context_line":"master tempest impossible."},{"line_number":30,"context_line":""},{"line_number":31,"context_line":"3. Most projects don\u0027t actually use the same tests across release boundaries"},{"line_number":32,"context_line":"(or just don\u0027t use plugins on stable branches), which means breaking api"}],"source_content_type":"text/x-rst","patch_set":5,"id":"7a3c09a3_19c8b773","line":29,"range":{"start_line":26,"start_character":0,"end_line":29,"end_character":26},"in_reply_to":"7a3c09a3_a1bccb06","updated":"2017-01-17 20:04:56.000000000","message":"There is a very long discussion about why tempest is branchless that\u0027s archived in the community. Tempest is branchless for very solid reasons.\n\nHaving projects create branchless test repositories is really the only reliable way to ensure backwards compatibility of their API to users.","commit_id":"fa140f6e54c11a91fe24812508291a2e6929a424"},{"author":{"_account_id":10459,"name":"Luigi Toscano","email":"ltoscano@redhat.com","username":"ltoscano"},"change_message_id":"5772ee72c6e3d831dc588d4578e38d028b983fed","unresolved":false,"context_lines":[{"line_number":61,"context_line":""},{"line_number":62,"context_line":"For all projects with a bundled tempest plugin:"},{"line_number":63,"context_line":""},{"line_number":64,"context_line":"#. Create a new seperate repo for the tempest plugin"},{"line_number":65,"context_line":"#. Migrate all the functionality from the bundled plugin to the new repo"},{"line_number":66,"context_line":"#. Switch gating jobs to use the new plugin project instead of the bundled one"},{"line_number":67,"context_line":"#. Delete the bundled tempest plugin from the project repo"}],"source_content_type":"text/x-rst","patch_set":5,"id":"ba2be162_115dae48","line":64,"range":{"start_line":64,"start_character":16,"end_line":64,"end_character":24},"updated":"2017-03-06 17:18:09.000000000","message":"typo: separate","commit_id":"fa140f6e54c11a91fe24812508291a2e6929a424"},{"author":{"_account_id":7102,"name":"Thomas Bechtold","email":"thomas.bechtold@chainguard.dev","username":"toabctl"},"change_message_id":"f498ccfc289581b2d50b700ced5829ecd151ce98","unresolved":false,"context_lines":[{"line_number":65,"context_line":"#. Migrate all the functionality from the bundled plugin to the new repo"},{"line_number":66,"context_line":"#. Switch gating jobs to use the new plugin project instead of the bundled one"},{"line_number":67,"context_line":"#. Delete the bundled tempest plugin from the project repo"},{"line_number":68,"context_line":""},{"line_number":69,"context_line":"Note, there is no change in governance or ownership of a repository involved"},{"line_number":70,"context_line":"with this change. The new plugin repo/project will still be owned by the"},{"line_number":71,"context_line":"project it was spun out from."}],"source_content_type":"text/x-rst","patch_set":5,"id":"fa31d9ce_1ddeaadc","line":68,"updated":"2017-02-16 07:31:09.000000000","message":"Could we add a naming proposal here? Like: \"the repo should be named $project-tempest-plugin (eg manila-tempest-plugin)\"","commit_id":"fa140f6e54c11a91fe24812508291a2e6929a424"},{"author":{"_account_id":5196,"name":"Matthew Treinish","email":"mtreinish@kortar.org","username":"treinish"},"change_message_id":"c78f7c58bccd3c388513783640cde9e20ab1f660","unresolved":false,"context_lines":[{"line_number":65,"context_line":"#. Migrate all the functionality from the bundled plugin to the new repo"},{"line_number":66,"context_line":"#. Switch gating jobs to use the new plugin project instead of the bundled one"},{"line_number":67,"context_line":"#. Delete the bundled tempest plugin from the project repo"},{"line_number":68,"context_line":""},{"line_number":69,"context_line":"Note, there is no change in governance or ownership of a repository involved"},{"line_number":70,"context_line":"with this change. The new plugin repo/project will still be owned by the"},{"line_number":71,"context_line":"project it was spun out from."}],"source_content_type":"text/x-rst","patch_set":5,"id":"fa31d9ce_a16ba284","line":68,"in_reply_to":"fa31d9ce_1ddeaadc","updated":"2017-02-16 15:07:43.000000000","message":"Yep, I can do that. While I don\u0027t think we should necessarily dictate what a project chooses to name the new repo, I think there is value in adding some recommendations here especially to try and keep things consistent. On the next respin I will add this (and likely also integrate the concrete steps from the etherpad I left in the comments earlier)","commit_id":"fa140f6e54c11a91fe24812508291a2e6929a424"},{"author":{"_account_id":8851,"name":"Valeriy Ponomaryov","email":"kiparis.kh@gmail.com","username":"vponomaryov"},"change_message_id":"6aa2ff50537cea093ab19ad44338fb4c43d28dfa","unresolved":false,"context_lines":[{"line_number":65,"context_line":"#. Migrate all the functionality from the bundled plugin to the new repo"},{"line_number":66,"context_line":"#. Switch gating jobs to use the new plugin project instead of the bundled one"},{"line_number":67,"context_line":"#. Delete the bundled tempest plugin from the project repo"},{"line_number":68,"context_line":""},{"line_number":69,"context_line":"Note, there is no change in governance or ownership of a repository involved"},{"line_number":70,"context_line":"with this change. The new plugin repo/project will still be owned by the"},{"line_number":71,"context_line":"project it was spun out from."}],"source_content_type":"text/x-rst","patch_set":5,"id":"da36d5c6_4759018e","line":68,"in_reply_to":"fa31d9ce_a16ba284","updated":"2017-02-28 11:49:30.000000000","message":"Ok, rephrasing Thomas, can we have \"recommendation\" for naming plugin projects, so, most of projects will be named using same template. Without it, project names will differ much more.","commit_id":"fa140f6e54c11a91fe24812508291a2e6929a424"},{"author":{"_account_id":10459,"name":"Luigi Toscano","email":"ltoscano@redhat.com","username":"ltoscano"},"change_message_id":"5772ee72c6e3d831dc588d4578e38d028b983fed","unresolved":false,"context_lines":[{"line_number":65,"context_line":"#. Migrate all the functionality from the bundled plugin to the new repo"},{"line_number":66,"context_line":"#. Switch gating jobs to use the new plugin project instead of the bundled one"},{"line_number":67,"context_line":"#. Delete the bundled tempest plugin from the project repo"},{"line_number":68,"context_line":""},{"line_number":69,"context_line":"Note, there is no change in governance or ownership of a repository involved"},{"line_number":70,"context_line":"with this change. The new plugin repo/project will still be owned by the"},{"line_number":71,"context_line":"project it was spun out from."}],"source_content_type":"text/x-rst","patch_set":5,"id":"ba2be162_71a6ea54","line":68,"in_reply_to":"fa31d9ce_a16ba284","updated":"2017-03-06 17:18:09.000000000","message":"Yes, please, no strong requirement on the name; I\u0027d like to not have to rename sahara-tests (which contains also other tests).\nOn the other side, it would be good to have recommendation on the name of the tempest plugin directory, relevant for the regexp; that could definitely be ${component}_tempest_plugin.","commit_id":"fa140f6e54c11a91fe24812508291a2e6929a424"},{"author":{"_account_id":782,"name":"John Garbutt","email":"john@johngarbutt.com","username":"johngarbutt"},"change_message_id":"b529c05e406706c1257a3b0e4b52b9295326cc25","unresolved":false,"context_lines":[{"line_number":190,"context_line":"  openstack/watcher"},{"line_number":191,"context_line":"  openstack/zaqar"},{"line_number":192,"context_line":"  openstack/zun"},{"line_number":193,"context_line":""},{"line_number":194,"context_line":"Project Teams"},{"line_number":195,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"},{"line_number":196,"context_line":""}],"source_content_type":"text/x-rst","patch_set":5,"id":"ba5201f7_3a4bfab1","line":193,"updated":"2017-01-04 10:39:35.000000000","message":"Oops, was on wrong revision. But still concerned about what Steve mentioned. Probably worth having a step by step method to switch to the new repo. I assume the obvious way works out for folks, but its good to state how folks have done this already?\n\nI guess I am assuming it has been done already, could we just link to how they did it?","commit_id":"fa140f6e54c11a91fe24812508291a2e6929a424"},{"author":{"_account_id":5196,"name":"Matthew Treinish","email":"mtreinish@kortar.org","username":"treinish"},"change_message_id":"caf46dd86e4a63bebc7a22f3401a52dc036d0066","unresolved":false,"context_lines":[{"line_number":190,"context_line":"  openstack/watcher"},{"line_number":191,"context_line":"  openstack/zaqar"},{"line_number":192,"context_line":"  openstack/zun"},{"line_number":193,"context_line":""},{"line_number":194,"context_line":"Project Teams"},{"line_number":195,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"},{"line_number":196,"context_line":""}],"source_content_type":"text/x-rst","patch_set":5,"id":"ba5201f7_b4c30c37","line":193,"in_reply_to":"ba5201f7_3a4bfab1","updated":"2017-01-06 23:00:15.000000000","message":"Heh, jroll is in the process of doing this now, I asked him to take notes so we can just dump that in here.\n\nThe only other example of a project taking an in-tree plugin repo and creating an out-of-tree plugin in from it is sahara. I can ask tosky if he has any notes from that process. Although that case was slightly different, because they were migrating it into an existing test repo, sahara-tests. In the case of this goal though, I would expect most of the projects listed above to be creating a new repo for it.","commit_id":"fa140f6e54c11a91fe24812508291a2e6929a424"},{"author":{"_account_id":10459,"name":"Luigi Toscano","email":"ltoscano@redhat.com","username":"ltoscano"},"change_message_id":"5772ee72c6e3d831dc588d4578e38d028b983fed","unresolved":false,"context_lines":[{"line_number":190,"context_line":"  openstack/watcher"},{"line_number":191,"context_line":"  openstack/zaqar"},{"line_number":192,"context_line":"  openstack/zun"},{"line_number":193,"context_line":""},{"line_number":194,"context_line":"Project Teams"},{"line_number":195,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"},{"line_number":196,"context_line":""}],"source_content_type":"text/x-rst","patch_set":5,"id":"ba2be162_bc2f593d","line":193,"in_reply_to":"ba5201f7_5e3e12db","updated":"2017-03-06 17:18:09.000000000","message":"For the record, we did both:\n- the initial sahara-tests came from a filtered version of sahara (thanks to https://github.com/ajdruff/git-splits ). That\u0027s the easy part.\n- two histories have been merged later; it required more bureaucracy than the technical part (extract with git-splits, clean with a rebase, push to a unrelated branch, subtree merge). You need someone from infra to directly push the imported commits, then enable enable merge push to push a merge review for the imported branch, check it and approve the review, disable the merge reviews.","commit_id":"fa140f6e54c11a91fe24812508291a2e6929a424"},{"author":{"_account_id":5196,"name":"Matthew Treinish","email":"mtreinish@kortar.org","username":"treinish"},"change_message_id":"434bb25807a2eaff6a6ecf0dd6baaff619e27bb4","unresolved":false,"context_lines":[{"line_number":190,"context_line":"  openstack/watcher"},{"line_number":191,"context_line":"  openstack/zaqar"},{"line_number":192,"context_line":"  openstack/zun"},{"line_number":193,"context_line":""},{"line_number":194,"context_line":"Project Teams"},{"line_number":195,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"},{"line_number":196,"context_line":""}],"source_content_type":"text/x-rst","patch_set":5,"id":"ba5201f7_5e3e12db","line":193,"in_reply_to":"ba5201f7_b4c30c37","updated":"2017-01-10 16:43:05.000000000","message":"I started an etherpad to document the steps needed for doing the process:\n\nhttps://etherpad.openstack.org/p/tempest-separate-plugin","commit_id":"fa140f6e54c11a91fe24812508291a2e6929a424"}],"goals/queens/split-tempest-plugins.rst":[{"author":{"_account_id":1921,"name":"Andrea Frittoli","email":"andrea.frittoli@gmail.com","username":"andrea-frittoli"},"change_message_id":"9be930dea17b48c8ed343faaa1c3bf974115d93f","unresolved":false,"context_lines":[{"line_number":30,"context_line":""},{"line_number":31,"context_line":"3. Most projects don\u0027t actually use the same tests across release boundaries"},{"line_number":32,"context_line":"(or just don\u0027t use plugins on stable branches), which means breaking api"},{"line_number":33,"context_line":"changes can land. There are 2 exceptions to this trove and ironic which we"},{"line_number":34,"context_line":"hacked together a mechanism to install the project repo from master in the"},{"line_number":35,"context_line":"tempest venv."},{"line_number":36,"context_line":""}],"source_content_type":"text/x-rst","patch_set":6,"id":"5ff73747_dd1f556e","line":33,"range":{"start_line":33,"start_character":72,"end_line":33,"end_character":74},"updated":"2017-04-25 21:20:11.000000000","message":"?","commit_id":"6f14e2298bb44070914801bfcc22b3c6dc8ac7a2"},{"author":{"_account_id":1921,"name":"Andrea Frittoli","email":"andrea.frittoli@gmail.com","username":"andrea-frittoli"},"change_message_id":"9be930dea17b48c8ed343faaa1c3bf974115d93f","unresolved":false,"context_lines":[{"line_number":30,"context_line":""},{"line_number":31,"context_line":"3. Most projects don\u0027t actually use the same tests across release boundaries"},{"line_number":32,"context_line":"(or just don\u0027t use plugins on stable branches), which means breaking api"},{"line_number":33,"context_line":"changes can land. There are 2 exceptions to this trove and ironic which we"},{"line_number":34,"context_line":"hacked together a mechanism to install the project repo from master in the"},{"line_number":35,"context_line":"tempest venv."},{"line_number":36,"context_line":""}],"source_content_type":"text/x-rst","patch_set":6,"id":"5ff73747_5d3465fb","line":33,"range":{"start_line":33,"start_character":28,"end_line":33,"end_character":29},"updated":"2017-04-25 21:20:11.000000000","message":"two","commit_id":"6f14e2298bb44070914801bfcc22b3c6dc8ac7a2"},{"author":{"_account_id":1921,"name":"Andrea Frittoli","email":"andrea.frittoli@gmail.com","username":"andrea-frittoli"},"change_message_id":"9be930dea17b48c8ed343faaa1c3bf974115d93f","unresolved":false,"context_lines":[{"line_number":41,"context_line":"whether all the requirements are installed."},{"line_number":42,"context_line":""},{"line_number":43,"context_line":"Having tempest plugins in a separate repo/project solves all of these issues"},{"line_number":44,"context_line":"and makes using a plugin opt-in in all cases. (ie you have to ``pip install"},{"line_number":45,"context_line":"plugin-foo`` to use it and won\u0027t show up as an unexpected side effect of"},{"line_number":46,"context_line":"installing a service) For community development and testing it means we are"},{"line_number":47,"context_line":"actually ensuring API compatibility across releases. It also means for packagers"}],"source_content_type":"text/x-rst","patch_set":6,"id":"5ff73747_d8c6434a","line":44,"range":{"start_line":44,"start_character":47,"end_line":44,"end_character":49},"updated":"2017-04-25 21:20:11.000000000","message":"i.e.","commit_id":"6f14e2298bb44070914801bfcc22b3c6dc8ac7a2"},{"author":{"_account_id":1921,"name":"Andrea Frittoli","email":"andrea.frittoli@gmail.com","username":"andrea-frittoli"},"change_message_id":"9be930dea17b48c8ed343faaa1c3bf974115d93f","unresolved":false,"context_lines":[{"line_number":155,"context_line":"it\u0027s not applicable here either) Also all the deb packaging repos were removed"},{"line_number":156,"context_line":"as they are duplicates. This list is:"},{"line_number":157,"context_line":""},{"line_number":158,"context_line":"::"},{"line_number":159,"context_line":""},{"line_number":160,"context_line":"  openstack/aodh"},{"line_number":161,"context_line":"  openstack/ceilometer"}],"source_content_type":"text/x-rst","patch_set":6,"id":"5ff73747_385c077d","line":158,"range":{"start_line":158,"start_character":0,"end_line":158,"end_character":2},"updated":"2017-04-25 21:20:11.000000000","message":"nova-lxd seems to be missing in this list - but I\u0027m not sure what the status of the project is, the last commit was 3 weeks ago.\n\nhttp://git.openstack.org/cgit/openstack/nova-lxd/tree/nova_lxd_tempest_plugin/plugin.py","commit_id":"6f14e2298bb44070914801bfcc22b3c6dc8ac7a2"},{"author":{"_account_id":2472,"name":"Doug Hellmann","email":"dhellmann@redhat.com","username":"doug-hellmann"},"change_message_id":"05013baff0bb6a4ee924a5f0610e5e5a7610b205","unresolved":false,"context_lines":[{"line_number":81,"context_line":"The following links may be useful to developers attempting to create their"},{"line_number":82,"context_line":"tempest plugin project:"},{"line_number":83,"context_line":""},{"line_number":84,"context_line":"https://www.openstack.org/videos/boston-2017/tempest-plugin-crash-course"},{"line_number":85,"context_line":"https://etherpad.openstack.org/p/tempest-separate-plugin"},{"line_number":86,"context_line":"https://etherpad.openstack.org/p/tempest-plugin-testing"},{"line_number":87,"context_line":"https://etherpad.openstack.org/p/pike-tempest-plugin-dev"}],"source_content_type":"text/x-rst","patch_set":7,"id":"ff0f0b1f_ee098731","line":84,"updated":"2017-05-17 13:24:03.000000000","message":"These all run together as a paragraph when this is rendered to HTML. Can you make it a bullet list?","commit_id":"eaaac0a4e4a15d891135c1fbae9acf33bf0f4467"},{"author":{"_account_id":15524,"name":"dane-fichter","email":"dane.fichter@jhuapl.edu","username":"dane-fichter"},"change_message_id":"a13af5d3107e9a76eda29f207c88060777d9326f","unresolved":false,"context_lines":[{"line_number":81,"context_line":"The following links may be useful to developers attempting to create their"},{"line_number":82,"context_line":"tempest plugin project:"},{"line_number":83,"context_line":""},{"line_number":84,"context_line":"https://www.openstack.org/videos/boston-2017/tempest-plugin-crash-course"},{"line_number":85,"context_line":"https://etherpad.openstack.org/p/tempest-separate-plugin"},{"line_number":86,"context_line":"https://etherpad.openstack.org/p/tempest-plugin-testing"},{"line_number":87,"context_line":"https://etherpad.openstack.org/p/pike-tempest-plugin-dev"}],"source_content_type":"text/x-rst","patch_set":7,"id":"ff0f0b1f_69933151","line":84,"in_reply_to":"ff0f0b1f_ee098731","updated":"2017-05-17 13:38:49.000000000","message":"Done","commit_id":"eaaac0a4e4a15d891135c1fbae9acf33bf0f4467"},{"author":{"_account_id":2417,"name":"Ben Swartzlander","email":"ben@swartzlander.org","username":"bswartz"},"change_message_id":"3550052d3165cd461fefbf106e68e0d70cae2148","unresolved":false,"context_lines":[{"line_number":62,"context_line":"For all projects with a bundled tempest plugin:"},{"line_number":63,"context_line":""},{"line_number":64,"context_line":"#. Create a new seperate repo for the tempest plugin"},{"line_number":65,"context_line":"#. Migrate all the functionality from the bundled plugin to the new repo"},{"line_number":66,"context_line":"#. Switch gating jobs to use the new plugin project instead of the bundled one"},{"line_number":67,"context_line":"#. Delete the bundled tempest plugin from the project repo"},{"line_number":68,"context_line":""}],"source_content_type":"text/x-rst","patch_set":8,"id":"ff0f0b1f_50e129fe","line":65,"updated":"2017-05-17 16:48:33.000000000","message":"It\u0027s unclear here what migrating the functionality means. Based on the discussions from the Atlanta PTG, I\u0027d been hoping that actual test code could remain where it lives while the new plugin repo pulls it in w/ a dependency.\n\nI understand the benefits of the opt-in nature of the test plugin and the ability to package these things in separate RPMs (or whatever package format). My understanding is that the only technical hurdle is the need for 2 setup.cfg files and thus 2 repos -- not actually moving any test code.\n\nMy concern remains that splitting code into more repos means it will be noticeably harder to fix bugs in cases where the test is wrong in addition to the code being wrong, especially if the tests are being used as a gate for code changes (which they are in my project).","commit_id":"f80aa4202efed418d70b579ea2ddc1a70d9f2f57"},{"author":{"_account_id":10459,"name":"Luigi Toscano","email":"ltoscano@redhat.com","username":"ltoscano"},"change_message_id":"c41d009daf3f7a2de0df4cb96dc9b28a1ba6da08","unresolved":false,"context_lines":[{"line_number":62,"context_line":"For all projects with a bundled tempest plugin:"},{"line_number":63,"context_line":""},{"line_number":64,"context_line":"#. Create a new seperate repo for the tempest plugin"},{"line_number":65,"context_line":"#. Migrate all the functionality from the bundled plugin to the new repo"},{"line_number":66,"context_line":"#. Switch gating jobs to use the new plugin project instead of the bundled one"},{"line_number":67,"context_line":"#. Delete the bundled tempest plugin from the project repo"},{"line_number":68,"context_line":""}],"source_content_type":"text/x-rst","patch_set":8,"id":"ff0f0b1f_6479c891","line":65,"in_reply_to":"ff0f0b1f_50e129fe","updated":"2017-05-19 14:16:00.000000000","message":"No, 2 setup.cfg is more complicated.\n\nWe (Sahara) were one of the first team to move the plugin in a separate repository (headless, follow same support policy as Tempest, roughly) and the experience was definitely positive, even when gating changes. A bit more of care is needed, but nothing that impacted the work of fixing. If you want to discuss specific issues, jump to our channel.","commit_id":"f80aa4202efed418d70b579ea2ddc1a70d9f2f57"}]}