)]}'
{"resolutions/20171120-interop-addon-test-location.rst":[{"author":{"_account_id":7822,"name":"Chris Hoge (not involved in upstream development)","email":"chris@hogepodge.com","username":"hogepodge"},"change_message_id":"2c818622b426c0aa16f18e3fd0aa2cb249d49b9e","unresolved":false,"context_lines":[{"line_number":7,"context_line":""},{"line_number":8,"context_line":"When :ref:`20160504_defcore_test_location` was proposed, the only trademark"},{"line_number":9,"context_line":"programs that were in place, or being discussed, used a very small subset of"},{"line_number":10,"context_line":"the projects available in OpenStack."},{"line_number":11,"context_line":""},{"line_number":12,"context_line":"With the proposed \"add on\" trademark programs there exists the possibility for"},{"line_number":13,"context_line":"many more projects to be included in the trademark programs."}],"source_content_type":"text/x-rst","patch_set":2,"id":"7f96bb07_d6b508f0","line":10,"updated":"2018-01-11 19:45:14.000000000","message":"Why don\u0027t we name the projects? They encompass what are the original foundational services. Nova and Swift, and additionally Glance, Cinder, Keystone, and Neutron. There\u0027s a historical basis for the choice of the projects, and why integration tests for those are all part of Tempest.","commit_id":"9a845c7a872d9f8f361c6842984cf50858f7d6c4"},{"author":{"_account_id":7822,"name":"Chris Hoge (not involved in upstream development)","email":"chris@hogepodge.com","username":"hogepodge"},"change_message_id":"2c818622b426c0aa16f18e3fd0aa2cb249d49b9e","unresolved":false,"context_lines":[{"line_number":10,"context_line":"the projects available in OpenStack."},{"line_number":11,"context_line":""},{"line_number":12,"context_line":"With the proposed \"add on\" trademark programs there exists the possibility for"},{"line_number":13,"context_line":"many more projects to be included in the trademark programs."},{"line_number":14,"context_line":""},{"line_number":15,"context_line":"The current tooling and specifications for the add on programs assumes that the"},{"line_number":16,"context_line":"tests will continue to stay in the projects own tempest plugins. This is in"}],"source_content_type":"text/x-rst","patch_set":2,"id":"7f96bb07_36a6acc3","line":13,"updated":"2018-01-11 19:45:14.000000000","message":"Note that the proposed add-on programs depend upon the existence of a base set of services to add the programs on to.","commit_id":"9a845c7a872d9f8f361c6842984cf50858f7d6c4"},{"author":{"_account_id":7822,"name":"Chris Hoge (not involved in upstream development)","email":"chris@hogepodge.com","username":"hogepodge"},"change_message_id":"2c818622b426c0aa16f18e3fd0aa2cb249d49b9e","unresolved":false,"context_lines":[{"line_number":14,"context_line":""},{"line_number":15,"context_line":"The current tooling and specifications for the add on programs assumes that the"},{"line_number":16,"context_line":"tests will continue to stay in the projects own tempest plugins. This is in"},{"line_number":17,"context_line":"conflict with the :ref:`20160504_defcore_test_location` resolution."},{"line_number":18,"context_line":""},{"line_number":19,"context_line":"New Location"},{"line_number":20,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"}],"source_content_type":"text/x-rst","patch_set":2,"id":"7f96bb07_96946046","line":17,"updated":"2018-01-11 19:45:14.000000000","message":"Currently, the Interop Working Group has expanded their candidates for add on and vertical programs to include projects that have working Tempest plug-ins.","commit_id":"9a845c7a872d9f8f361c6842984cf50858f7d6c4"},{"author":{"_account_id":782,"name":"John Garbutt","email":"john@johngarbutt.com","username":"johngarbutt"},"change_message_id":"e9b633bf530ff2c5c7fc12833e15ee524f0a0cd8","unresolved":false,"context_lines":[{"line_number":58,"context_line":"cause unexpected consequences for their use in InterOp processes and"},{"line_number":59,"context_line":"validation. Placing the tests in a separate repository will make it"},{"line_number":60,"context_line":"easier to maintain consistency and avoid breaking the trademark"},{"line_number":61,"context_line":"enforcement tool."},{"line_number":62,"context_line":""},{"line_number":63,"context_line":"Separating the tests into a single plugin also makes it easier for anyone"},{"line_number":64,"context_line":"running the validation tool against their cloud or cloud distribution to use"}],"source_content_type":"text/x-rst","patch_set":2,"id":"7f96bb07_79a8c0da","line":61,"updated":"2018-01-19 15:23:16.000000000","message":"For me its about the single responsiblity principle.\n\nWhen I was running these tests, changes that happened for reasons unrelated to interop caused the cloud I was working with to fail.\n\nNow if we wanted that test change for interop reasons (which we probably did), then that is fine, but those changes after the test is added need great care. Probably better to add a new test and deprecate the old one, to allow for smooth transitions that are easier to track.\n\nEither way we build the things, we want the tests to gate any changes in related projects.\n\nSo I don\u0027t see these as tests that we need to keep in-sync with tempest, its a one time copy, and we ensure the test keeps running correctly in the gate.\n\n... but I totally get if we don\u0027t have the effort available to build it that way.","commit_id":"9a845c7a872d9f8f361c6842984cf50858f7d6c4"}],"resolutions/20180305-trademark-program-test-location.rst":[{"author":{"_account_id":4257,"name":"Zane Bitter","email":"zbitter@redhat.com","username":"zaneb"},"change_message_id":"312471c40bdb24c4ae6f64e18be6bea65ebbbf95","unresolved":false,"context_lines":[{"line_number":43,"context_line":"team is understandably hesitant to host all functional and integration"},{"line_number":44,"context_line":"tests for all projects. However, there are several reasons for"},{"line_number":45,"context_line":"maintaining a controlled set of the tests we use for trademark"},{"line_number":46,"context_line":"enforcement."},{"line_number":47,"context_line":""},{"line_number":48,"context_line":"First, as interop tests are adopted for Trademark Programs, an additional set"},{"line_number":49,"context_line":"of review requirements for changes becomes especially important. For"}],"source_content_type":"text/x-rst","patch_set":3,"id":"ff6b8bd7_b4bba7e3","line":46,"updated":"2018-03-06 15:20:05.000000000","message":"Maybe mention that the QA team lack the review bandwidth to be solely responsible for even just the trademark tests for projects in the add-on trademark programs, since that is a key driver of the policy change.","commit_id":"8d156924bb19bd3c2305750a86a4cf35280d9aa7"},{"author":{"_account_id":8099,"name":"Graham Hayes","email":"gr@ham.ie","username":"graham"},"change_message_id":"1d0949224aeba462943aa513c10876bb3a658385","unresolved":false,"context_lines":[{"line_number":43,"context_line":"team is understandably hesitant to host all functional and integration"},{"line_number":44,"context_line":"tests for all projects. However, there are several reasons for"},{"line_number":45,"context_line":"maintaining a controlled set of the tests we use for trademark"},{"line_number":46,"context_line":"enforcement."},{"line_number":47,"context_line":""},{"line_number":48,"context_line":"First, as interop tests are adopted for Trademark Programs, an additional set"},{"line_number":49,"context_line":"of review requirements for changes becomes especially important. For"}],"source_content_type":"text/x-rst","patch_set":3,"id":"df7087c5_39a9efde","line":46,"in_reply_to":"ff6b8bd7_b4bba7e3","updated":"2018-03-06 22:57:54.000000000","message":"Done","commit_id":"8d156924bb19bd3c2305750a86a4cf35280d9aa7"},{"author":{"_account_id":4257,"name":"Zane Bitter","email":"zbitter@redhat.com","username":"zaneb"},"change_message_id":"312471c40bdb24c4ae6f64e18be6bea65ebbbf95","unresolved":false,"context_lines":[{"line_number":60,"context_line":"location."},{"line_number":61,"context_line":""},{"line_number":62,"context_line":"The TC therefore encourages the InterOp committee to consider it an indication"},{"line_number":63,"context_line":"of future technical direction that tests will live in either:"},{"line_number":64,"context_line":""},{"line_number":65,"context_line":"1. The `Tempest repository`_"},{"line_number":66,"context_line":"2. A tempest plugin that contains only Trademark program tests"}],"source_content_type":"text/x-rst","patch_set":3,"id":"ff6b8bd7_592126b4","line":63,"updated":"2018-03-06 15:20:05.000000000","message":"Is this really just an indication to the InterOp committee? I think of it more as a directive to the projects in question. RefStack already supports getting tests from Tempest or any plugin, so the InterOp committee doesn\u0027t really care.","commit_id":"8d156924bb19bd3c2305750a86a4cf35280d9aa7"},{"author":{"_account_id":6754,"name":"Mark T. Voelker","email":"markvoelker@gmail.com","username":"mvoelker"},"change_message_id":"5bc1f8a89abaddaff0954a6c713d2392e2efb08b","unresolved":false,"context_lines":[{"line_number":60,"context_line":"location."},{"line_number":61,"context_line":""},{"line_number":62,"context_line":"The TC therefore encourages the InterOp committee to consider it an indication"},{"line_number":63,"context_line":"of future technical direction that tests will live in either:"},{"line_number":64,"context_line":""},{"line_number":65,"context_line":"1. The `Tempest repository`_"},{"line_number":66,"context_line":"2. A tempest plugin that contains only Trademark program tests"}],"source_content_type":"text/x-rst","patch_set":3,"id":"ff6b8bd7_bd0dc438","line":63,"in_reply_to":"ff6b8bd7_592126b4","updated":"2018-03-06 20:21:46.000000000","message":"I believe that the \"indication of future technical direction\" bit is a reference to one the Criteria that the InteropWG uses when evaluating whether a Capability should become part of a Guideline.  Check out:\n\n\nhttp://git.openstack.org/cgit/openstack/interop/tree/doc/source/process/CoreCriteria.rst#n58\n\n\nI think basically the intent here is that the TC wants to say \"if the Capability doesn\u0027t have tests in [whatever place we ultimately decide here] and the project doesn\u0027t work on moving them there, then the InteropWG should award 0 points for the Future Direction criteria, therefore making it less likely that the Capability could be added into a Guideline.\"  \n\n\nBut yes, it\u0027s certainly a directive to the projects in question too.","commit_id":"8d156924bb19bd3c2305750a86a4cf35280d9aa7"},{"author":{"_account_id":4257,"name":"Zane Bitter","email":"zbitter@redhat.com","username":"zaneb"},"change_message_id":"312471c40bdb24c4ae6f64e18be6bea65ebbbf95","unresolved":false,"context_lines":[{"line_number":63,"context_line":"of future technical direction that tests will live in either:"},{"line_number":64,"context_line":""},{"line_number":65,"context_line":"1. The `Tempest repository`_"},{"line_number":66,"context_line":"2. A tempest plugin that contains only Trademark program tests"},{"line_number":67,"context_line":""},{"line_number":68,"context_line":"In the case of **2** this plugin can be on a project by project basis, (e.g.:"},{"line_number":69,"context_line":"`glance-trademark-tempest-plugin`), an overall"}],"source_content_type":"text/x-rst","patch_set":3,"id":"ff6b8bd7_34dc7773","line":66,"range":{"start_line":66,"start_character":34,"end_line":66,"end_character":62},"updated":"2018-03-06 15:20:05.000000000","message":"or candidate tests?","commit_id":"8d156924bb19bd3c2305750a86a4cf35280d9aa7"},{"author":{"_account_id":8099,"name":"Graham Hayes","email":"gr@ham.ie","username":"graham"},"change_message_id":"1d0949224aeba462943aa513c10876bb3a658385","unresolved":false,"context_lines":[{"line_number":63,"context_line":"of future technical direction that tests will live in either:"},{"line_number":64,"context_line":""},{"line_number":65,"context_line":"1. The `Tempest repository`_"},{"line_number":66,"context_line":"2. A tempest plugin that contains only Trademark program tests"},{"line_number":67,"context_line":""},{"line_number":68,"context_line":"In the case of **2** this plugin can be on a project by project basis, (e.g.:"},{"line_number":69,"context_line":"`glance-trademark-tempest-plugin`), an overall"}],"source_content_type":"text/x-rst","patch_set":3,"id":"df7087c5_39824f51","line":66,"range":{"start_line":66,"start_character":34,"end_line":66,"end_character":62},"in_reply_to":"ff6b8bd7_34dc7773","updated":"2018-03-06 22:57:54.000000000","message":"Done","commit_id":"8d156924bb19bd3c2305750a86a4cf35280d9aa7"},{"author":{"_account_id":6754,"name":"Mark T. Voelker","email":"markvoelker@gmail.com","username":"mvoelker"},"change_message_id":"868e64b732c1466a62f3caf060b9c9fac1983922","unresolved":false,"context_lines":[{"line_number":63,"context_line":"of future technical direction that tests will live in either:"},{"line_number":64,"context_line":""},{"line_number":65,"context_line":"1. The `Tempest repository`_"},{"line_number":66,"context_line":"2. A tempest plugin that contains only Trademark program tests"},{"line_number":67,"context_line":""},{"line_number":68,"context_line":"In the case of **2** this plugin can be on a project by project basis, (e.g.:"},{"line_number":69,"context_line":"`glance-trademark-tempest-plugin`), an overall"}],"source_content_type":"text/x-rst","patch_set":3,"id":"ff6b8bd7_9df24006","line":66,"range":{"start_line":66,"start_character":34,"end_line":66,"end_character":62},"in_reply_to":"ff6b8bd7_34dc7773","updated":"2018-03-06 20:29:58.000000000","message":"Thinking back, this would be a good thing to clarify.  We discussed at the TC meeting in Dublin and in the QA session that we wanted to clarify the \"chicken-or-egg problem\" that we\u0027ve run up against in the past by adding something in the InteropWG working procedures that tests need to get into [whatever location the TC ultimately decides on] *before* we add those tests to a Guideline.  That being the case, sounds like we do need to include candidate tests here.","commit_id":"8d156924bb19bd3c2305750a86a4cf35280d9aa7"},{"author":{"_account_id":4257,"name":"Zane Bitter","email":"zbitter@redhat.com","username":"zaneb"},"change_message_id":"312471c40bdb24c4ae6f64e18be6bea65ebbbf95","unresolved":false,"context_lines":[{"line_number":66,"context_line":"2. A tempest plugin that contains only Trademark program tests"},{"line_number":67,"context_line":""},{"line_number":68,"context_line":"In the case of **2** this plugin can be on a project by project basis, (e.g.:"},{"line_number":69,"context_line":"`glance-trademark-tempest-plugin`), an overall"},{"line_number":70,"context_line":"`openstack-trademark-tempest-plugin`, or a combination of the above (e.g.:"},{"line_number":71,"context_line":"`nfv-vertical-trademark-tempest-plugin`)."},{"line_number":72,"context_line":""}],"source_content_type":"text/x-rst","patch_set":3,"id":"ff6b8bd7_d1ed1826","line":69,"updated":"2018-03-06 15:20:05.000000000","message":"I suggest that we use the names defined in the service type authority rather than project names here.\n\nFWIW my preferred naming scheme would be tempest-\u003cservice-type\u003e-(interop|trademark)[-plugin]","commit_id":"8d156924bb19bd3c2305750a86a4cf35280d9aa7"},{"author":{"_account_id":8099,"name":"Graham Hayes","email":"gr@ham.ie","username":"graham"},"change_message_id":"1d0949224aeba462943aa513c10876bb3a658385","unresolved":false,"context_lines":[{"line_number":66,"context_line":"2. A tempest plugin that contains only Trademark program tests"},{"line_number":67,"context_line":""},{"line_number":68,"context_line":"In the case of **2** this plugin can be on a project by project basis, (e.g.:"},{"line_number":69,"context_line":"`glance-trademark-tempest-plugin`), an overall"},{"line_number":70,"context_line":"`openstack-trademark-tempest-plugin`, or a combination of the above (e.g.:"},{"line_number":71,"context_line":"`nfv-vertical-trademark-tempest-plugin`)."},{"line_number":72,"context_line":""}],"source_content_type":"text/x-rst","patch_set":3,"id":"df7087c5_d996b395","line":69,"in_reply_to":"ff6b8bd7_d1ed1826","updated":"2018-03-06 22:57:54.000000000","message":"Done","commit_id":"8d156924bb19bd3c2305750a86a4cf35280d9aa7"},{"author":{"_account_id":4257,"name":"Zane Bitter","email":"zbitter@redhat.com","username":"zaneb"},"change_message_id":"312471c40bdb24c4ae6f64e18be6bea65ebbbf95","unresolved":false,"context_lines":[{"line_number":68,"context_line":"In the case of **2** this plugin can be on a project by project basis, (e.g.:"},{"line_number":69,"context_line":"`glance-trademark-tempest-plugin`), an overall"},{"line_number":70,"context_line":"`openstack-trademark-tempest-plugin`, or a combination of the above (e.g.:"},{"line_number":71,"context_line":"`nfv-vertical-trademark-tempest-plugin`)."},{"line_number":72,"context_line":""},{"line_number":73,"context_line":"In these cases, the service clients in use by the tests should be either in"},{"line_number":74,"context_line":"the `Tempest repository`_ or in the `*-trademark-tempest-plugin` repository"}],"source_content_type":"text/x-rst","patch_set":3,"id":"ff6b8bd7_31c5748f","line":71,"updated":"2018-03-06 15:20:05.000000000","message":"Should mention here that these plugins would be owned by the QA program as far as governance is concerned, but project core teams would share core reviewer rights with the QA and RefStack core teams for the project-specific ones.","commit_id":"8d156924bb19bd3c2305750a86a4cf35280d9aa7"},{"author":{"_account_id":8099,"name":"Graham Hayes","email":"gr@ham.ie","username":"graham"},"change_message_id":"1d0949224aeba462943aa513c10876bb3a658385","unresolved":false,"context_lines":[{"line_number":68,"context_line":"In the case of **2** this plugin can be on a project by project basis, (e.g.:"},{"line_number":69,"context_line":"`glance-trademark-tempest-plugin`), an overall"},{"line_number":70,"context_line":"`openstack-trademark-tempest-plugin`, or a combination of the above (e.g.:"},{"line_number":71,"context_line":"`nfv-vertical-trademark-tempest-plugin`)."},{"line_number":72,"context_line":""},{"line_number":73,"context_line":"In these cases, the service clients in use by the tests should be either in"},{"line_number":74,"context_line":"the `Tempest repository`_ or in the `*-trademark-tempest-plugin` repository"}],"source_content_type":"text/x-rst","patch_set":3,"id":"df7087c5_f99377a2","line":71,"in_reply_to":"ff6b8bd7_31c5748f","updated":"2018-03-06 22:57:54.000000000","message":"Done","commit_id":"8d156924bb19bd3c2305750a86a4cf35280d9aa7"},{"author":{"_account_id":4257,"name":"Zane Bitter","email":"zbitter@redhat.com","username":"zaneb"},"change_message_id":"312471c40bdb24c4ae6f64e18be6bea65ebbbf95","unresolved":false,"context_lines":[{"line_number":73,"context_line":"In these cases, the service clients in use by the tests should be either in"},{"line_number":74,"context_line":"the `Tempest repository`_ or in the `*-trademark-tempest-plugin` repository"},{"line_number":75,"context_line":"to allow for easier code review that ensures that changes will not break"},{"line_number":76,"context_line":"existing certifications."},{"line_number":77,"context_line":""},{"line_number":78,"context_line":"Project teams should work with the InterOp WG to move any"},{"line_number":79,"context_line":"existing tests that need to move as a result of this policy. Project"}],"source_content_type":"text/x-rst","patch_set":3,"id":"ff6b8bd7_31733483","line":76,"updated":"2018-03-06 15:20:05.000000000","message":"Maybe generalise this to saying that the trademark program plugins should not depend on any external utilities (including service clients) that could cause breakage to interop tests, other than those in Tempest itself. i.e. projects\u0027 regular tempest plugins can import from the trademark testing plugins, but not the other way around.","commit_id":"8d156924bb19bd3c2305750a86a4cf35280d9aa7"},{"author":{"_account_id":8099,"name":"Graham Hayes","email":"gr@ham.ie","username":"graham"},"change_message_id":"73153eed74be55bc6d6104e825e6123d36f9e29b","unresolved":false,"context_lines":[{"line_number":82,"context_line":"needed for Trademark Programs, but where the test implementation is not"},{"line_number":83,"context_line":"suitable."},{"line_number":84,"context_line":""},{"line_number":85,"context_line":"Teams should work with both the InterOp WG and the QA team to ensure that"},{"line_number":86,"context_line":"adequate code review procedures are in place (especially in the case of an"},{"line_number":87,"context_line":"external plugin) to avoid any unintended breakage of Trademark program tests."},{"line_number":88,"context_line":""},{"line_number":89,"context_line":".. _QA, DefCore, and interoperability testing: https://etherpad.openstack.org/p/newton-qa-defcore-and-interoperability"},{"line_number":90,"context_line":".. _Tempest repository: http://git.openstack.org/cgit/openstack/tempest"}],"source_content_type":"text/x-rst","patch_set":3,"id":"ff6b8bd7_489f1d50","line":87,"range":{"start_line":85,"start_character":0,"end_line":87,"end_character":77},"updated":"2018-03-05 23:00:55.000000000","message":"I didn\u0027t want to codify this in a resolution, but I would suggest we do this the following way:\n\nAdd 2 new labels to the plugin repo, \"QA Verified\" and \"InterOp Verified\", both of which will be required to merge (on top of the projects own standard \"Code Review / Workflow\" labels).\n\nThat way, QA only has to review tests that actually impact trademarks, but still have good oversight, and the InterOp WG can approve the inclusion of more tests in the repo.","commit_id":"8d156924bb19bd3c2305750a86a4cf35280d9aa7"},{"author":{"_account_id":4257,"name":"Zane Bitter","email":"zbitter@redhat.com","username":"zaneb"},"change_message_id":"312471c40bdb24c4ae6f64e18be6bea65ebbbf95","unresolved":false,"context_lines":[{"line_number":82,"context_line":"needed for Trademark Programs, but where the test implementation is not"},{"line_number":83,"context_line":"suitable."},{"line_number":84,"context_line":""},{"line_number":85,"context_line":"Teams should work with both the InterOp WG and the QA team to ensure that"},{"line_number":86,"context_line":"adequate code review procedures are in place (especially in the case of an"},{"line_number":87,"context_line":"external plugin) to avoid any unintended breakage of Trademark program tests."},{"line_number":88,"context_line":""},{"line_number":89,"context_line":".. _QA, DefCore, and interoperability testing: https://etherpad.openstack.org/p/newton-qa-defcore-and-interoperability"},{"line_number":90,"context_line":".. _Tempest repository: http://git.openstack.org/cgit/openstack/tempest"}],"source_content_type":"text/x-rst","patch_set":3,"id":"ff6b8bd7_d493bb4e","line":87,"range":{"start_line":85,"start_character":0,"end_line":87,"end_character":77},"in_reply_to":"ff6b8bd7_489f1d50","updated":"2018-03-06 15:20:05.000000000","message":"Honestly that sounds like overkill to me; the text you have in the resolution is sufficient.\n\nI want to give the QA team the ability to kill changes by reviewing them. I don\u0027t, however, want to give them the ability to kill changes by *not* reviewing them. Let\u0027s not try to get too formal.","commit_id":"8d156924bb19bd3c2305750a86a4cf35280d9aa7"},{"author":{"_account_id":4257,"name":"Zane Bitter","email":"zbitter@redhat.com","username":"zaneb"},"change_message_id":"312471c40bdb24c4ae6f64e18be6bea65ebbbf95","unresolved":false,"context_lines":[{"line_number":88,"context_line":""},{"line_number":89,"context_line":".. _QA, DefCore, and interoperability testing: https://etherpad.openstack.org/p/newton-qa-defcore-and-interoperability"},{"line_number":90,"context_line":".. _Tempest repository: http://git.openstack.org/cgit/openstack/tempest"},{"line_number":91,"context_line":".. _scope of what they consider suitable for Tempest: https://wiki.openstack.org/wiki/QA/Tempest-test-removal#Tempest_Scope"},{"line_number":92,"context_line":""},{"line_number":93,"context_line":""},{"line_number":94,"context_line":""}],"source_content_type":"text/x-rst","patch_set":3,"id":"ff6b8bd7_d432dba7","line":91,"updated":"2018-03-06 15:20:05.000000000","message":"None of these links are needed afaict.","commit_id":"8d156924bb19bd3c2305750a86a4cf35280d9aa7"},{"author":{"_account_id":8099,"name":"Graham Hayes","email":"gr@ham.ie","username":"graham"},"change_message_id":"1d0949224aeba462943aa513c10876bb3a658385","unresolved":false,"context_lines":[{"line_number":88,"context_line":""},{"line_number":89,"context_line":".. _QA, DefCore, and interoperability testing: https://etherpad.openstack.org/p/newton-qa-defcore-and-interoperability"},{"line_number":90,"context_line":".. _Tempest repository: http://git.openstack.org/cgit/openstack/tempest"},{"line_number":91,"context_line":".. _scope of what they consider suitable for Tempest: https://wiki.openstack.org/wiki/QA/Tempest-test-removal#Tempest_Scope"},{"line_number":92,"context_line":""},{"line_number":93,"context_line":""},{"line_number":94,"context_line":""}],"source_content_type":"text/x-rst","patch_set":3,"id":"df7087c5_d9db137b","line":91,"in_reply_to":"ff6b8bd7_d432dba7","updated":"2018-03-06 22:57:54.000000000","message":"the scope one is not, the other 2 are used","commit_id":"8d156924bb19bd3c2305750a86a4cf35280d9aa7"},{"author":{"_account_id":8556,"name":"Ghanshyam Maan","display_name":"Ghanshyam Maan","email":"gmaan.os14@gmail.com","username":"ghanshyam"},"change_message_id":"1c99a825d32c6ae252f53b8d7b77be95e9485433","unresolved":false,"context_lines":[{"line_number":60,"context_line":"location."},{"line_number":61,"context_line":""},{"line_number":62,"context_line":"As the QA team is resource constrained, they do not have the people available"},{"line_number":63,"context_line":"to review a growing list of projects tests, which means that we should not"},{"line_number":64,"context_line":"add more tests to the `Tempest repository`_."},{"line_number":65,"context_line":""},{"line_number":66,"context_line":"The TC therefore encourages the InterOp committee to consider it an indication"},{"line_number":67,"context_line":"of future technical direction that tests will live in either:"}],"source_content_type":"text/x-rst","patch_set":4,"id":"df7087c5_e7466150","line":64,"range":{"start_line":63,"start_character":44,"end_line":64,"end_character":21},"updated":"2018-03-07 13:06:39.000000000","message":"This is opposite to what we discussed in PTG sessions in QA room. We decided to host the test after all points from projects, interop, TC to host the interop tests in Tempest plugins only.","commit_id":"db3d0611d84c92a8f50842a5d72370c043c9ee4c"},{"author":{"_account_id":8556,"name":"Ghanshyam Maan","display_name":"Ghanshyam Maan","email":"gmaan.os14@gmail.com","username":"ghanshyam"},"change_message_id":"5fca1ee6206d00cc5f1779e39e5fb609f6932c9b","unresolved":false,"context_lines":[{"line_number":60,"context_line":"location."},{"line_number":61,"context_line":""},{"line_number":62,"context_line":"As the QA team is resource constrained, they do not have the people available"},{"line_number":63,"context_line":"to review a growing list of projects tests, which means that we should not"},{"line_number":64,"context_line":"add more tests to the `Tempest repository`_."},{"line_number":65,"context_line":""},{"line_number":66,"context_line":"The TC therefore encourages the InterOp committee to consider it an indication"},{"line_number":67,"context_line":"of future technical direction that tests will live in either:"}],"source_content_type":"text/x-rst","patch_set":4,"id":"df7087c5_fffc9c3f","line":64,"range":{"start_line":63,"start_character":44,"end_line":64,"end_character":21},"in_reply_to":"df7087c5_addcc718","updated":"2018-03-07 23:11:25.000000000","message":"Yea that\u0027s true that Tempest would not be able to have gabbi dependency. And so it was agreed by all on Wed sessions with dhellmann that any tests coming to tempest should be tempest like test.","commit_id":"db3d0611d84c92a8f50842a5d72370c043c9ee4c"},{"author":{"_account_id":4257,"name":"Zane Bitter","email":"zbitter@redhat.com","username":"zaneb"},"change_message_id":"d3c44f32286f8b77f8f8a7afbc39770156e3bdad","unresolved":false,"context_lines":[{"line_number":60,"context_line":"location."},{"line_number":61,"context_line":""},{"line_number":62,"context_line":"As the QA team is resource constrained, they do not have the people available"},{"line_number":63,"context_line":"to review a growing list of projects tests, which means that we should not"},{"line_number":64,"context_line":"add more tests to the `Tempest repository`_."},{"line_number":65,"context_line":""},{"line_number":66,"context_line":"The TC therefore encourages the InterOp committee to consider it an indication"},{"line_number":67,"context_line":"of future technical direction that tests will live in either:"}],"source_content_type":"text/x-rst","patch_set":4,"id":"df7087c5_addcc718","line":64,"range":{"start_line":63,"start_character":44,"end_line":64,"end_character":21},"in_reply_to":"df7087c5_e7466150","updated":"2018-03-07 18:13:57.000000000","message":"Unfortunately I wasn\u0027t at that discussion, but if the outcome from that as Graham understood it (Tempest would accept all interop tests in-tree) had stuck then that would work fine. However, by Friday Andrea was already walking it back, saying that the QA team would refuse to accept tests from Heat already approved by the board because you don\u0027t want a dependency on gabbi in Tempest. (And I see that today you retroactively edited the etherpad to substitute that conclusion.) So, here we are.","commit_id":"db3d0611d84c92a8f50842a5d72370c043c9ee4c"},{"author":{"_account_id":11564,"name":"Chris Dent","email":"cdent@anticdent.org","username":"chdent"},"change_message_id":"5fc69f48b3ad700515eb0d27530d18972616c1cc","unresolved":false,"context_lines":[{"line_number":64,"context_line":"add more tests to the `Tempest repository`_."},{"line_number":65,"context_line":""},{"line_number":66,"context_line":"The TC therefore encourages the InterOp committee to consider it an indication"},{"line_number":67,"context_line":"of future technical direction that tests will live in either:"},{"line_number":68,"context_line":""},{"line_number":69,"context_line":"1. The `Tempest repository`_"},{"line_number":70,"context_line":"2. A tempest plugin that contains only Trademark program tests or tests that"}],"source_content_type":"text/x-rst","patch_set":4,"id":"df7087c5_18ae5c2e","line":67,"range":{"start_line":67,"start_character":35,"end_line":67,"end_character":40},"updated":"2018-03-07 10:10:04.000000000","message":"May wish to be explicit here and say that these are trademark/interop tests, not all tests.\n\nJust in case we end up doing yet more lawyering...","commit_id":"db3d0611d84c92a8f50842a5d72370c043c9ee4c"},{"author":{"_account_id":8556,"name":"Ghanshyam Maan","display_name":"Ghanshyam Maan","email":"gmaan.os14@gmail.com","username":"ghanshyam"},"change_message_id":"1c99a825d32c6ae252f53b8d7b77be95e9485433","unresolved":false,"context_lines":[{"line_number":75,"context_line":"`openstack-trademark-tempest-plugin`, or a combination of the above (e.g.:"},{"line_number":76,"context_line":"`nfv-vertical-trademark-tempest-plugin`)."},{"line_number":77,"context_line":""},{"line_number":78,"context_line":"These plugin repos should be owned by the QA team, but have a shared core code"},{"line_number":79,"context_line":"reviewer group from both the project and QA."},{"line_number":80,"context_line":""},{"line_number":81,"context_line":"Trademark program plugins should not depend on any external utilities"}],"source_content_type":"text/x-rst","patch_set":4,"id":"df7087c5_a79f09b3","line":78,"range":{"start_line":78,"start_character":0,"end_line":78,"end_character":49},"updated":"2018-03-07 13:06:39.000000000","message":"I am not sure how this will work and how it will solve the problem mentioned at L62 about QA team have limited bandwidth. \n\nMaintaining separate repo by QA team is more difficult than maintaining tests in Tempest repo.","commit_id":"db3d0611d84c92a8f50842a5d72370c043c9ee4c"},{"author":{"_account_id":11564,"name":"Chris Dent","email":"cdent@anticdent.org","username":"chdent"},"change_message_id":"5fc69f48b3ad700515eb0d27530d18972616c1cc","unresolved":false,"context_lines":[{"line_number":75,"context_line":"`openstack-trademark-tempest-plugin`, or a combination of the above (e.g.:"},{"line_number":76,"context_line":"`nfv-vertical-trademark-tempest-plugin`)."},{"line_number":77,"context_line":""},{"line_number":78,"context_line":"These plugin repos should be owned by the QA team, but have a shared core code"},{"line_number":79,"context_line":"reviewer group from both the project and QA."},{"line_number":80,"context_line":""},{"line_number":81,"context_line":"Trademark program plugins should not depend on any external utilities"}],"source_content_type":"text/x-rst","patch_set":4,"id":"df7087c5_78199886","line":78,"range":{"start_line":78,"start_character":29,"end_line":78,"end_character":49},"updated":"2018-03-07 10:10:04.000000000","message":"I do not think this is a good idea as it causes a bit of startup roadblock and distributes ownership away from the two parties that care about the tests the most: the project and the trademark program.\n\nI think:\n\n* If the repo is single service, it should be owned by the project that delivers that service.\n* If the repo is for a vertical or global trademark it should be owned by the trademark program","commit_id":"db3d0611d84c92a8f50842a5d72370c043c9ee4c"},{"author":{"_account_id":4257,"name":"Zane Bitter","email":"zbitter@redhat.com","username":"zaneb"},"change_message_id":"d3c44f32286f8b77f8f8a7afbc39770156e3bdad","unresolved":false,"context_lines":[{"line_number":75,"context_line":"`openstack-trademark-tempest-plugin`, or a combination of the above (e.g.:"},{"line_number":76,"context_line":"`nfv-vertical-trademark-tempest-plugin`)."},{"line_number":77,"context_line":""},{"line_number":78,"context_line":"These plugin repos should be owned by the QA team, but have a shared core code"},{"line_number":79,"context_line":"reviewer group from both the project and QA."},{"line_number":80,"context_line":""},{"line_number":81,"context_line":"Trademark program plugins should not depend on any external utilities"}],"source_content_type":"text/x-rst","patch_set":4,"id":"df7087c5_dc6c4b9e","line":78,"range":{"start_line":78,"start_character":0,"end_line":78,"end_character":49},"in_reply_to":"df7087c5_a79f09b3","updated":"2018-03-07 18:13:57.000000000","message":"It does a number of things to help.\n\nFor one, it provides these plugin repos with more core reviewers, so that e.g. housekeeping-type changes won\u0027t be blocked on QA cores.\n\nAnother is that it avoids mixing interop and non-interop tests in the same repo. This means 2 things: first, that the tests themselves will basically never change and the review burden will be quite low other than adding new tests (which is followed by a board approval process, so is fairly rare). Second, reviewers don\u0027t need the expert knowledge of tempest required to figure out which parts are interop-critical and which are not, since they all are.\n\nThis is also a more scalable approach for the future than \u0027throw everything in tempest\u0027, since there\u0027s no reason to think that the number of add-on trademark programs will be capped at 2.","commit_id":"db3d0611d84c92a8f50842a5d72370c043c9ee4c"},{"author":{"_account_id":11564,"name":"Chris Dent","email":"cdent@anticdent.org","username":"chdent"},"change_message_id":"5fc69f48b3ad700515eb0d27530d18972616c1cc","unresolved":false,"context_lines":[{"line_number":76,"context_line":"`nfv-vertical-trademark-tempest-plugin`)."},{"line_number":77,"context_line":""},{"line_number":78,"context_line":"These plugin repos should be owned by the QA team, but have a shared core code"},{"line_number":79,"context_line":"reviewer group from both the project and QA."},{"line_number":80,"context_line":""},{"line_number":81,"context_line":"Trademark program plugins should not depend on any external utilities"},{"line_number":82,"context_line":"(including service clients) that could cause breakage to interop tests, other"}],"source_content_type":"text/x-rst","patch_set":4,"id":"df7087c5_d803040f","line":79,"range":{"start_line":79,"start_character":15,"end_line":79,"end_character":43},"updated":"2018-03-07 10:10:04.000000000","message":"This leaves out a critical party that we discussed on Friday: the trademark group. Shouldn\u0027t they must review power too?","commit_id":"db3d0611d84c92a8f50842a5d72370c043c9ee4c"},{"author":{"_account_id":4257,"name":"Zane Bitter","email":"zbitter@redhat.com","username":"zaneb"},"change_message_id":"d3c44f32286f8b77f8f8a7afbc39770156e3bdad","unresolved":false,"context_lines":[{"line_number":76,"context_line":"`nfv-vertical-trademark-tempest-plugin`)."},{"line_number":77,"context_line":""},{"line_number":78,"context_line":"These plugin repos should be owned by the QA team, but have a shared core code"},{"line_number":79,"context_line":"reviewer group from both the project and QA."},{"line_number":80,"context_line":""},{"line_number":81,"context_line":"Trademark program plugins should not depend on any external utilities"},{"line_number":82,"context_line":"(including service clients) that could cause breakage to interop tests, other"}],"source_content_type":"text/x-rst","patch_set":4,"id":"df7087c5_b03660b4","line":79,"range":{"start_line":79,"start_character":15,"end_line":79,"end_character":43},"in_reply_to":"df7087c5_07c5958e","updated":"2018-03-07 18:13:57.000000000","message":"\u003e This leaves out a critical party that we discussed on Friday: the trademark group. Shouldn\u0027t they must review power too?\n\nI believe that was intended to be part of the proposal.\n\n\u003e If we share the review core power than how it will take care of each responsibility ?\n\nThe same way all core review teams comprising more than one person share this responsibility already? This proposal is only specifying how the core review teams for these repos will be selected (by combining the project and QA core teams, and the RefStack team).\n\n\u003e I mean if you need input from QA then it can be with +1.\n\nSometimes you might want input with -2. Or -A.","commit_id":"db3d0611d84c92a8f50842a5d72370c043c9ee4c"},{"author":{"_account_id":8556,"name":"Ghanshyam Maan","display_name":"Ghanshyam Maan","email":"gmaan.os14@gmail.com","username":"ghanshyam"},"change_message_id":"1c99a825d32c6ae252f53b8d7b77be95e9485433","unresolved":false,"context_lines":[{"line_number":76,"context_line":"`nfv-vertical-trademark-tempest-plugin`)."},{"line_number":77,"context_line":""},{"line_number":78,"context_line":"These plugin repos should be owned by the QA team, but have a shared core code"},{"line_number":79,"context_line":"reviewer group from both the project and QA."},{"line_number":80,"context_line":""},{"line_number":81,"context_line":"Trademark program plugins should not depend on any external utilities"},{"line_number":82,"context_line":"(including service clients) that could cause breakage to interop tests, other"}],"source_content_type":"text/x-rst","patch_set":4,"id":"df7087c5_07c5958e","line":79,"range":{"start_line":79,"start_character":15,"end_line":79,"end_character":43},"in_reply_to":"df7087c5_d803040f","updated":"2018-03-07 13:06:39.000000000","message":"If we share the review core power than how it will take care of each responsibility ? I mean if you need input from QA then it can be with +1.","commit_id":"db3d0611d84c92a8f50842a5d72370c043c9ee4c"},{"author":{"_account_id":2472,"name":"Doug Hellmann","email":"dhellmann@redhat.com","username":"doug-hellmann"},"change_message_id":"07202f51ff509974d007400242ce038154dff6b3","unresolved":false,"context_lines":[{"line_number":81,"context_line":"Trademark program plugins should not depend on any external utilities"},{"line_number":82,"context_line":"(including service clients) that could cause breakage to interop tests, other"},{"line_number":83,"context_line":"than those in Tempest itself. For example projects\u0027 regular tempest plugins"},{"line_number":84,"context_line":"can import from the trademark testing plugins, but not the other way around."},{"line_number":85,"context_line":""},{"line_number":86,"context_line":"Project teams should work with the InterOp WG to move any"},{"line_number":87,"context_line":"existing tests that need to move as a result of this policy. Project"}],"source_content_type":"text/x-rst","patch_set":4,"id":"df7087c5_d0ed1426","line":84,"updated":"2018-03-07 15:34:36.000000000","message":"Does this include gabbi?","commit_id":"db3d0611d84c92a8f50842a5d72370c043c9ee4c"},{"author":{"_account_id":4257,"name":"Zane Bitter","email":"zbitter@redhat.com","username":"zaneb"},"change_message_id":"d3c44f32286f8b77f8f8a7afbc39770156e3bdad","unresolved":false,"context_lines":[{"line_number":81,"context_line":"Trademark program plugins should not depend on any external utilities"},{"line_number":82,"context_line":"(including service clients) that could cause breakage to interop tests, other"},{"line_number":83,"context_line":"than those in Tempest itself. For example projects\u0027 regular tempest plugins"},{"line_number":84,"context_line":"can import from the trademark testing plugins, but not the other way around."},{"line_number":85,"context_line":""},{"line_number":86,"context_line":"Project teams should work with the InterOp WG to move any"},{"line_number":87,"context_line":"existing tests that need to move as a result of this policy. Project"}],"source_content_type":"text/x-rst","patch_set":4,"id":"df7087c5_10186c1b","line":84,"in_reply_to":"df7087c5_d0ed1426","updated":"2018-03-07 18:13:57.000000000","message":"It was not intended to forbid gabbi, but I agree that \u0027utilities\u0027 is the wrong word here (I couldn\u0027t think of anything better at the time). The point is that people innocently changing code in a non-interop-ringfenced tempest plugin shouldn\u0027t be able to break an interop plugin, because we don\u0027t allow dependencies to run in that direction. If we want to write code that is critical to the stability of interop tests, we should put it in the Tempest repository or a plugin specifically for the interop tests.","commit_id":"db3d0611d84c92a8f50842a5d72370c043c9ee4c"},{"author":{"_account_id":4257,"name":"Zane Bitter","email":"zbitter@redhat.com","username":"zaneb"},"change_message_id":"767983570a673428359fb65e4c5fa29802ac1bc0","unresolved":false,"context_lines":[{"line_number":95,"context_line":"external plugin) to avoid any unintended breakage of Trademark program tests."},{"line_number":96,"context_line":""},{"line_number":97,"context_line":".. _QA, DefCore, and interoperability testing: https://etherpad.openstack.org/p/newton-qa-defcore-and-interoperability"},{"line_number":98,"context_line":".. _Tempest repository: http://git.openstack.org/cgit/openstack/tempest"}],"source_content_type":"text/x-rst","patch_set":4,"id":"df7087c5_3c4a3d80","line":98,"updated":"2018-03-07 00:43:56.000000000","message":"FWIW if you don\u0027t convert this to https I guarantee some muppet trolling for internet points will submit a review to \u0027fix\u0027 it ;)","commit_id":"db3d0611d84c92a8f50842a5d72370c043c9ee4c"}]}