)]}'
{"specs/backlog/functional-testing-setup.rst":[{"author":{"_account_id":6804,"name":"bruce-benjamin","email":"bruce.benjamin@jhuapl.edu","username":"bruce-benjamin"},"change_message_id":"983ab0b67c7f4b75306462a40a4648a60b2aa3e2","unresolved":false,"context_lines":[{"line_number":74,"context_line":"  the KeystoneClient API"},{"line_number":75,"context_line":""},{"line_number":76,"context_line":"shared tests"},{"line_number":77,"context_line":"  the scenario tests the show the base behavior of the system; these will"},{"line_number":78,"context_line":"  run under every configuration"},{"line_number":79,"context_line":""},{"line_number":80,"context_line":"Basic idea"}],"source_content_type":"text/x-rst","patch_set":2,"id":"9aa53dc9_98e36e33","line":77,"updated":"2015-04-08 23:26:27.000000000","message":"Did you mean to say \"tests to show\" here?","commit_id":"c475ce77bacb2902bfb2028f0d6fa40ca7f121a6"}],"specs/kilo/functional-testing-setup.rst":[{"author":{"_account_id":5196,"name":"Matthew Treinish","email":"mtreinish@kortar.org","username":"treinish"},"change_message_id":"536c0a2908402b57a386ffc69d2fb3917040ec01","unresolved":false,"context_lines":[{"line_number":12,"context_line":"\u003chttps://blueprints.launchpad.net/keystone/+spec/functional-testing-setup\u003e`_"},{"line_number":13,"context_line":""},{"line_number":14,"context_line":""},{"line_number":15,"context_line":"The new direction for functional testing is to `move`_ it out of tempest and"},{"line_number":16,"context_line":"into the projects. This means that we need to provide direction and some sort"},{"line_number":17,"context_line":"of framework for how this is to be done in Keystone."},{"line_number":18,"context_line":""}],"source_content_type":"text/x-rst","patch_set":1,"id":"ba7be1f8_a8a16547","line":15,"updated":"2015-02-26 16:43:18.000000000","message":"Well, to be fair functional testing was never really supposed to be in tempest, it\u0027s just that tempest was the only place most projects had a full running stack in the gate. \n\nThe push is really to fill the gap between unit tests and tempest tests. In the process if there are tests in tempest which probably are more proper as a project specific functional test we can move it. \n\nThe process is being documented here: https://wiki.openstack.org/wiki/QA/Tempest-test-removal","commit_id":"26a95003a8493ae6d49301d47ad52d3cc0e6a72a"},{"author":{"_account_id":5196,"name":"Matthew Treinish","email":"mtreinish@kortar.org","username":"treinish"},"change_message_id":"536c0a2908402b57a386ffc69d2fb3917040ec01","unresolved":false,"context_lines":[{"line_number":57,"context_line":"  the box"},{"line_number":58,"context_line":"* it should be very easy to add new configurations that need to be tested"},{"line_number":59,"context_line":"* the tests should be runnable by the gate for enforcement"},{"line_number":60,"context_line":"* the tests should be runnable by developers using devstack"},{"line_number":61,"context_line":""},{"line_number":62,"context_line":"Concepts"},{"line_number":63,"context_line":"--------"}],"source_content_type":"text/x-rst","patch_set":1,"id":"ba7be1f8_15dfe5d6","line":60,"updated":"2015-02-26 16:43:18.000000000","message":"So while I agree it should be runnable by developers, I\u0027m not sure having a running devstack is a requirement. For example, bits that require a db and the api layer to test db interactions directly should require a db and spin up the api layer on it\u0027s own. (although that probably also requires apache) While this would all work in devstack, I think saying devstack is required for all functional testing might be a bit too heavyweight.","commit_id":"26a95003a8493ae6d49301d47ad52d3cc0e6a72a"},{"author":{"_account_id":7725,"name":"David Stanek","email":"dstanek@dstanek.com","username":"dstanek"},"change_message_id":"be69cc117cf39a4502ff2992e091bb6f033f3531","unresolved":false,"context_lines":[{"line_number":57,"context_line":"  the box"},{"line_number":58,"context_line":"* it should be very easy to add new configurations that need to be tested"},{"line_number":59,"context_line":"* the tests should be runnable by the gate for enforcement"},{"line_number":60,"context_line":"* the tests should be runnable by developers using devstack"},{"line_number":61,"context_line":""},{"line_number":62,"context_line":"Concepts"},{"line_number":63,"context_line":"--------"}],"source_content_type":"text/x-rst","patch_set":1,"id":"9a80dd14_4874ad35","line":60,"in_reply_to":"ba7be1f8_15dfe5d6","updated":"2015-03-08 04:02:09.000000000","message":"I added another line to say that when possible the tests shouldn\u0027t need devstack. Certain tests, like federation, will need to have a real configured server or a devstack server.","commit_id":"26a95003a8493ae6d49301d47ad52d3cc0e6a72a"},{"author":{"_account_id":5196,"name":"Matthew Treinish","email":"mtreinish@kortar.org","username":"treinish"},"change_message_id":"536c0a2908402b57a386ffc69d2fb3917040ec01","unresolved":false,"context_lines":[{"line_number":73,"context_line":""},{"line_number":74,"context_line":"shared tests"},{"line_number":75,"context_line":"  the scenario tests the show the base behavior of the system; these will"},{"line_number":76,"context_line":"  run under every configuration"},{"line_number":77,"context_line":""},{"line_number":78,"context_line":"Basic idea"},{"line_number":79,"context_line":"----------"}],"source_content_type":"text/x-rst","patch_set":1,"id":"ba7be1f8_55695df7","line":76,"updated":"2015-02-26 16:43:18.000000000","message":"So these tests would probably still be in scope of tempest. High level tests which aren\u0027t configuration specific that only interact through the api is kinda what tempest does. I\u0027m not sure there is value in reinventing that inside of keystone. You could just have a targeted tempest job which only runs the keystone subset of tempest.","commit_id":"26a95003a8493ae6d49301d47ad52d3cc0e6a72a"},{"author":{"_account_id":7725,"name":"David Stanek","email":"dstanek@dstanek.com","username":"dstanek"},"change_message_id":"be69cc117cf39a4502ff2992e091bb6f033f3531","unresolved":false,"context_lines":[{"line_number":73,"context_line":""},{"line_number":74,"context_line":"shared tests"},{"line_number":75,"context_line":"  the scenario tests the show the base behavior of the system; these will"},{"line_number":76,"context_line":"  run under every configuration"},{"line_number":77,"context_line":""},{"line_number":78,"context_line":"Basic idea"},{"line_number":79,"context_line":"----------"}],"source_content_type":"text/x-rst","patch_set":1,"id":"9a80dd14_c897bd38","line":76,"in_reply_to":"ba7be1f8_55695df7","updated":"2015-03-08 04:02:09.000000000","message":"This maybe something to discuss further. The intent here is really to take some of the tests we already have in tree and repurpose them. We call them unit tests, but they first up a server for each one.\n\nI thought tempest was going to be used for just the integration tests. Is there a guideline for what belongs in tempest vs. in tree?","commit_id":"26a95003a8493ae6d49301d47ad52d3cc0e6a72a"}]}
