)]}'
{"specs/remove-duplicated-negative-tests.rst":[{"author":{"_account_id":5196,"name":"Matthew Treinish","email":"mtreinish@kortar.org","username":"treinish"},"change_message_id":"846cdefe3fb9eabf2eb74bdfd936c29bcfbef844","unresolved":false,"context_lines":[{"line_number":4,"context_line":" http://creativecommons.org/licenses/by/3.0/legalcode"},{"line_number":5,"context_line":""},{"line_number":6,"context_line":".."},{"line_number":7,"context_line":"  This template should be in ReSTructured text. The filename in the git"},{"line_number":8,"context_line":"  repository should match the launchpad URL, for example a URL of"},{"line_number":9,"context_line":"  https://blueprints.launchpad.net/tempest/+spec/awesome-thing should be named"},{"line_number":10,"context_line":"  awesome-thing.rst .  Please do not delete any of the sections in this"}],"source_content_type":"text/x-rst","patch_set":4,"id":"5a890539_70cb4172","line":7,"updated":"2014-12-02 19:46:07.000000000","message":"You don\u0027t need this block","commit_id":"0ab039fa32057769aec8d5300a1573ab35414548"},{"author":{"_account_id":5196,"name":"Matthew Treinish","email":"mtreinish@kortar.org","username":"treinish"},"change_message_id":"846cdefe3fb9eabf2eb74bdfd936c29bcfbef844","unresolved":false,"context_lines":[{"line_number":31,"context_line":"when receiving a bad request. However, most these behaviors are implemented"},{"line_number":32,"context_line":"on the surface of each API. So they are not integrated tests. In addition,"},{"line_number":33,"context_line":"many negative tests are duplicated between Tempest and unit tests on each"},{"line_number":34,"context_line":"project. Now, we are spending much time on the gate due to duplicated tests."},{"line_number":35,"context_line":"For example, test_reset_state_server_invalid_state of Tempest is duplicated"},{"line_number":36,"context_line":"to test_bad_state of Nova unit tests."},{"line_number":37,"context_line":""}],"source_content_type":"text/x-rst","patch_set":4,"id":"5a890539_90734d15","line":34,"updated":"2014-12-02 19:46:07.000000000","message":"So I actually disagree about this being a matter of run time. The negative tests you\u0027re talking about removing here are executed very quickly. You probably could take the run time of all of them serially and it wouldn\u0027t even match some of the scenario tests. \n\nWhat I feel the real issue here is code complexity and mixed messaging for contributors around what a good negative test is.","commit_id":"0ab039fa32057769aec8d5300a1573ab35414548"},{"author":{"_account_id":5196,"name":"Matthew Treinish","email":"mtreinish@kortar.org","username":"treinish"},"change_message_id":"846cdefe3fb9eabf2eb74bdfd936c29bcfbef844","unresolved":false,"context_lines":[{"line_number":62,"context_line":"tests to each project when we decide it. In addition, we need to clarify the"},{"line_number":63,"context_line":"moratorium of new negative tests on Tempest and suggest to add these tests to"},{"line_number":64,"context_line":"each project directly. We have already declared the moratorium 1.5 year ago[2],"},{"line_number":65,"context_line":"but we need to do it again because of long time."},{"line_number":66,"context_line":""},{"line_number":67,"context_line":"The patches which remove duplicated tests should contain the duplicating info"},{"line_number":68,"context_line":"like:"}],"source_content_type":"text/x-rst","patch_set":4,"id":"5a890539_0bb1ec00","line":65,"updated":"2014-12-02 19:46:07.000000000","message":"So is this just a matter of enforcement? Have we been lax in enforcing this rule? I\u0027m fine with redclaring it but I just want to understand the rationale so we can make the language more clear. We have been granting exceptions to projects which had no api coverage I guess we can stop that.\n\n\nAlso, I think the 3 pronged rule I proposed in Paris for test removal should still apply here:\n\nhttps://etherpad.openstack.org/p/kilo-summit-tempest-scope-in-the-brave-new-world\n\nI understand the desire to do this quickly, but I think we should follow the same procedure for all the test deduplication. (I also doubt there will be an issue with 2 and 3 for negative tests)\n\nFWIW, we have subunit2sql running in infra so we can track the failure rates now we just need to add the mechanism to make the data public.","commit_id":"0ab039fa32057769aec8d5300a1573ab35414548"},{"author":{"_account_id":5196,"name":"Matthew Treinish","email":"mtreinish@kortar.org","username":"treinish"},"change_message_id":"846cdefe3fb9eabf2eb74bdfd936c29bcfbef844","unresolved":false,"context_lines":[{"line_number":99,"context_line":"Work Items"},{"line_number":100,"context_line":"----------"},{"line_number":101,"context_line":""},{"line_number":102,"context_line":"* Declare the moratorium of new negative tests on Tempest"},{"line_number":103,"context_line":"* Remove negative tests, which are duplicated between Tempest and unit tests"},{"line_number":104,"context_line":"  of each project, from Tempest."},{"line_number":105,"context_line":""}],"source_content_type":"text/x-rst","patch_set":4,"id":"5a890539_8bc4fc60","line":102,"updated":"2014-12-02 19:46:07.000000000","message":"In what format did you want to do this again? Another ML post? I think we should also add a section to the openstack developer docs (hacking.rst and reviewing.rst probably) to outline this rule again.","commit_id":"0ab039fa32057769aec8d5300a1573ab35414548"}]}