)]}'
{"/COMMIT_MSG":[{"author":{"_account_id":8873,"name":"Assaf Muller","email":"amuller@redhat.com","username":"amuller"},"change_message_id":"1b5586ddefaa47ba0e4361d751d47caa23dcfb5f","unresolved":false,"context_lines":[{"line_number":16,"context_line":"nature of the deployment, since tempest is an external test suite that"},{"line_number":17,"context_line":"interacts with any deployment only through the api. The configuration"},{"line_number":18,"context_line":"and run mechanics do not change whether your cloud is 1 or multiple"},{"line_number":19,"context_line":"nodes. The real difficulty lies in setting up a multinode deployment."},{"line_number":20,"context_line":""},{"line_number":21,"context_line":"For the failure reporting, if you can\u0027t figure out why something"},{"line_number":22,"context_line":"failed from a tempest run it\u0027s the same for any end user of the API."}],"source_content_type":"text/x-gerrit-commit-message","patch_set":1,"id":"1a4dcd0f_cebf185f","line":19,"updated":"2015-08-14 16:49:46.000000000","message":"You are correct in the sense that Tempest can run against a cloud weather its all-in-one or multi-node. The distinction still needs to be made: Tempest requires an opertional cloud, which assumes some sort of deployment tool (The gate uses devstack) while the fullstack testing infrastructure takes care of deployment itself, according to the topology that the test requires, thereby is a factor easier for developers. I cannot stress how important this point is.\n\nI would ask you to reword the text you removed in the .rst. Instead of saying that Tempest is difficult to run against a multi node cloud (Which, I agree it is not as the problem is not Tempest), it should say that fullstack deploys the test topology itself, and is an enabler for developers that want to easily test inherently multinode features.","commit_id":"1d28221e87151a26f85f9ac41fa686a41b8b488f"},{"author":{"_account_id":5196,"name":"Matthew Treinish","email":"mtreinish@kortar.org","username":"treinish"},"change_message_id":"5f90776931974987f7313d83f5e1b4df045edc39","unresolved":false,"context_lines":[{"line_number":16,"context_line":"nature of the deployment, since tempest is an external test suite that"},{"line_number":17,"context_line":"interacts with any deployment only through the api. The configuration"},{"line_number":18,"context_line":"and run mechanics do not change whether your cloud is 1 or multiple"},{"line_number":19,"context_line":"nodes. The real difficulty lies in setting up a multinode deployment."},{"line_number":20,"context_line":""},{"line_number":21,"context_line":"For the failure reporting, if you can\u0027t figure out why something"},{"line_number":22,"context_line":"failed from a tempest run it\u0027s the same for any end user of the API."}],"source_content_type":"text/x-gerrit-commit-message","patch_set":1,"id":"1a4dcd0f_098c1a7e","line":19,"in_reply_to":"1a4dcd0f_cebf185f","updated":"2015-08-14 16:54:23.000000000","message":"Yep I agree, that\u0027s why I left the part about the ease for developers and the benefits of fullstack doing that at the end of the paragraph. I\u0027ll respin this in a bit to reword the removed piece to make the distinction more clear.","commit_id":"1d28221e87151a26f85f9ac41fa686a41b8b488f"}],"doc/source/devref/fullstack_testing.rst":[{"author":{"_account_id":8873,"name":"Assaf Muller","email":"amuller@redhat.com","username":"amuller"},"change_message_id":"1b5586ddefaa47ba0e4361d751d47caa23dcfb5f","unresolved":false,"context_lines":[{"line_number":30,"context_line":"The idea behind \"fullstack\" testing is to fill a gap between unit + functional"},{"line_number":31,"context_line":"tests and Tempest. Tempest tests are expensive to run, and operate only"},{"line_number":32,"context_line":"through the REST API. So they can only provide an explanation of what went wrong"},{"line_number":33,"context_line":"through gets reported to an end user through the REST API, which is often too"},{"line_number":34,"context_line":"high level. Developers further benefit from full stack testing as it can"},{"line_number":35,"context_line":"sufficiently simulate a real environment and provide a rapidly reproducible way"},{"line_number":36,"context_line":"to verify code as you\u0027re still writing it."}],"source_content_type":"text/x-rst","patch_set":1,"id":"1a4dcd0f_cef1586c","line":33,"updated":"2015-08-14 16:49:46.000000000","message":"Something is off in this sentence: \u0027...what went wrong through gets reported...\u0027\n\nMaybe: \u0027...what went wrong by what gets reported via the REST API,...\u0027","commit_id":"1d28221e87151a26f85f9ac41fa686a41b8b488f"},{"author":{"_account_id":5196,"name":"Matthew Treinish","email":"mtreinish@kortar.org","username":"treinish"},"change_message_id":"6b2bba25b66380afed22ea8c04b391d98840df72","unresolved":false,"context_lines":[{"line_number":30,"context_line":"The idea behind \"fullstack\" testing is to fill a gap between unit + functional"},{"line_number":31,"context_line":"tests and Tempest. Tempest tests are expensive to run, and operate only"},{"line_number":32,"context_line":"through the REST API. So they can only provide an explanation of what went wrong"},{"line_number":33,"context_line":"through gets reported to an end user through the REST API, which is often too"},{"line_number":34,"context_line":"high level. Developers further benefit from full stack testing as it can"},{"line_number":35,"context_line":"sufficiently simulate a real environment and provide a rapidly reproducible way"},{"line_number":36,"context_line":"to verify code as you\u0027re still writing it."}],"source_content_type":"text/x-rst","patch_set":1,"id":"1a4dcd0f_996cf337","line":33,"in_reply_to":"1a4dcd0f_cef1586c","updated":"2015-08-14 19:17:29.000000000","message":"Done","commit_id":"1d28221e87151a26f85f9ac41fa686a41b8b488f"},{"author":{"_account_id":6951,"name":"Brandon Logan","email":"brandon.logan@rackspace.com","username":"brandon-logan"},"change_message_id":"31da22c045c5e76d6f672b095cab57bc9521584a","unresolved":false,"context_lines":[{"line_number":30,"context_line":"The idea behind \"fullstack\" testing is to fill a gap between unit + functional"},{"line_number":31,"context_line":"tests and Tempest. Tempest tests are expensive to run, and operate only"},{"line_number":32,"context_line":"through the REST API. So they can only provide an explanation of what went wrong"},{"line_number":33,"context_line":"gets reported to an end user via the REST API, which is often too high level."},{"line_number":34,"context_line":"Additionally, Tempest requires an OpenStack deployment to be run against, which"},{"line_number":35,"context_line":"can be difficult to configure and setup. The full stack testing addresses"},{"line_number":36,"context_line":"these issues by taking care of the deployment itself, according to the topology"}],"source_content_type":"text/x-rst","patch_set":2,"id":"1a4dcd0f_7c1475e8","line":33,"updated":"2015-08-14 19:33:14.000000000","message":"this sentence reads a bit odd.  Perhaps the word \"that\" between \"wrong\" and \"gets\"?  Or just remove the \"gets reported\".","commit_id":"fdd96e9a00cdb537217de51488aa59890564d0a8"},{"author":{"_account_id":17096,"name":"Aditi Rajagopal","email":"arajagopal@us.ibm.com","username":"aditirajagopal"},"change_message_id":"7dbff0db874359dc24e1587c9ec4bb03d18bce79","unresolved":false,"context_lines":[{"line_number":37,"context_line":"that the test requires. Developers further benefit from full stack testing as"},{"line_number":38,"context_line":"it can sufficiently simulate a real environment and provide a rapidly"},{"line_number":39,"context_line":"reproducible way to verify code as you\u0027re still writing it."},{"line_number":40,"context_line":""},{"line_number":41,"context_line":"How?"},{"line_number":42,"context_line":"----"},{"line_number":43,"context_line":""}],"source_content_type":"text/x-rst","patch_set":2,"id":"fa1b9901_6ebeecd5","line":40,"updated":"2015-08-18 17:55:42.000000000","message":"Considering the previous feedback, here is a suggested re-write for this section:\n\n\"The goal of full stack testing is to bridge the gap between unit, functional and tempest tests. Tempest tests are expensive to run, and only operate through the REST API. As a result, tempest tests can only provide a high-level explanation of what went wrong via the REST API. Additionally, tempest requires an OpenStack deployment in order to run, which can be difficult to configure/setup to the user. Overall, the cost to run tempest tests outweighs the benefits. \n\nFull stack testing addresses these issues:\n1) Full stack sets up and configures OpenStack deployments according to the topology associated with each test\n2) Full stack testing sufficiently simulates a real environment, and provides a rapidly reproducible way to verify code as the developer is writing it\"","commit_id":"fdd96e9a00cdb537217de51488aa59890564d0a8"}]}