)]}'
{"specs/proposed/refstack-org-tcup-base.rst":[{"author":{"_account_id":8434,"name":"Rochelle Grober","email":"rochelle.grober@huawei.com","username":"rockyg"},"change_message_id":"18362541f7e97cb9dbaf243d8e78adeb6671bebf","unresolved":false,"context_lines":[{"line_number":31,"context_line":""},{"line_number":32,"context_line":"TCUP should be designed in as simple a way as possible."},{"line_number":33,"context_line":""},{"line_number":34,"context_line":"Running TCUP should only require Docker (.9+), a single tcup.py file with minimal dependencies, working OpenStack cloud credentials and an Internet connection."},{"line_number":35,"context_line":""},{"line_number":36,"context_line":"Environment variables from the host (OS-*) will be passed into the container.  The container should not start unless critical OS_ variables are in place."},{"line_number":37,"context_line":""}],"source_content_type":"text/x-rst","patch_set":1,"id":"AAAAXH%2F%2FRX8%3D","line":34,"updated":"2014-04-22 21:10:58.000000000","message":"I thought we had agreed that an internet connection was required to get and maybe configure TCUP, but not to run it because the private clouds might not have internet access.","commit_id":"405d3b355ec2bfbd31425bccf0fdb8fbc81012a4"},{"author":{"_account_id":8434,"name":"Rochelle Grober","email":"rochelle.grober@huawei.com","username":"rockyg"},"change_message_id":"18362541f7e97cb9dbaf243d8e78adeb6671bebf","unresolved":false,"context_lines":[{"line_number":40,"context_line":"There must be both a way to use local code (refstack) to run TCUP and also a simple file download approach.  These methods should be functionally identical."},{"line_number":41,"context_line":""},{"line_number":42,"context_line":"While there may be broad uses of TCUP for test automation, it is not desirable to overload them at the expense of manual usability.  TCUP should be very simple of users in this pass."},{"line_number":43,"context_line":""},{"line_number":44,"context_line":"Alternatives"},{"line_number":45,"context_line":"------------"},{"line_number":46,"context_line":""}],"source_content_type":"text/x-rst","patch_set":1,"id":"AAAAXH%2F%2FRSo%3D","line":43,"updated":"2014-04-22 21:10:58.000000000","message":"There should be a way to locally save each run of test results as a uniquely named file and to upload any or all of them at some future time and from a different system.\n\n** I\u0027m thinking that TCUP should not do the test results upload, but that it should be packaged with a simple script with the Upload API that could be run whenever and would first look in a known location for the test results, or would bundle the results up for shipment.  Need more discussion/inputs on this one.\n\nanother aspect of security:  The test results should be signed with a key that is on file with OpenStack.org when uploaded to Refstack.","commit_id":"405d3b355ec2bfbd31425bccf0fdb8fbc81012a4"},{"author":{"_account_id":10527,"name":"Raymond Wong","email":"waiman@us.ibm.com","username":"waiman"},"change_message_id":"9d07745de533757a7c40a8891e509a12aad3f2b2","unresolved":false,"context_lines":[{"line_number":1,"context_line":"Base TCUP environment for refstack.org"},{"line_number":2,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"},{"line_number":3,"context_line":""},{"line_number":4,"context_line":"TCUP (Tempest in a Container, Upload from Probe) is a self-contained, universal Tempest environment that can be widely used by the community with minimal effort AND minimal support effort by the Refstack team."},{"line_number":5,"context_line":""},{"line_number":6,"context_line":"Problem description"},{"line_number":7,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"}],"source_content_type":"text/x-rst","patch_set":2,"id":"AAAAXX%2F%2F%2Bdg%3D","line":4,"updated":"2014-04-24 23:05:19.000000000","message":"minor: better put Refstack in one line.","commit_id":"cb5414f263bebe64eba1def47432af601ef9acfe"},{"author":{"_account_id":10527,"name":"Raymond Wong","email":"waiman@us.ibm.com","username":"waiman"},"change_message_id":"9d07745de533757a7c40a8891e509a12aad3f2b2","unresolved":false,"context_lines":[{"line_number":15,"context_line":"Requirements:"},{"line_number":16,"context_line":""},{"line_number":17,"context_line":"* It should not matter which Linux distro they are using"},{"line_number":18,"context_line":"* Users should not have to figure out which code to check out"},{"line_number":19,"context_line":"* Users should not have to deal with packages or pips"},{"line_number":20,"context_line":"* Users should not have to determine where to upload their results"},{"line_number":21,"context_line":"* Users identities must be hidden unless they agree/ask to have them published"}],"source_content_type":"text/x-rst","patch_set":2,"id":"AAAAXX%2F%2F%2Bc8%3D","line":18,"updated":"2014-04-24 23:05:19.000000000","message":"I assume this mean \"Tempest\" code, right?\nUsers still need to download the tcup related file(s).","commit_id":"cb5414f263bebe64eba1def47432af601ef9acfe"},{"author":{"_account_id":10527,"name":"Raymond Wong","email":"waiman@us.ibm.com","username":"waiman"},"change_message_id":"9d07745de533757a7c40a8891e509a12aad3f2b2","unresolved":false,"context_lines":[{"line_number":16,"context_line":""},{"line_number":17,"context_line":"* It should not matter which Linux distro they are using"},{"line_number":18,"context_line":"* Users should not have to figure out which code to check out"},{"line_number":19,"context_line":"* Users should not have to deal with packages or pips"},{"line_number":20,"context_line":"* Users should not have to determine where to upload their results"},{"line_number":21,"context_line":"* Users identities must be hidden unless they agree/ask to have them published"},{"line_number":22,"context_line":"* When the test is complete, the test system dissolves"}],"source_content_type":"text/x-rst","patch_set":2,"id":"AAAAXX%2F%2F%2Bcg%3D","line":19,"updated":"2014-04-24 23:05:19.000000000","message":"Users still need to install and configure docker.\nAlso the python env for tcup.py.","commit_id":"cb5414f263bebe64eba1def47432af601ef9acfe"},{"author":{"_account_id":10527,"name":"Raymond Wong","email":"waiman@us.ibm.com","username":"waiman"},"change_message_id":"9d07745de533757a7c40a8891e509a12aad3f2b2","unresolved":false,"context_lines":[{"line_number":17,"context_line":"* It should not matter which Linux distro they are using"},{"line_number":18,"context_line":"* Users should not have to figure out which code to check out"},{"line_number":19,"context_line":"* Users should not have to deal with packages or pips"},{"line_number":20,"context_line":"* Users should not have to determine where to upload their results"},{"line_number":21,"context_line":"* Users identities must be hidden unless they agree/ask to have them published"},{"line_number":22,"context_line":"* When the test is complete, the test system dissolves"},{"line_number":23,"context_line":""}],"source_content_type":"text/x-rst","patch_set":2,"id":"AAAAXX%2F%2F%2Bc0%3D","line":20,"updated":"2014-04-24 23:05:19.000000000","message":"So, the upload location will be hardcoded, and it probably be Refstack.org, right?","commit_id":"cb5414f263bebe64eba1def47432af601ef9acfe"},{"author":{"_account_id":8434,"name":"Rochelle Grober","email":"rochelle.grober@huawei.com","username":"rockyg"},"change_message_id":"326744590cf0fe8a0ce9e4a3e28d2598c1782ea6","unresolved":false,"context_lines":[{"line_number":17,"context_line":"* It should not matter which Linux distro they are using"},{"line_number":18,"context_line":"* Users should not have to figure out which code to check out"},{"line_number":19,"context_line":"* Users should not have to deal with packages or pips"},{"line_number":20,"context_line":"* Users should not have to determine where to upload their results"},{"line_number":21,"context_line":"* Users identities must be hidden unless they agree/ask to have them published"},{"line_number":22,"context_line":"* When the test is complete, the test system dissolves"},{"line_number":23,"context_line":""}],"source_content_type":"text/x-rst","patch_set":2,"id":"AAAAXX%2F%2F9bo%3D","line":20,"in_reply_to":"AAAAXX%2F%2F%2Bc0%3D","updated":"2014-04-25 01:48:35.000000000","message":"I don\u0027t think hardcoding is what is meant by this.  From this morning\u0027s discussion, it sound more like a file will be generated and another tool will post it to Refstack.  So you can either keep the file local, or upload it to Refstack","commit_id":"cb5414f263bebe64eba1def47432af601ef9acfe"},{"author":{"_account_id":10527,"name":"Raymond Wong","email":"waiman@us.ibm.com","username":"waiman"},"change_message_id":"9d07745de533757a7c40a8891e509a12aad3f2b2","unresolved":false,"context_lines":[{"line_number":18,"context_line":"* Users should not have to figure out which code to check out"},{"line_number":19,"context_line":"* Users should not have to deal with packages or pips"},{"line_number":20,"context_line":"* Users should not have to determine where to upload their results"},{"line_number":21,"context_line":"* Users identities must be hidden unless they agree/ask to have them published"},{"line_number":22,"context_line":"* When the test is complete, the test system dissolves"},{"line_number":23,"context_line":""},{"line_number":24,"context_line":"Anti-Requirements:"}],"source_content_type":"text/x-rst","patch_set":2,"id":"AAAAXX%2F%2F%2Bbg%3D","line":21,"updated":"2014-04-24 23:05:19.000000000","message":"Right, we only upload tempest results (the subunit output), not the tempest config nor other information.\n\nHowever, there is a potential risk.\nWhen tempest test fails, it writes  the exception in the subunit output.\nIn some case (like login failure), the exception message shows the provided user id and password.\nThis is a tempest issue, and not easy to fix from Refstack side (if ever possible)","commit_id":"cb5414f263bebe64eba1def47432af601ef9acfe"},{"author":{"_account_id":10527,"name":"Raymond Wong","email":"waiman@us.ibm.com","username":"waiman"},"change_message_id":"9d07745de533757a7c40a8891e509a12aad3f2b2","unresolved":false,"context_lines":[{"line_number":19,"context_line":"* Users should not have to deal with packages or pips"},{"line_number":20,"context_line":"* Users should not have to determine where to upload their results"},{"line_number":21,"context_line":"* Users identities must be hidden unless they agree/ask to have them published"},{"line_number":22,"context_line":"* When the test is complete, the test system dissolves"},{"line_number":23,"context_line":""},{"line_number":24,"context_line":"Anti-Requirements:"},{"line_number":25,"context_line":""}],"source_content_type":"text/x-rst","patch_set":2,"id":"AAAAXX%2F%2F%2BNg%3D","line":22,"updated":"2014-04-24 23:05:19.000000000","message":"Does this mean destroying the docker images and containers too?\nOtherwise, their number will keep growing as users run more tests.\n\nHowever, users will not be able to login to the docker container to debug after the test.","commit_id":"cb5414f263bebe64eba1def47432af601ef9acfe"},{"author":{"_account_id":10527,"name":"Raymond Wong","email":"waiman@us.ibm.com","username":"waiman"},"change_message_id":"9d07745de533757a7c40a8891e509a12aad3f2b2","unresolved":false,"context_lines":[{"line_number":23,"context_line":""},{"line_number":24,"context_line":"Anti-Requirements:"},{"line_number":25,"context_line":""},{"line_number":26,"context_line":"* Users should not need to checkout or clone any code"},{"line_number":27,"context_line":"* Users should not have to edit configuration files"},{"line_number":28,"context_line":""},{"line_number":29,"context_line":".. image:: ../images/tcup-flow.prg"}],"source_content_type":"text/x-rst","patch_set":2,"id":"AAAAXX%2F%2F%2BaI%3D","line":26,"updated":"2014-04-24 23:05:19.000000000","message":"As mentioned above, users need to at least \"get\" the tcup code somehow.","commit_id":"cb5414f263bebe64eba1def47432af601ef9acfe"},{"author":{"_account_id":10527,"name":"Raymond Wong","email":"waiman@us.ibm.com","username":"waiman"},"change_message_id":"9d07745de533757a7c40a8891e509a12aad3f2b2","unresolved":false,"context_lines":[{"line_number":24,"context_line":"Anti-Requirements:"},{"line_number":25,"context_line":""},{"line_number":26,"context_line":"* Users should not need to checkout or clone any code"},{"line_number":27,"context_line":"* Users should not have to edit configuration files"},{"line_number":28,"context_line":""},{"line_number":29,"context_line":".. image:: ../images/tcup-flow.prg"},{"line_number":30,"context_line":"   :width: 700px"}],"source_content_type":"text/x-rst","patch_set":2,"id":"AAAAXX%2F%2F%2BZs%3D","line":27,"updated":"2014-04-24 23:05:19.000000000","message":"I assume this refers to tempest.config file.\nUsers may still need to edit rc file that contains credentials.","commit_id":"cb5414f263bebe64eba1def47432af601ef9acfe"},{"author":{"_account_id":6287,"name":"David Lenwell","email":"dlenwell@gmail.com","username":"dlenwell"},"change_message_id":"04b48c5ee9a155382874feb951592053704a395a","unresolved":false,"context_lines":[{"line_number":26,"context_line":"* Users should not need to checkout or clone any code"},{"line_number":27,"context_line":"* Users should not have to edit configuration files"},{"line_number":28,"context_line":""},{"line_number":29,"context_line":".. image:: ../images/tcup-flow.prg"},{"line_number":30,"context_line":"   :width: 700px"},{"line_number":31,"context_line":"   :alt: TCUP workflow"},{"line_number":32,"context_line":""}],"source_content_type":"text/x-rst","patch_set":2,"id":"AAAAXH%2F%2FEYU%3D","line":29,"updated":"2014-04-24 09:15:37.000000000","message":"surely this is a type-o .. .prg should be .png ..\nAlso .. since these are currently hard to view in the review process.. (I had to cherry pick the review then open it on my computer directly to see your drawing) lets put images on the openstack wiki.. since these articles will be translated into docs once approved anyways.","commit_id":"cb5414f263bebe64eba1def47432af601ef9acfe"},{"author":{"_account_id":6287,"name":"David Lenwell","email":"dlenwell@gmail.com","username":"dlenwell"},"change_message_id":"04b48c5ee9a155382874feb951592053704a395a","unresolved":false,"context_lines":[{"line_number":35,"context_line":""},{"line_number":36,"context_line":"TCUP should be designed in as simple a way as possible."},{"line_number":37,"context_line":""},{"line_number":38,"context_line":"Running TCUP should only require Docker (.9+), a single tcup.py file with minimal dependencies, working OpenStack cloud credentials and an Internet connection.  Internet connectivity is required to download TCUP and upload the test results.  The target cloud does _not_ need to public/Internet accessible but the user much have access to it for testing."},{"line_number":39,"context_line":""},{"line_number":40,"context_line":"Environment variables from the host (OS-*) will be passed into the container.  The container should not start unless critical OS_* variables are in place."},{"line_number":41,"context_line":""}],"source_content_type":"text/x-rst","patch_set":2,"id":"AAAAXH%2F%2FEU4%3D","line":38,"updated":"2014-04-24 09:15:37.000000000","message":"typo ... *must\n\nAlso we should elaborate that the machine running docker will need internet access.. even if the cloud doesn\u0027t.  the testing machine can be on a dual head network and be able to talk to the net and the private cloud.. this covers most admins laptops.","commit_id":"cb5414f263bebe64eba1def47432af601ef9acfe"},{"author":{"_account_id":8434,"name":"Rochelle Grober","email":"rochelle.grober@huawei.com","username":"rockyg"},"change_message_id":"326744590cf0fe8a0ce9e4a3e28d2598c1782ea6","unresolved":false,"context_lines":[{"line_number":35,"context_line":""},{"line_number":36,"context_line":"TCUP should be designed in as simple a way as possible."},{"line_number":37,"context_line":""},{"line_number":38,"context_line":"Running TCUP should only require Docker (.9+), a single tcup.py file with minimal dependencies, working OpenStack cloud credentials and an Internet connection.  Internet connectivity is required to download TCUP and upload the test results.  The target cloud does _not_ need to public/Internet accessible but the user much have access to it for testing."},{"line_number":39,"context_line":""},{"line_number":40,"context_line":"Environment variables from the host (OS-*) will be passed into the container.  The container should not start unless critical OS_* variables are in place."},{"line_number":41,"context_line":""}],"source_content_type":"text/x-rst","patch_set":2,"id":"AAAAXX%2F%2F9aQ%3D","line":38,"in_reply_to":"AAAAXH%2F%2FEU4%3D","updated":"2014-04-25 01:48:35.000000000","message":"I can foresee where we may lose some tests/testers on locked down clouds, but I guess we are willing to have that small loss for our purposes.  Also, I guess the locked down clouds may not care as much about OpenStack compatibillity validation or certification","commit_id":"cb5414f263bebe64eba1def47432af601ef9acfe"},{"author":{"_account_id":10527,"name":"Raymond Wong","email":"waiman@us.ibm.com","username":"waiman"},"change_message_id":"9d07745de533757a7c40a8891e509a12aad3f2b2","unresolved":false,"context_lines":[{"line_number":35,"context_line":""},{"line_number":36,"context_line":"TCUP should be designed in as simple a way as possible."},{"line_number":37,"context_line":""},{"line_number":38,"context_line":"Running TCUP should only require Docker (.9+), a single tcup.py file with minimal dependencies, working OpenStack cloud credentials and an Internet connection.  Internet connectivity is required to download TCUP and upload the test results.  The target cloud does _not_ need to public/Internet accessible but the user much have access to it for testing."},{"line_number":39,"context_line":""},{"line_number":40,"context_line":"Environment variables from the host (OS-*) will be passed into the container.  The container should not start unless critical OS_* variables are in place."},{"line_number":41,"context_line":""}],"source_content_type":"text/x-rst","patch_set":2,"id":"AAAAXX%2F%2F%2BWE%3D","line":38,"in_reply_to":"AAAAXH%2F%2FEU4%3D","updated":"2014-04-24 23:05:19.000000000","message":"Should we integrate all 3 steps together?\n(1) build docker image\n(2) run tempest test with the docker image\n(3) upload the data\n\nWould it be more user friendly to allow users to run these 3 steps separately.\n\nSeparating 1 and 2 can allow users to\n- build image on machine with internet access, and run it on machine without internet access.\n- build image once, and run test many times\n\nSeparating 2 and 3 can allow users to\n- review the data, and decide if they want to upload especially when it fails during experimenting the setup.\n\nFor implementation, we don\u0027t need multiple scripts.\nSame script can provide different options.\nOne option can be do-all-steps too.\nJust to make it more flexible.","commit_id":"cb5414f263bebe64eba1def47432af601ef9acfe"},{"author":{"_account_id":8434,"name":"Rochelle Grober","email":"rochelle.grober@huawei.com","username":"rockyg"},"change_message_id":"326744590cf0fe8a0ce9e4a3e28d2598c1782ea6","unresolved":false,"context_lines":[{"line_number":35,"context_line":""},{"line_number":36,"context_line":"TCUP should be designed in as simple a way as possible."},{"line_number":37,"context_line":""},{"line_number":38,"context_line":"Running TCUP should only require Docker (.9+), a single tcup.py file with minimal dependencies, working OpenStack cloud credentials and an Internet connection.  Internet connectivity is required to download TCUP and upload the test results.  The target cloud does _not_ need to public/Internet accessible but the user much have access to it for testing."},{"line_number":39,"context_line":""},{"line_number":40,"context_line":"Environment variables from the host (OS-*) will be passed into the container.  The container should not start unless critical OS_* variables are in place."},{"line_number":41,"context_line":""}],"source_content_type":"text/x-rst","patch_set":2,"id":"AAAAXX%2F%2F9Z0%3D","line":38,"in_reply_to":"AAAAXX%2F%2F%2BWE%3D","updated":"2014-04-25 01:48:35.000000000","message":"This section really needs more discussion to clarify both purpose and approach.","commit_id":"cb5414f263bebe64eba1def47432af601ef9acfe"},{"author":{"_account_id":10527,"name":"Raymond Wong","email":"waiman@us.ibm.com","username":"waiman"},"change_message_id":"9d07745de533757a7c40a8891e509a12aad3f2b2","unresolved":false,"context_lines":[{"line_number":37,"context_line":""},{"line_number":38,"context_line":"Running TCUP should only require Docker (.9+), a single tcup.py file with minimal dependencies, working OpenStack cloud credentials and an Internet connection.  Internet connectivity is required to download TCUP and upload the test results.  The target cloud does _not_ need to public/Internet accessible but the user much have access to it for testing."},{"line_number":39,"context_line":""},{"line_number":40,"context_line":"Environment variables from the host (OS-*) will be passed into the container.  The container should not start unless critical OS_* variables are in place."},{"line_number":41,"context_line":""},{"line_number":42,"context_line":"There should be a `--debug` mode to allow for user testing and debug.  The debug flag should NOT start tests automatically and should map the user pwd into /dev in the TCUP container."},{"line_number":43,"context_line":""}],"source_content_type":"text/x-rst","patch_set":2,"id":"AAAAXX%2F%2F%2BVM%3D","line":40,"updated":"2014-04-24 23:05:19.000000000","message":"We need to determine the list of \"critical\" parameters.\nRunning openrc.sh does not provide enough input for most tempest testcases.","commit_id":"cb5414f263bebe64eba1def47432af601ef9acfe"},{"author":{"_account_id":10527,"name":"Raymond Wong","email":"waiman@us.ibm.com","username":"waiman"},"change_message_id":"9d07745de533757a7c40a8891e509a12aad3f2b2","unresolved":false,"context_lines":[{"line_number":38,"context_line":"Running TCUP should only require Docker (.9+), a single tcup.py file with minimal dependencies, working OpenStack cloud credentials and an Internet connection.  Internet connectivity is required to download TCUP and upload the test results.  The target cloud does _not_ need to public/Internet accessible but the user much have access to it for testing."},{"line_number":39,"context_line":""},{"line_number":40,"context_line":"Environment variables from the host (OS-*) will be passed into the container.  The container should not start unless critical OS_* variables are in place."},{"line_number":41,"context_line":""},{"line_number":42,"context_line":"There should be a `--debug` mode to allow for user testing and debug.  The debug flag should NOT start tests automatically and should map the user pwd into /dev in the TCUP container."},{"line_number":43,"context_line":""},{"line_number":44,"context_line":"There must be both a way to use local code (refstack) to run TCUP and also a simple file download approach.  These methods should be functionally identical."}],"source_content_type":"text/x-rst","patch_set":2,"id":"AAAAXX%2F%2F%2BVA%3D","line":41,"updated":"2014-04-24 23:05:19.000000000","message":"Another potential option is to allow user to provide their own tempest.config.\n\nAdvanced users can customize/reuse their tempest.config and simply ask tcup to run with it.","commit_id":"cb5414f263bebe64eba1def47432af601ef9acfe"},{"author":{"_account_id":10527,"name":"Raymond Wong","email":"waiman@us.ibm.com","username":"waiman"},"change_message_id":"9d07745de533757a7c40a8891e509a12aad3f2b2","unresolved":false,"context_lines":[{"line_number":39,"context_line":""},{"line_number":40,"context_line":"Environment variables from the host (OS-*) will be passed into the container.  The container should not start unless critical OS_* variables are in place."},{"line_number":41,"context_line":""},{"line_number":42,"context_line":"There should be a `--debug` mode to allow for user testing and debug.  The debug flag should NOT start tests automatically and should map the user pwd into /dev in the TCUP container."},{"line_number":43,"context_line":""},{"line_number":44,"context_line":"There must be both a way to use local code (refstack) to run TCUP and also a simple file download approach.  These methods should be functionally identical."},{"line_number":45,"context_line":""}],"source_content_type":"text/x-rst","patch_set":2,"id":"AAAAXX%2F%2F%2BUg%3D","line":42,"updated":"2014-04-24 23:05:19.000000000","message":"As a user, I don\u0027t care about debugging the code...\nProbably what I would like to verify is just the generated tempest.config (so that I will know what TCUP is testing with).\nSo, a useful option maybe just to return a tempest.config for user to verify before executing the test.","commit_id":"cb5414f263bebe64eba1def47432af601ef9acfe"},{"author":{"_account_id":6287,"name":"David Lenwell","email":"dlenwell@gmail.com","username":"dlenwell"},"change_message_id":"04b48c5ee9a155382874feb951592053704a395a","unresolved":false,"context_lines":[{"line_number":41,"context_line":""},{"line_number":42,"context_line":"There should be a `--debug` mode to allow for user testing and debug.  The debug flag should NOT start tests automatically and should map the user pwd into /dev in the TCUP container."},{"line_number":43,"context_line":""},{"line_number":44,"context_line":"There must be both a way to use local code (refstack) to run TCUP and also a simple file download approach.  These methods should be functionally identical."},{"line_number":45,"context_line":""},{"line_number":46,"context_line":"While there may be broad uses of TCUP for test automation, it is not desirable to overload them at the expense of manual usability.  TCUP should be very simple of users in this pass."},{"line_number":47,"context_line":""}],"source_content_type":"text/x-rst","patch_set":2,"id":"AAAAXH%2F%2FEUY%3D","line":44,"updated":"2014-04-24 09:15:37.000000000","message":"execute_test should be spun off into its own repo and packaged so that it can be versioned with the api and is easy to install in docker or single use vms.","commit_id":"cb5414f263bebe64eba1def47432af601ef9acfe"},{"author":{"_account_id":10527,"name":"Raymond Wong","email":"waiman@us.ibm.com","username":"waiman"},"change_message_id":"9d07745de533757a7c40a8891e509a12aad3f2b2","unresolved":false,"context_lines":[{"line_number":42,"context_line":"There should be a `--debug` mode to allow for user testing and debug.  The debug flag should NOT start tests automatically and should map the user pwd into /dev in the TCUP container."},{"line_number":43,"context_line":""},{"line_number":44,"context_line":"There must be both a way to use local code (refstack) to run TCUP and also a simple file download approach.  These methods should be functionally identical."},{"line_number":45,"context_line":""},{"line_number":46,"context_line":"While there may be broad uses of TCUP for test automation, it is not desirable to overload them at the expense of manual usability.  TCUP should be very simple of users in this pass."},{"line_number":47,"context_line":""},{"line_number":48,"context_line":"By default, TCUP will upload results to the Refstack site.  By changing their API target parameters, users may instead upload their results to an internal Refstack site or simply save the results to their local drive."}],"source_content_type":"text/x-rst","patch_set":2,"id":"AAAAXX%2F%2F%2BR0%3D","line":45,"updated":"2014-04-24 23:05:19.000000000","message":"I guess this means users with local refstack installed, they can run tcup the same was as if they have just downloaded the tcup related file(s), right?","commit_id":"cb5414f263bebe64eba1def47432af601ef9acfe"},{"author":{"_account_id":8434,"name":"Rochelle Grober","email":"rochelle.grober@huawei.com","username":"rockyg"},"change_message_id":"326744590cf0fe8a0ce9e4a3e28d2598c1782ea6","unresolved":false,"context_lines":[{"line_number":42,"context_line":"There should be a `--debug` mode to allow for user testing and debug.  The debug flag should NOT start tests automatically and should map the user pwd into /dev in the TCUP container."},{"line_number":43,"context_line":""},{"line_number":44,"context_line":"There must be both a way to use local code (refstack) to run TCUP and also a simple file download approach.  These methods should be functionally identical."},{"line_number":45,"context_line":""},{"line_number":46,"context_line":"While there may be broad uses of TCUP for test automation, it is not desirable to overload them at the expense of manual usability.  TCUP should be very simple of users in this pass."},{"line_number":47,"context_line":""},{"line_number":48,"context_line":"By default, TCUP will upload results to the Refstack site.  By changing their API target parameters, users may instead upload their results to an internal Refstack site or simply save the results to their local drive."}],"source_content_type":"text/x-rst","patch_set":2,"id":"AAAAXX%2F%2F9ZI%3D","line":45,"updated":"2014-04-25 01:48:35.000000000","message":"So, this sounds like the \"run without internet\" option, but above we say you can\u0027t do that.  Hmmmm.","commit_id":"cb5414f263bebe64eba1def47432af601ef9acfe"},{"author":{"_account_id":8434,"name":"Rochelle Grober","email":"rochelle.grober@huawei.com","username":"rockyg"},"change_message_id":"326744590cf0fe8a0ce9e4a3e28d2598c1782ea6","unresolved":false,"context_lines":[{"line_number":45,"context_line":""},{"line_number":46,"context_line":"While there may be broad uses of TCUP for test automation, it is not desirable to overload them at the expense of manual usability.  TCUP should be very simple of users in this pass."},{"line_number":47,"context_line":""},{"line_number":48,"context_line":"By default, TCUP will upload results to the Refstack site.  By changing their API target parameters, users may instead upload their results to an internal Refstack site or simply save the results to their local drive."},{"line_number":49,"context_line":""},{"line_number":50,"context_line":"Alternatives"},{"line_number":51,"context_line":"------------"}],"source_content_type":"text/x-rst","patch_set":2,"id":"AAAAXX%2F%2F9Yg%3D","line":48,"updated":"2014-04-25 01:48:35.000000000","message":"Agree to some extent with Raymond.  I think we need a clearer and more detailed understanding of the evolving nature of execute_test and its interfaces to TCUP","commit_id":"cb5414f263bebe64eba1def47432af601ef9acfe"},{"author":{"_account_id":10527,"name":"Raymond Wong","email":"waiman@us.ibm.com","username":"waiman"},"change_message_id":"9d07745de533757a7c40a8891e509a12aad3f2b2","unresolved":false,"context_lines":[{"line_number":45,"context_line":""},{"line_number":46,"context_line":"While there may be broad uses of TCUP for test automation, it is not desirable to overload them at the expense of manual usability.  TCUP should be very simple of users in this pass."},{"line_number":47,"context_line":""},{"line_number":48,"context_line":"By default, TCUP will upload results to the Refstack site.  By changing their API target parameters, users may instead upload their results to an internal Refstack site or simply save the results to their local drive."},{"line_number":49,"context_line":""},{"line_number":50,"context_line":"Alternatives"},{"line_number":51,"context_line":"------------"}],"source_content_type":"text/x-rst","patch_set":2,"id":"AAAAXX%2F%2F%2BRY%3D","line":48,"updated":"2014-04-24 23:05:19.000000000","message":"This sounds like it is in conflict with line 20 above:\n\nUsers should not have to determine where to upload their results","commit_id":"cb5414f263bebe64eba1def47432af601ef9acfe"},{"author":{"_account_id":6287,"name":"David Lenwell","email":"dlenwell@gmail.com","username":"dlenwell"},"change_message_id":"04b48c5ee9a155382874feb951592053704a395a","unresolved":false,"context_lines":[{"line_number":51,"context_line":"------------"},{"line_number":52,"context_line":""},{"line_number":53,"context_line":"It would be possible to create a single-use VM for this testing.  That would require much more download time, build effort and maintenance."},{"line_number":54,"context_line":""},{"line_number":55,"context_line":"It would be possible to setup a cloud-based process to run Tempest (this is a Refstack use case); however, this would not reach private clouds.  It also does not give the user control of the data."},{"line_number":56,"context_line":""},{"line_number":57,"context_line":"Data model impact"}],"source_content_type":"text/x-rst","patch_set":2,"id":"AAAAXH%2F%2FEQw%3D","line":54,"updated":"2014-04-24 09:15:37.000000000","message":"An additional method is to not provide a full single use vm but to package execute_test on its own allowing it to install on fedora and ubuntu.. it already has tempest havana stable as a dependancy. it can be installed .. the rc file can be sourced and it can be kicked off. no container needed.. you can log into any cloud instance on any cloud provider that has network reach to the cloud you want to test.. start an ephemeral vm.. log into it and run two commands.","commit_id":"cb5414f263bebe64eba1def47432af601ef9acfe"},{"author":{"_account_id":10527,"name":"Raymond Wong","email":"waiman@us.ibm.com","username":"waiman"},"change_message_id":"9d07745de533757a7c40a8891e509a12aad3f2b2","unresolved":false,"context_lines":[{"line_number":51,"context_line":"------------"},{"line_number":52,"context_line":""},{"line_number":53,"context_line":"It would be possible to create a single-use VM for this testing.  That would require much more download time, build effort and maintenance."},{"line_number":54,"context_line":""},{"line_number":55,"context_line":"It would be possible to setup a cloud-based process to run Tempest (this is a Refstack use case); however, this would not reach private clouds.  It also does not give the user control of the data."},{"line_number":56,"context_line":""},{"line_number":57,"context_line":"Data model impact"}],"source_content_type":"text/x-rst","patch_set":2,"id":"AAAAXX%2F%2F%2BQI%3D","line":54,"in_reply_to":"AAAAXH%2F%2FEQw%3D","updated":"2014-04-24 23:05:19.000000000","message":"If we can assume tempest havana is already installed,\nanother alternative can be having tcup.py invokes execute_test directly without using docker or any container.","commit_id":"cb5414f263bebe64eba1def47432af601ef9acfe"},{"author":{"_account_id":6287,"name":"David Lenwell","email":"dlenwell@gmail.com","username":"dlenwell"},"change_message_id":"04b48c5ee9a155382874feb951592053704a395a","unresolved":false,"context_lines":[{"line_number":62,"context_line":"REST API impact"},{"line_number":63,"context_line":"---------------"},{"line_number":64,"context_line":""},{"line_number":65,"context_line":"None; however, TCUP will rely on a stable upload REST API."},{"line_number":66,"context_line":""},{"line_number":67,"context_line":""},{"line_number":68,"context_line":"Security impact"}],"source_content_type":"text/x-rst","patch_set":2,"id":"AAAAXH%2F%2FEQQ%3D","line":65,"updated":"2014-04-24 09:15:37.000000000","message":"there is impact in that for this to possibly work we have to version the api asap.. which means we need a full spec for a v1 api.. I\u0027ll start on that asap .. but you should list the api calls that tcup will need logically here.","commit_id":"cb5414f263bebe64eba1def47432af601ef9acfe"},{"author":{"_account_id":6287,"name":"David Lenwell","email":"dlenwell@gmail.com","username":"dlenwell"},"change_message_id":"04b48c5ee9a155382874feb951592053704a395a","unresolved":false,"context_lines":[{"line_number":69,"context_line":"---------------"},{"line_number":70,"context_line":""},{"line_number":71,"context_line":"User passwords are passed into the container and should be redacted from log entries."},{"line_number":72,"context_line":""},{"line_number":73,"context_line":"We should prompt the user (from the tcup.py) code to enter a password if none is provided in the environment."},{"line_number":74,"context_line":""},{"line_number":75,"context_line":"Passwords must not be stored by TCUP!"}],"source_content_type":"text/x-rst","patch_set":2,"id":"AAAAXH%2F%2FEPk%3D","line":72,"updated":"2014-04-24 09:15:37.000000000","message":"passwords will be prompted for when the used sources the openstack rc file they downloaded from their cloud. we shouldn\u0027t have to create extra code for that .. its already built into the way those files work.","commit_id":"cb5414f263bebe64eba1def47432af601ef9acfe"},{"author":{"_account_id":10527,"name":"Raymond Wong","email":"waiman@us.ibm.com","username":"waiman"},"change_message_id":"9d07745de533757a7c40a8891e509a12aad3f2b2","unresolved":false,"context_lines":[{"line_number":69,"context_line":"---------------"},{"line_number":70,"context_line":""},{"line_number":71,"context_line":"User passwords are passed into the container and should be redacted from log entries."},{"line_number":72,"context_line":""},{"line_number":73,"context_line":"We should prompt the user (from the tcup.py) code to enter a password if none is provided in the environment."},{"line_number":74,"context_line":""},{"line_number":75,"context_line":"Passwords must not be stored by TCUP!"}],"source_content_type":"text/x-rst","patch_set":2,"id":"AAAAXX%2F%2F%2BOY%3D","line":72,"in_reply_to":"AAAAXH%2F%2FEPk%3D","updated":"2014-04-24 23:05:19.000000000","message":"As mentioned above, the tempest subunit output (not the log file) may have the userid and password when the tempest test fails.\n\nWe can figure out how to hide them, or ask Tempest team to NOT put them in the output.","commit_id":"cb5414f263bebe64eba1def47432af601ef9acfe"},{"author":{"_account_id":8434,"name":"Rochelle Grober","email":"rochelle.grober@huawei.com","username":"rockyg"},"change_message_id":"326744590cf0fe8a0ce9e4a3e28d2598c1782ea6","unresolved":false,"context_lines":[{"line_number":69,"context_line":"---------------"},{"line_number":70,"context_line":""},{"line_number":71,"context_line":"User passwords are passed into the container and should be redacted from log entries."},{"line_number":72,"context_line":""},{"line_number":73,"context_line":"We should prompt the user (from the tcup.py) code to enter a password if none is provided in the environment."},{"line_number":74,"context_line":""},{"line_number":75,"context_line":"Passwords must not be stored by TCUP!"}],"source_content_type":"text/x-rst","patch_set":2,"id":"AAAAXX%2F%2F9X4%3D","line":72,"in_reply_to":"AAAAXX%2F%2F%2BOY%3D","updated":"2014-04-25 01:48:35.000000000","message":"I like the idea of having the passwords obfuscated in the tempest output.  I think it\u0027s going to need to happen with other projects using tempest outside of OpenStack.org.","commit_id":"cb5414f263bebe64eba1def47432af601ef9acfe"},{"author":{"_account_id":10527,"name":"Raymond Wong","email":"waiman@us.ibm.com","username":"waiman"},"change_message_id":"9d07745de533757a7c40a8891e509a12aad3f2b2","unresolved":false,"context_lines":[{"line_number":72,"context_line":""},{"line_number":73,"context_line":"We should prompt the user (from the tcup.py) code to enter a password if none is provided in the environment."},{"line_number":74,"context_line":""},{"line_number":75,"context_line":"Passwords must not be stored by TCUP!"},{"line_number":76,"context_line":""},{"line_number":77,"context_line":"Notifications impact"},{"line_number":78,"context_line":"--------------------"}],"source_content_type":"text/x-rst","patch_set":2,"id":"AAAAXX%2F%2F%2BN0%3D","line":75,"updated":"2014-04-24 23:05:19.000000000","message":"Passwords will be seen in the tempest.config file.\nSo, upon test completion, we should delete the tempest.config or at least mask them out.","commit_id":"cb5414f263bebe64eba1def47432af601ef9acfe"},{"author":{"_account_id":6287,"name":"David Lenwell","email":"dlenwell@gmail.com","username":"dlenwell"},"change_message_id":"04b48c5ee9a155382874feb951592053704a395a","unresolved":false,"context_lines":[{"line_number":82,"context_line":"Other end user impact"},{"line_number":83,"context_line":"---------------------"},{"line_number":84,"context_line":""},{"line_number":85,"context_line":"TCUP is designed as a stand-alone capability.  It should not have interactions with other parts of the system."},{"line_number":86,"context_line":""},{"line_number":87,"context_line":"Performance Impact"},{"line_number":88,"context_line":"------------------"}],"source_content_type":"text/x-rst","patch_set":2,"id":"AAAAXH%2F%2FEPU%3D","line":85,"updated":"2014-04-24 09:15:37.000000000","message":"except the api","commit_id":"cb5414f263bebe64eba1def47432af601ef9acfe"},{"author":{"_account_id":10527,"name":"Raymond Wong","email":"waiman@us.ibm.com","username":"waiman"},"change_message_id":"9d07745de533757a7c40a8891e509a12aad3f2b2","unresolved":false,"context_lines":[{"line_number":122,"context_line":"* build TCUP docker container (via Dockerfile)"},{"line_number":123,"context_line":"* build tcup.py to build and launch docker"},{"line_number":124,"context_line":"* document run process"},{"line_number":125,"context_line":"* update configuration generator to use environment variables"},{"line_number":126,"context_line":"* integrate run_test scripts into TCUP"},{"line_number":127,"context_line":"* integrate upload target into TCUP"},{"line_number":128,"context_line":""}],"source_content_type":"text/x-rst","patch_set":2,"id":"AAAAXX%2F%2F%2BLg%3D","line":125,"updated":"2014-04-24 23:05:19.000000000","message":"The current execute_test takes most input from arguments.\nInstead of updating execute_test, an alternative is to have the tcup code to construct the argument from environment variables.\n\nIt probably be the same code, just where to put the code.\n\nNote: execute_test is getting very complex already. it is supporting multiple paths, and not all paths utilize enviornment variables. So, I suggest to put it in the caller rather than in execute_test.","commit_id":"cb5414f263bebe64eba1def47432af601ef9acfe"},{"author":{"_account_id":10527,"name":"Raymond Wong","email":"waiman@us.ibm.com","username":"waiman"},"change_message_id":"9d07745de533757a7c40a8891e509a12aad3f2b2","unresolved":false,"context_lines":[{"line_number":123,"context_line":"* build tcup.py to build and launch docker"},{"line_number":124,"context_line":"* document run process"},{"line_number":125,"context_line":"* update configuration generator to use environment variables"},{"line_number":126,"context_line":"* integrate run_test scripts into TCUP"},{"line_number":127,"context_line":"* integrate upload target into TCUP"},{"line_number":128,"context_line":""},{"line_number":129,"context_line":"Dependencies"}],"source_content_type":"text/x-rst","patch_set":2,"id":"AAAAXX%2F%2F%2BLE%3D","line":126,"updated":"2014-04-24 23:05:19.000000000","message":"I think you mean the current \"execute_test\" not \"run_test\".","commit_id":"cb5414f263bebe64eba1def47432af601ef9acfe"},{"author":{"_account_id":8434,"name":"Rochelle Grober","email":"rochelle.grober@huawei.com","username":"rockyg"},"change_message_id":"326744590cf0fe8a0ce9e4a3e28d2598c1782ea6","unresolved":false,"context_lines":[{"line_number":124,"context_line":"* document run process"},{"line_number":125,"context_line":"* update configuration generator to use environment variables"},{"line_number":126,"context_line":"* integrate run_test scripts into TCUP"},{"line_number":127,"context_line":"* integrate upload target into TCUP"},{"line_number":128,"context_line":""},{"line_number":129,"context_line":"Dependencies"},{"line_number":130,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"}],"source_content_type":"text/x-rst","patch_set":2,"id":"AAAAXX%2F%2F9Xk%3D","line":127,"updated":"2014-04-25 01:48:35.000000000","message":"This should have a default but be modifiable.  And it should be easy to find out how/where to modify.","commit_id":"cb5414f263bebe64eba1def47432af601ef9acfe"},{"author":{"_account_id":6287,"name":"David Lenwell","email":"dlenwell@gmail.com","username":"dlenwell"},"change_message_id":"04b48c5ee9a155382874feb951592053704a395a","unresolved":false,"context_lines":[{"line_number":129,"context_line":"Dependencies"},{"line_number":130,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"},{"line_number":131,"context_line":""},{"line_number":132,"context_line":"* run_test scripts must support environment variables"},{"line_number":133,"context_line":"* upload API must function correctly"},{"line_number":134,"context_line":""},{"line_number":135,"context_line":"Testing"}],"source_content_type":"text/x-rst","patch_set":2,"id":"AAAAXH%2F%2FEOs%3D","line":132,"updated":"2014-04-24 09:15:37.000000000","message":"execute_test.. this should be supporting env variables already .. however if it isn\u0027t lets get that fixed asap.","commit_id":"cb5414f263bebe64eba1def47432af601ef9acfe"},{"author":{"_account_id":10527,"name":"Raymond Wong","email":"waiman@us.ibm.com","username":"waiman"},"change_message_id":"9d07745de533757a7c40a8891e509a12aad3f2b2","unresolved":false,"context_lines":[{"line_number":129,"context_line":"Dependencies"},{"line_number":130,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"},{"line_number":131,"context_line":""},{"line_number":132,"context_line":"* run_test scripts must support environment variables"},{"line_number":133,"context_line":"* upload API must function correctly"},{"line_number":134,"context_line":""},{"line_number":135,"context_line":"Testing"}],"source_content_type":"text/x-rst","patch_set":2,"id":"AAAAXX%2F%2F%2BKw%3D","line":132,"in_reply_to":"AAAAXH%2F%2FEOs%3D","updated":"2014-04-24 23:05:19.000000000","message":"is this the same as line 125 above?","commit_id":"cb5414f263bebe64eba1def47432af601ef9acfe"},{"author":{"_account_id":8434,"name":"Rochelle Grober","email":"rochelle.grober@huawei.com","username":"rockyg"},"change_message_id":"42514245842b13d1deee00179d6ec2f9d314824e","unresolved":false,"context_lines":[{"line_number":37,"context_line":""},{"line_number":38,"context_line":"TCUP should be designed in as simple a way as possible."},{"line_number":39,"context_line":""},{"line_number":40,"context_line":"Running TCUP should only require Docker (.9+), a single tcup.py file with minimal dependencies, working OpenStack cloud credentials and an Internet connection for install and results upload.  The cloud being tested does _not_ have to be public.  TCUP will work as long as the user has network access to it."},{"line_number":41,"context_line":""},{"line_number":42,"context_line":"Environment variables from the host (OS_*) will be passed into the container.  The container should not start unless critical OS_ variables are in place.  The specific OS_ items are limited:"},{"line_number":43,"context_line":""}],"source_content_type":"text/x-rst","patch_set":4,"id":"9ad9bd40_0a668851","line":40,"updated":"2014-05-02 00:12:46.000000000","message":"last word:  \"it\".  Maybe \"as long as the user and the TCUP install has network access to the cloud being tested\".  I\u0027ll +1 it either way, but line 53 I want fixed for sure.","commit_id":"6f1e2684cea35323085783a426b8da360d9faf10"},{"author":{"_account_id":8434,"name":"Rochelle Grober","email":"rochelle.grober@huawei.com","username":"rockyg"},"change_message_id":"42514245842b13d1deee00179d6ec2f9d314824e","unresolved":false,"context_lines":[{"line_number":50,"context_line":""},{"line_number":51,"context_line":"There must be both a way to use local code (refstack) to run TCUP and also a simple file download approach.  These methods should be functionally identical."},{"line_number":52,"context_line":""},{"line_number":53,"context_line":"While there may be broad uses of TCUP for test automation, it is not desirable to overload them at the expense of manual usability.  TCUP should be very simple of users in this pass."},{"line_number":54,"context_line":""},{"line_number":55,"context_line":"By default, TCUP will upload results to the Refstack site (this is a requirement above); however, we anticipate other use cases.  For users who do not want to upload their results, they can change their API target parameters.  This will allow users to instead upload their results to an internal Refstack site or simply save the results to their local drive."},{"line_number":56,"context_line":""}],"source_content_type":"text/x-rst","patch_set":4,"id":"9ad9bd40_8a3c5831","line":53,"updated":"2014-05-02 00:12:46.000000000","message":"Last sentence.  Dropped a word or to or should \"of\" be \"for\"?","commit_id":"6f1e2684cea35323085783a426b8da360d9faf10"}]}