)]}'
{"specs/meta-data-and-uuid-for-tests.rst":[{"author":{"_account_id":5196,"name":"Matthew Treinish","email":"mtreinish@kortar.org","username":"treinish"},"change_message_id":"a9188c32851604599e9b085b250be662051d1c32","unresolved":false,"context_lines":[{"line_number":30,"context_line":""},{"line_number":31,"context_line":"Each test will have a decorator that will accept a required arguments"},{"line_number":32,"context_line":"containing a uuid and meta as kwargs. This open ended approach for meta data"},{"line_number":33,"context_line":"should insure flexibility for the future."},{"line_number":34,"context_line":""},{"line_number":35,"context_line":"This change will also require the development of a tool for inserting the"},{"line_number":36,"context_line":"initial set of decorators with uuid content for all the existing tests."}],"source_content_type":"text/x-rst","patch_set":2,"id":"3a961159_352264b9","line":33,"updated":"2015-01-07 02:31:19.000000000","message":"Could you elaborate on the why we need this aspect. I\u0027m actually not clear on the advantage of having this here. We already have test attrs which we can assign tags to tests. Does allowing for it to be key-value buy us something extra which I\u0027m missing. (not saying we shouldn\u0027t add this, but I\u0027m more curious)","commit_id":"8ff284cbf39477d5a843b06126c00fe91a68a2df"},{"author":{"_account_id":217,"name":"Rob Hirschfeld","email":"rob@zehicle.com","username":"robhirschfeld"},"change_message_id":"0f1c726185a3b4db3742aeb660b810be902af080","unresolved":false,"context_lines":[{"line_number":30,"context_line":""},{"line_number":31,"context_line":"Each test will have a decorator that will accept a required arguments"},{"line_number":32,"context_line":"containing a uuid and meta as kwargs. This open ended approach for meta data"},{"line_number":33,"context_line":"should insure flexibility for the future."},{"line_number":34,"context_line":""},{"line_number":35,"context_line":"This change will also require the development of a tool for inserting the"},{"line_number":36,"context_line":"initial set of decorators with uuid content for all the existing tests."}],"source_content_type":"text/x-rst","patch_set":2,"id":"3a961159_62ae749a","line":33,"in_reply_to":"3a961159_352264b9","updated":"2015-01-12 15:46:41.000000000","message":"The reason for this proposal is so that we can track tests across multiple releases even if the test is moved or renamed.  Down stream users of the tests (Refstack \u0026 DefCore) need to be able to track results over a multiple releases.  Adding a UUID helps decouple the name/location of the test.","commit_id":"8ff284cbf39477d5a843b06126c00fe91a68a2df"},{"author":{"_account_id":5196,"name":"Matthew Treinish","email":"mtreinish@kortar.org","username":"treinish"},"change_message_id":"1ffdce1c3c354e8e4b1d8a266f7f059ae3bb3836","unresolved":false,"context_lines":[{"line_number":30,"context_line":""},{"line_number":31,"context_line":"Each test will have a decorator that will accept a required arguments"},{"line_number":32,"context_line":"containing a uuid and meta as kwargs. This open ended approach for meta data"},{"line_number":33,"context_line":"should insure flexibility for the future."},{"line_number":34,"context_line":""},{"line_number":35,"context_line":"This change will also require the development of a tool for inserting the"},{"line_number":36,"context_line":"initial set of decorators with uuid content for all the existing tests."}],"source_content_type":"text/x-rst","patch_set":2,"id":"3a961159_c2e1dd2b","line":33,"in_reply_to":"3a961159_62ae749a","updated":"2015-01-12 20:39:41.000000000","message":"I wasn\u0027t asking about the proposal on the whole. I understand the need for the UUID tagging. (which is probably outlined sufficiently above) This comment was specificly about the arbitraty key value pair metadata part of this paragraph.","commit_id":"8ff284cbf39477d5a843b06126c00fe91a68a2df"},{"author":{"_account_id":7822,"name":"Chris Hoge (not involved in upstream development)","email":"chris@hogepodge.com","username":"hogepodge"},"change_message_id":"2d0b5eb78f74c559caeeff357ad15b54d500b3ae","unresolved":false,"context_lines":[{"line_number":30,"context_line":""},{"line_number":31,"context_line":"Each test will have a decorator that will accept a required arguments"},{"line_number":32,"context_line":"containing a uuid and meta as kwargs. This open ended approach for meta data"},{"line_number":33,"context_line":"should insure flexibility for the future."},{"line_number":34,"context_line":""},{"line_number":35,"context_line":"This change will also require the development of a tool for inserting the"},{"line_number":36,"context_line":"initial set of decorators with uuid content for all the existing tests."}],"source_content_type":"text/x-rst","patch_set":2,"id":"3a961159_f793cefd","line":33,"in_reply_to":"3a961159_a22cd1ad","updated":"2015-01-13 20:51:25.000000000","message":"Since the UUID represents the first bit of metadata that\u0027s being attached to the tests, it makes sense to extend it to capture future metadata.\n\nAn example that could happen immediately is the addition of core capability tagging.","commit_id":"8ff284cbf39477d5a843b06126c00fe91a68a2df"},{"author":{"_account_id":217,"name":"Rob Hirschfeld","email":"rob@zehicle.com","username":"robhirschfeld"},"change_message_id":"547deeca9ef0cdbd95993ac0b4506de058615dd6","unresolved":false,"context_lines":[{"line_number":30,"context_line":""},{"line_number":31,"context_line":"Each test will have a decorator that will accept a required arguments"},{"line_number":32,"context_line":"containing a uuid and meta as kwargs. This open ended approach for meta data"},{"line_number":33,"context_line":"should insure flexibility for the future."},{"line_number":34,"context_line":""},{"line_number":35,"context_line":"This change will also require the development of a tool for inserting the"},{"line_number":36,"context_line":"initial set of decorators with uuid content for all the existing tests."}],"source_content_type":"text/x-rst","patch_set":2,"id":"3a961159_a22cd1ad","line":33,"in_reply_to":"3a961159_c2e1dd2b","updated":"2015-01-12 21:04:45.000000000","message":"Ah, ok.  My understanding of how the meta tag works is that you\u0027d have to have uuid as a key word.","commit_id":"8ff284cbf39477d5a843b06126c00fe91a68a2df"},{"author":{"_account_id":5196,"name":"Matthew Treinish","email":"mtreinish@kortar.org","username":"treinish"},"change_message_id":"a9188c32851604599e9b085b250be662051d1c32","unresolved":false,"context_lines":[{"line_number":33,"context_line":"should insure flexibility for the future."},{"line_number":34,"context_line":""},{"line_number":35,"context_line":"This change will also require the development of a tool for inserting the"},{"line_number":36,"context_line":"initial set of decorators with uuid content for all the existing tests."},{"line_number":37,"context_line":""},{"line_number":38,"context_line":"A gate test for enforcing that all tests have the required decorator and uuid"},{"line_number":39,"context_line":"will also be required."}],"source_content_type":"text/x-rst","patch_set":2,"id":"3a961159_95187885","line":36,"updated":"2015-01-07 02:31:19.000000000","message":"I would drop initial set here, the tool is really intended to add decorators whenever they are missing from test_* methods in test_*py files. Running this when there aren\u0027t any decorators will populate the initial set, but it\u0027s used for more than that.","commit_id":"8ff284cbf39477d5a843b06126c00fe91a68a2df"},{"author":{"_account_id":5196,"name":"Matthew Treinish","email":"mtreinish@kortar.org","username":"treinish"},"change_message_id":"a9188c32851604599e9b085b250be662051d1c32","unresolved":false,"context_lines":[{"line_number":35,"context_line":"This change will also require the development of a tool for inserting the"},{"line_number":36,"context_line":"initial set of decorators with uuid content for all the existing tests."},{"line_number":37,"context_line":""},{"line_number":38,"context_line":"A gate test for enforcing that all tests have the required decorator and uuid"},{"line_number":39,"context_line":"will also be required."},{"line_number":40,"context_line":""},{"line_number":41,"context_line":"Alternatives"}],"source_content_type":"text/x-rst","patch_set":2,"id":"3a961159_d83dd78f","line":38,"updated":"2015-01-07 02:31:19.000000000","message":"I think we should be explicity here, \u0027gate test\u0027 is ambiguous. In the context of tempest\u0027s review workflow the entire repo is a basically a a gate test, as it\u0027s either a self validating tempest test case, or a unit test. Just say a hacking check.\n\nAlso you should probably mention the uniqueness check here too.","commit_id":"8ff284cbf39477d5a843b06126c00fe91a68a2df"},{"author":{"_account_id":217,"name":"Rob Hirschfeld","email":"rob@zehicle.com","username":"robhirschfeld"},"change_message_id":"fcb68c0af311b2a0a2e9b8440687755f454fa6ce","unresolved":false,"context_lines":[{"line_number":71,"context_line":"**meta decorator**"},{"line_number":72,"context_line":"The UUID and meta decorator format will be as follows::"},{"line_number":73,"context_line":""},{"line_number":74,"context_line":"  \u0027@test.meta(uuid\u003d\u002712345678-1234-5678-1234-567812345678\u0027,"},{"line_number":75,"context_line":"              otherdata\u003d\u0027another value\u0027)"},{"line_number":76,"context_line":""},{"line_number":77,"context_line":"Which will append::"}],"source_content_type":"text/x-rst","patch_set":2,"id":"3a961159_c249e861","line":74,"updated":"2014-12-29 19:23:53.000000000","message":"I think that we need to make sure they UUIDs are short and readable.  Making the UUIDs long will make it much harder to manage the DefCore files.","commit_id":"8ff284cbf39477d5a843b06126c00fe91a68a2df"},{"author":{"_account_id":7822,"name":"Chris Hoge (not involved in upstream development)","email":"chris@hogepodge.com","username":"hogepodge"},"change_message_id":"05ffa3c9b04e6ce5094daa6833ffdcafdba79837","unresolved":false,"context_lines":[{"line_number":71,"context_line":"**meta decorator**"},{"line_number":72,"context_line":"The UUID and meta decorator format will be as follows::"},{"line_number":73,"context_line":""},{"line_number":74,"context_line":"  \u0027@test.meta(uuid\u003d\u002712345678-1234-5678-1234-567812345678\u0027,"},{"line_number":75,"context_line":"              otherdata\u003d\u0027another value\u0027)"},{"line_number":76,"context_line":""},{"line_number":77,"context_line":"Which will append::"}],"source_content_type":"text/x-rst","patch_set":2,"id":"3a961159_b0bcd7dc","line":74,"in_reply_to":"3a961159_0d579127","updated":"2015-01-05 16:27:58.000000000","message":"I had originally looked at UUID (1 or 4) from the python uuid library because they are easy to use and available. There are ways to shorten random UUIDs (the most obvious is github style with truncation), but I\u0027m not opposed to a method that generates UUIDs with meaning. I\u0027d hate to shut down discussion if there are merits, but I\u0027m not an expert on tagging tests. My requirements would be that whatever the alternative is it could be made readily available and could be used to automate the initial test tagging.","commit_id":"8ff284cbf39477d5a843b06126c00fe91a68a2df"},{"author":{"_account_id":5196,"name":"Matthew Treinish","email":"mtreinish@kortar.org","username":"treinish"},"change_message_id":"a9188c32851604599e9b085b250be662051d1c32","unresolved":false,"context_lines":[{"line_number":71,"context_line":"**meta decorator**"},{"line_number":72,"context_line":"The UUID and meta decorator format will be as follows::"},{"line_number":73,"context_line":""},{"line_number":74,"context_line":"  \u0027@test.meta(uuid\u003d\u002712345678-1234-5678-1234-567812345678\u0027,"},{"line_number":75,"context_line":"              otherdata\u003d\u0027another value\u0027)"},{"line_number":76,"context_line":""},{"line_number":77,"context_line":"Which will append::"}],"source_content_type":"text/x-rst","patch_set":2,"id":"3a961159_55a61055","line":74,"in_reply_to":"3a961159_26424eab","updated":"2015-01-07 02:31:19.000000000","message":"@Rob why would it make it more harder to manage. If you intend to show the uuid in something user facing (which I\u0027m not sure why you would need to) like Chris said you can just shorten it in refstack to make it simpler to view. There are several ways to do this which won\u0027t really increase the risk of a collision. (especially in the context of this use case)\n\nWe can bikeshed on this if you really want, but for providing a unique identifier uuid is the ietf standard. The python library for doing this just makes it stupid simple. Sure we could go and come up with a new algorithm to do it which uses less bits, but why go to the effort.","commit_id":"8ff284cbf39477d5a843b06126c00fe91a68a2df"},{"author":{"_account_id":217,"name":"Rob Hirschfeld","email":"rob@zehicle.com","username":"robhirschfeld"},"change_message_id":"0f1c726185a3b4db3742aeb660b810be902af080","unresolved":false,"context_lines":[{"line_number":71,"context_line":"**meta decorator**"},{"line_number":72,"context_line":"The UUID and meta decorator format will be as follows::"},{"line_number":73,"context_line":""},{"line_number":74,"context_line":"  \u0027@test.meta(uuid\u003d\u002712345678-1234-5678-1234-567812345678\u0027,"},{"line_number":75,"context_line":"              otherdata\u003d\u0027another value\u0027)"},{"line_number":76,"context_line":""},{"line_number":77,"context_line":"Which will append::"}],"source_content_type":"text/x-rst","patch_set":2,"id":"3a961159_9cb746f8","line":74,"in_reply_to":"3a961159_55a61055","updated":"2015-01-12 15:46:41.000000000","message":"@Matthew I\u0027m not pushing this anymore.  No further discussion is needed.","commit_id":"8ff284cbf39477d5a843b06126c00fe91a68a2df"},{"author":{"_account_id":7822,"name":"Chris Hoge (not involved in upstream development)","email":"chris@hogepodge.com","username":"hogepodge"},"change_message_id":"2d0b5eb78f74c559caeeff357ad15b54d500b3ae","unresolved":false,"context_lines":[{"line_number":71,"context_line":"**meta decorator**"},{"line_number":72,"context_line":"The UUID and meta decorator format will be as follows::"},{"line_number":73,"context_line":""},{"line_number":74,"context_line":"  \u0027@test.meta(uuid\u003d\u002712345678-1234-5678-1234-567812345678\u0027,"},{"line_number":75,"context_line":"              otherdata\u003d\u0027another value\u0027)"},{"line_number":76,"context_line":""},{"line_number":77,"context_line":"Which will append::"}],"source_content_type":"text/x-rst","patch_set":2,"id":"3a961159_d7289299","line":74,"in_reply_to":"3a961159_9cb746f8","updated":"2015-01-13 20:51:25.000000000","message":"One point of consideration, it could improve readability to have UUID with a format something like this:\n\n\u003cproject\u003e-\u003cuse\u003e-\u003chash\u003e\n\nThere project is a three letter code representing the project (KEY, NOV, GLA for example), the use is a code representing what the test is targeting (crud operations, for example), and a hash (truncated uuid?) that guarantees uniqueness.\n\nThe reasoning is that the UUID can give an immediate indication of what the test is trying to accomplish, even if just a high-leve view.\n\nNot married to the idea, just want to throw something out there in case it makes sense and we want to consider it. I\u0027ve had discussions with people who have done testing and performance and they think the idea has merits.","commit_id":"8ff284cbf39477d5a843b06126c00fe91a68a2df"},{"author":{"_account_id":9377,"name":"Sergey Slipushenko","email":"slipushenko@gmail.com","username":"sslypushenko"},"change_message_id":"3ddadc66f378884bca865c69bc00e98270a1d8af","unresolved":false,"context_lines":[{"line_number":71,"context_line":"**meta decorator**"},{"line_number":72,"context_line":"The UUID and meta decorator format will be as follows::"},{"line_number":73,"context_line":""},{"line_number":74,"context_line":"  \u0027@test.meta(uuid\u003d\u002712345678-1234-5678-1234-567812345678\u0027,"},{"line_number":75,"context_line":"              otherdata\u003d\u0027another value\u0027)"},{"line_number":76,"context_line":""},{"line_number":77,"context_line":"Which will append::"}],"source_content_type":"text/x-rst","patch_set":2,"id":"3a961159_26424eab","line":74,"in_reply_to":"3a961159_b0bcd7dc","updated":"2015-01-06 16:33:38.000000000","message":"+1 for uuid4","commit_id":"8ff284cbf39477d5a843b06126c00fe91a68a2df"},{"author":{"_account_id":6287,"name":"David Lenwell","email":"dlenwell@gmail.com","username":"dlenwell"},"change_message_id":"9086d198766c5305dcce29e2e0666911c00b9934","unresolved":false,"context_lines":[{"line_number":71,"context_line":"**meta decorator**"},{"line_number":72,"context_line":"The UUID and meta decorator format will be as follows::"},{"line_number":73,"context_line":""},{"line_number":74,"context_line":"  \u0027@test.meta(uuid\u003d\u002712345678-1234-5678-1234-567812345678\u0027,"},{"line_number":75,"context_line":"              otherdata\u003d\u0027another value\u0027)"},{"line_number":76,"context_line":""},{"line_number":77,"context_line":"Which will append::"}],"source_content_type":"text/x-rst","patch_set":2,"id":"3a961159_0d579127","line":74,"in_reply_to":"3a961159_c249e861","updated":"2014-12-30 17:46:49.000000000","message":"The q/a team and most of the rest of the community has spoken loud and clear that they want uuid4. I\u0027m happy to have this discussion with you and members of that team.. however at this point it isn\u0027t up to us.","commit_id":"8ff284cbf39477d5a843b06126c00fe91a68a2df"},{"author":{"_account_id":9377,"name":"Sergey Slipushenko","email":"slipushenko@gmail.com","username":"sslypushenko"},"change_message_id":"3ddadc66f378884bca865c69bc00e98270a1d8af","unresolved":false,"context_lines":[{"line_number":79,"context_line":"  uuid: \u002712345678-1234-5678-1234-567812345678\u0027"},{"line_number":80,"context_line":"  otherdata :\u0027another value\u0027"},{"line_number":81,"context_line":""},{"line_number":82,"context_line":"to the doc string and add the testtools attr::"},{"line_number":83,"context_line":""},{"line_number":84,"context_line":"  uuid\u003d\u002712345678-1234-5678-1234-567812345678\u0027"},{"line_number":85,"context_line":"  otherdata\u003d\u0027another value\u0027"}],"source_content_type":"text/x-rst","patch_set":2,"id":"3a961159_a6e9de80","line":82,"updated":"2015-01-06 16:33:38.000000000","message":"Do we really need this feature (doc string injection)? How should it works? Update doc string in source file or do injection in run time? \nAlso I guess it will be good to specify exact place in doc strings where we want to inject our metadata.","commit_id":"8ff284cbf39477d5a843b06126c00fe91a68a2df"},{"author":{"_account_id":5196,"name":"Matthew Treinish","email":"mtreinish@kortar.org","username":"treinish"},"change_message_id":"a9188c32851604599e9b085b250be662051d1c32","unresolved":false,"context_lines":[{"line_number":79,"context_line":"  uuid: \u002712345678-1234-5678-1234-567812345678\u0027"},{"line_number":80,"context_line":"  otherdata :\u0027another value\u0027"},{"line_number":81,"context_line":""},{"line_number":82,"context_line":"to the doc string and add the testtools attr::"},{"line_number":83,"context_line":""},{"line_number":84,"context_line":"  uuid\u003d\u002712345678-1234-5678-1234-567812345678\u0027"},{"line_number":85,"context_line":"  otherdata\u003d\u0027another value\u0027"}],"source_content_type":"text/x-rst","patch_set":2,"id":"3a961159_95d31837","line":82,"in_reply_to":"3a961159_a6e9de80","updated":"2015-01-07 02:31:19.000000000","message":"Yes, I agree probably being explicit here about why we\u0027re adding this to the docstring. Honestly this spec is fairly straightforward in implementation why it matters is for documenting the \u0027why\u0027 because this will be archived and we can refer back to it when needed.\n\nFWIW, I understood the reason being for when we enabled sphinx autodoc generation on test files we can use the uuid (and other meta I guess) for a consistent lookup between versions of tempest.","commit_id":"8ff284cbf39477d5a843b06126c00fe91a68a2df"},{"author":{"_account_id":5196,"name":"Matthew Treinish","email":"mtreinish@kortar.org","username":"treinish"},"change_message_id":"a9188c32851604599e9b085b250be662051d1c32","unresolved":false,"context_lines":[{"line_number":87,"context_line":"**hacking**"},{"line_number":88,"context_line":""},{"line_number":89,"context_line":"New hacking will need to be added to insure that a decorator is set for all"},{"line_number":90,"context_line":"tests."},{"line_number":91,"context_line":""},{"line_number":92,"context_line":""},{"line_number":93,"context_line":"Assignee(s)"}],"source_content_type":"text/x-rst","patch_set":2,"id":"3a961159_9888af23","line":90,"updated":"2015-01-07 02:31:19.000000000","message":"So I think you need to elaborate on this a bit. This is really 2 things the first is just adding a new rule method to tempest/hacking/checks.py (which you should just be clear about)\n\nThe other aspect is the uniqueness check. Since adding a uuid is still a manual step (even with calling a script to do it) we need to ensure that there aren\u0027t any duplicate ids being used. I can easily envision a developer just copy and pasting a uuid accidentally when adding a new test, and as a reviewer I have no way of identifying that this is the case when there are ~1500 tests in the tree.\n\nI\u0027m also pretty sure this will have to be an external tool (not in the pep8/flake8/hacking framework) because it will require scanning all the files at once. But, this shouldn\u0027t be an issue since you can just put it in the pep8 tox target, like we do for check_uptodate.sh with the sample config. (I guess a unit test would be possible too, but I think a separate tool would be simpler)","commit_id":"8ff284cbf39477d5a843b06126c00fe91a68a2df"},{"author":{"_account_id":5196,"name":"Matthew Treinish","email":"mtreinish@kortar.org","username":"treinish"},"change_message_id":"a9188c32851604599e9b085b250be662051d1c32","unresolved":false,"context_lines":[{"line_number":111,"context_line":"- creation of the generate_meta tool"},{"line_number":112,"context_line":"- use the generate_meta tool to generate default meta data for all existing"},{"line_number":113,"context_line":"  tests."},{"line_number":114,"context_line":"- implementation of gate tests"},{"line_number":115,"context_line":""},{"line_number":116,"context_line":"Dependencies"},{"line_number":117,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"}],"source_content_type":"text/x-rst","patch_set":2,"id":"3a961159_b8ee4b20","line":114,"updated":"2015-01-07 02:31:19.000000000","message":"So there\u0027s a bit of a chicken and egg problem here. We want to have enforcement on as soon as we populate all the decorators. While I\u0027m sure the generate_meta tool will be fine as a reviewer I don\u0027t want to have to worry about it.\n\nI think we need to shake up these last 2 bullets up to make sure that the uniqueness check tool is in the repo before we start adding uuids. That way we can land the big bang patch and the hacking check in one big patch. (which will also add the tool to the tox target. If all the auto enforcement is enabled at the same time as the uuid adds then the ~1500 new lines for the decorators don\u0027t have to be reviewed.","commit_id":"8ff284cbf39477d5a843b06126c00fe91a68a2df"},{"author":{"_account_id":217,"name":"Rob Hirschfeld","email":"rob@zehicle.com","username":"robhirschfeld"},"change_message_id":"547deeca9ef0cdbd95993ac0b4506de058615dd6","unresolved":false,"context_lines":[{"line_number":111,"context_line":"- creation of the generate_meta tool"},{"line_number":112,"context_line":"- use the generate_meta tool to generate default meta data for all existing"},{"line_number":113,"context_line":"  tests."},{"line_number":114,"context_line":"- implementation of gate tests"},{"line_number":115,"context_line":""},{"line_number":116,"context_line":"Dependencies"},{"line_number":117,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"}],"source_content_type":"text/x-rst","patch_set":2,"id":"3a961159_7d344adf","line":114,"in_reply_to":"3a961159_b8ee4b20","updated":"2015-01-12 21:04:45.000000000","message":"I like this idea; however, we may be able to start from the egg.\n\nFrom a refstack perspective, it would be possible to have them come in over time (a few weeks) so that we don\u0027t have a synchronize the change over every project.  We already have to deal w/ tests that don\u0027t have UUIDs.  This is a needed improvement, not a short term blocker.","commit_id":"8ff284cbf39477d5a843b06126c00fe91a68a2df"},{"author":{"_account_id":9377,"name":"Sergey Slipushenko","email":"slipushenko@gmail.com","username":"sslypushenko"},"change_message_id":"78ff004b4ec501fd3f02d2bb48c3700514f829a2","unresolved":false,"context_lines":[{"line_number":49,"context_line":""},{"line_number":50,"context_line":"- uniqueness tester / detect missing meta / insert meta"},{"line_number":51,"context_line":""},{"line_number":52,"context_line":"This tool will check for unique UUID decorators for all test_* methods in"},{"line_number":53,"context_line":"test_*py files. This is done in a single pass to build a list of undecorated"},{"line_number":54,"context_line":"tests, which can the be used as a gate check in the qa pipeline. From"},{"line_number":55,"context_line":"this list another tool will generate unique UUID metadata and insert it"}],"source_content_type":"text/x-rst","patch_set":11,"id":"da86d52c_29ab60d7","line":52,"updated":"2015-02-02 23:08:39.000000000","message":"Unfortunately, it is not enough to look-up all test_* methods. Due to recommended workflow, negative tests in Tempest should be auto generated:\nhttp://docs.openstack.org/developer/tempest/HACKING.html#negative-tests\nSo, we need a solution for tagging such kind of tests.","commit_id":"691bde6086c1a24863227fd456783d76db4060bb"},{"author":{"_account_id":5196,"name":"Matthew Treinish","email":"mtreinish@kortar.org","username":"treinish"},"change_message_id":"57723756743ee1a90ef0cef86843c45bc0eb5d30","unresolved":false,"context_lines":[{"line_number":49,"context_line":""},{"line_number":50,"context_line":"- uniqueness tester / detect missing meta / insert meta"},{"line_number":51,"context_line":""},{"line_number":52,"context_line":"This tool will check for unique UUID decorators for all test_* methods in"},{"line_number":53,"context_line":"test_*py files. This is done in a single pass to build a list of undecorated"},{"line_number":54,"context_line":"tests, which can the be used as a gate check in the qa pipeline. From"},{"line_number":55,"context_line":"this list another tool will generate unique UUID metadata and insert it"}],"source_content_type":"text/x-rst","patch_set":11,"id":"da86d52c_a91cb078","line":52,"in_reply_to":"da86d52c_29ab60d7","updated":"2015-02-03 00:05:38.000000000","message":"So, I actually don\u0027t think we need to worry about this at a first pass. Adding a note about this and the other use of test scenarios would\u0027ve been nice, but I\u0027d rather see this land for traditional tests before we worry about that. We have very few autogenerated tests at this point so it\u0027s not too large a concern.\n\nIIRC the autogenerated tests still need to have a stub test_* method in the files where testscenarios is being called over the matrix for. If we tag this with the decorator all children tests that our iterating over the schema will have the tag, which I think is probably fine. If my memory is correct then I just view it as just an extension of the subclass case.","commit_id":"691bde6086c1a24863227fd456783d76db4060bb"},{"author":{"_account_id":9377,"name":"Sergey Slipushenko","email":"slipushenko@gmail.com","username":"sslypushenko"},"change_message_id":"69ec89ae780d1932a051575d48173b2063e5f683","unresolved":false,"context_lines":[{"line_number":49,"context_line":""},{"line_number":50,"context_line":"- uniqueness tester / detect missing meta / insert meta"},{"line_number":51,"context_line":""},{"line_number":52,"context_line":"This tool will check for unique UUID decorators for all test_* methods in"},{"line_number":53,"context_line":"test_*py files. This is done in a single pass to build a list of undecorated"},{"line_number":54,"context_line":"tests, which can the be used as a gate check in the qa pipeline. From"},{"line_number":55,"context_line":"this list another tool will generate unique UUID metadata and insert it"}],"source_content_type":"text/x-rst","patch_set":11,"id":"da86d52c_b3eb0812","line":52,"in_reply_to":"da86d52c_a91cb078","updated":"2015-02-04 00:24:16.000000000","message":"Actually, not exactly. For example here:\nhttps://github.com/openstack/tempest/blob/master/tempest/api/compute/flavors/test_flavors_negative.py\nthere is no stub methods here. \n\nBut I agree with you that we should keep things move forward.","commit_id":"691bde6086c1a24863227fd456783d76db4060bb"},{"author":{"_account_id":9377,"name":"Sergey Slipushenko","email":"slipushenko@gmail.com","username":"sslypushenko"},"change_message_id":"78ff004b4ec501fd3f02d2bb48c3700514f829a2","unresolved":false,"context_lines":[{"line_number":76,"context_line":"  \u0027@test.meta(uuid\u003d\u002712345678-1234-5678-1234-567812345678\u0027,"},{"line_number":77,"context_line":"              otherdata\u003d\u0027another value\u0027)"},{"line_number":78,"context_line":""},{"line_number":79,"context_line":"For sphinx autodoc generation it will append the following to the doc string::"},{"line_number":80,"context_line":""},{"line_number":81,"context_line":"    uuid: \u002712345678-1234-5678-1234-567812345678\u0027"},{"line_number":82,"context_line":"    otherdata :\u0027another value\u0027"}],"source_content_type":"text/x-rst","patch_set":11,"id":"da86d52c_299fe051","line":79,"updated":"2015-02-02 23:08:39.000000000","message":"I\u0027m not sure that this is necessary. \nFirst of all, we didn\u0027t have auto generated documentation covered tempest tests (please correct me if I am wrong).\n\nRight now huge amount of tests have doc strings like this \nhttps://github.com/openstack/tempest/blob/master/tempest/api/identity/admin/test_services.py#L80\nAutomatically added doc string with uuid will look like a mess in this case.\n\nEven if test has proper doc string. Duplication information of test uid will look strange. For example test type is not duplicated in doc string.\n\nFinally, I guess that we can extract information about test uid directly from decorator during doc generation (I have no proof here, I don\u0027t see any obstacles for that)","commit_id":"691bde6086c1a24863227fd456783d76db4060bb"},{"author":{"_account_id":5196,"name":"Matthew Treinish","email":"mtreinish@kortar.org","username":"treinish"},"change_message_id":"57723756743ee1a90ef0cef86843c45bc0eb5d30","unresolved":false,"context_lines":[{"line_number":76,"context_line":"  \u0027@test.meta(uuid\u003d\u002712345678-1234-5678-1234-567812345678\u0027,"},{"line_number":77,"context_line":"              otherdata\u003d\u0027another value\u0027)"},{"line_number":78,"context_line":""},{"line_number":79,"context_line":"For sphinx autodoc generation it will append the following to the doc string::"},{"line_number":80,"context_line":""},{"line_number":81,"context_line":"    uuid: \u002712345678-1234-5678-1234-567812345678\u0027"},{"line_number":82,"context_line":"    otherdata :\u0027another value\u0027"}],"source_content_type":"text/x-rst","patch_set":11,"id":"da86d52c_84c8b323","line":79,"in_reply_to":"da86d52c_299fe051","updated":"2015-02-03 00:05:38.000000000","message":"If you have the decorator either append or pre-pend the string into the docstring to the modified method in a known format and that is all that\u0027s needed here. The doc autogeneration is not done on test files right now, mostly because it would look like an incosistent mess, because we had to rip a lot of docstrings out of tempest tests from back in the days things ran with nose. And the docstrings that are there are of questionable value.\n\nBut, if it\u0027s in the docstring from the decorator this should work when the time is right to start using autodoc having the info there is what we want so there is a mapping in the docs to the uuid. Honestly the docstring injection is the whole reason for having a separate decorator because we could just reuse @test.attr() to get the \u0027uuid\u003dX\u0027 in the subunit stream with the test_id and do everything else using that instead.","commit_id":"691bde6086c1a24863227fd456783d76db4060bb"},{"author":{"_account_id":9377,"name":"Sergey Slipushenko","email":"slipushenko@gmail.com","username":"sslypushenko"},"change_message_id":"69ec89ae780d1932a051575d48173b2063e5f683","unresolved":false,"context_lines":[{"line_number":76,"context_line":"  \u0027@test.meta(uuid\u003d\u002712345678-1234-5678-1234-567812345678\u0027,"},{"line_number":77,"context_line":"              otherdata\u003d\u0027another value\u0027)"},{"line_number":78,"context_line":""},{"line_number":79,"context_line":"For sphinx autodoc generation it will append the following to the doc string::"},{"line_number":80,"context_line":""},{"line_number":81,"context_line":"    uuid: \u002712345678-1234-5678-1234-567812345678\u0027"},{"line_number":82,"context_line":"    otherdata :\u0027another value\u0027"}],"source_content_type":"text/x-rst","patch_set":11,"id":"da86d52c_68fa7b44","line":79,"in_reply_to":"da86d52c_84c8b323","updated":"2015-02-04 00:24:16.000000000","message":"I have understood this point wrong. If have no plans to update docstrings in source files and just provide on-fly metadata injection than it will work for me.","commit_id":"691bde6086c1a24863227fd456783d76db4060bb"}]}