)]}'
{"specs/tempest/nova-microversions-tests.rst":[{"author":{"_account_id":8556,"name":"Ghanshyam Maan","display_name":"Ghanshyam Maan","email":"gmaan.os14@gmail.com","username":"ghanshyam"},"change_message_id":"3ca7bf61d7a4616ae39aa36e89ca5a7dad3b2a8b","unresolved":false,"context_lines":[{"line_number":61,"context_line":""},{"line_number":62,"context_line":"* Make service clients switch response definition for each microversion"},{"line_number":63,"context_line":"  Service clients of Nova will switch the definition based on the microversion."},{"line_number":64,"context_line":""},{"line_number":65,"context_line":"Which microversions on the gate"},{"line_number":66,"context_line":"-------------------------------"},{"line_number":67,"context_line":""}],"source_content_type":"text/x-rst","patch_set":3,"id":"1a4dcd0f_7cbdb100","line":64,"updated":"2015-08-07 09:41:58.000000000","message":"As we cannot tests each microversion in Tempest and gate, are we going to keep removing older versions testing from Tempest as well from gate?\n\nI mean-\n- Liberty, Tempest/gate tests v2.1, v2.3, v2.11(max version at liberty release)\n- Mitaak, we add v2.15 (max version in Mitaka resales)\n- N, we add v2.20 (max version in N resales) \n\nSo will Tempest/gate keep all (v2.1, v2.3, v2.11, v2.15, v2.20) version testing? \n\nTempest side it might be ok but on gate it would not be good to tests all. we might need some policy for gate job. 1 way is we keep \n-min\n-max\n- interesting mivroversion (max version at release) only for supported stable branches\n\n So in N it looks like - v2.1_job , v2.11_job, v2.15_job, v2.20_job (remove v2.3_job,)","commit_id":"5c82682627599a136e728142b9d84824f99de8c6"},{"author":{"_account_id":6167,"name":"Ken\u0027ichi Ohmichi","email":"ken1ohmichi@gmail.com","username":"oomichi"},"change_message_id":"d3523917932914ffa53796c1461bf19c2a7931db","unresolved":false,"context_lines":[{"line_number":61,"context_line":""},{"line_number":62,"context_line":"* Make service clients switch response definition for each microversion"},{"line_number":63,"context_line":"  Service clients of Nova will switch the definition based on the microversion."},{"line_number":64,"context_line":""},{"line_number":65,"context_line":"Which microversions on the gate"},{"line_number":66,"context_line":"-------------------------------"},{"line_number":67,"context_line":""}],"source_content_type":"text/x-rst","patch_set":3,"id":"ba15a1d1_2e3e95b8","line":64,"in_reply_to":"1a4dcd0f_7cbdb100","updated":"2015-09-15 23:16:17.000000000","message":"Yeah, that is very nice point.\nAfter bumping a lot of microversions, it is very hard to run all microversions of release boundaries.\nSo as you said, we need some policies how to select interesting microversions.\n\nOne option is that we select N(max), N-1 and N-2 release like Tempest gate as the interesting microversions.\nHow about this?","commit_id":"5c82682627599a136e728142b9d84824f99de8c6"},{"author":{"_account_id":8556,"name":"Ghanshyam Maan","display_name":"Ghanshyam Maan","email":"gmaan.os14@gmail.com","username":"ghanshyam"},"change_message_id":"0c62cb2297eabfd89ab73672ca214f74008894d4","unresolved":false,"context_lines":[{"line_number":61,"context_line":""},{"line_number":62,"context_line":"* Make service clients switch response definition for each microversion"},{"line_number":63,"context_line":"  Service clients of Nova will switch the definition based on the microversion."},{"line_number":64,"context_line":""},{"line_number":65,"context_line":"Which microversions on the gate"},{"line_number":66,"context_line":"-------------------------------"},{"line_number":67,"context_line":""}],"source_content_type":"text/x-rst","patch_set":3,"id":"3a29b11f_77031f0e","line":64,"in_reply_to":"ba15a1d1_2e3e95b8","updated":"2015-10-20 00:40:48.000000000","message":"Sorry, i did not fully understand that. You mean testing microversion of current branch only if so how about stable branches one. Correct me if i misunderstood your comment.","commit_id":"5c82682627599a136e728142b9d84824f99de8c6"},{"author":{"_account_id":8556,"name":"Ghanshyam Maan","display_name":"Ghanshyam Maan","email":"gmaan.os14@gmail.com","username":"ghanshyam"},"change_message_id":"3ca7bf61d7a4616ae39aa36e89ca5a7dad3b2a8b","unresolved":false,"context_lines":[{"line_number":83,"context_line":""},{"line_number":84,"context_line":"* check-tempest-dsvm-nova-v21-full (minimum, already exists)"},{"line_number":85,"context_line":"* check-tempest-dsvm-nova-v23-full (interesting)"},{"line_number":86,"context_line":"* check-tempest-dsvm-nova-maximum-mv-full (maximum)"},{"line_number":87,"context_line":""},{"line_number":88,"context_line":"These jobs should run on Nova and Tempest gates only."},{"line_number":89,"context_line":""}],"source_content_type":"text/x-rst","patch_set":3,"id":"1a4dcd0f_61666e52","line":86,"updated":"2015-08-07 09:41:58.000000000","message":"maximum microversion will gets increased during the release and we cannot tests with \"latest\" also, so Maximum microversion here means max version for which tempest has tests ?\n\nfor job \"check-tempest-dsvm-nova-maximum-mv-full\" - Each microversion bump needs changes in job every time. I do not have good solution for this but how about naming this job to actual version it is testing?\n\nFor example, we have version 2.11 out and Tempest has test for it\nthen max version job - \"check-tempest-dsvm-nova-v11-full\"\n\nif 2.12 is out then after Tempest tests we can rename it to \"check-tempest-dsvm-nova-v11-full\"  -\u003e \"check-tempest-dsvm-nova-v12-full\"\n\nSo that we get clear pic that what max version gate is testing.","commit_id":"5c82682627599a136e728142b9d84824f99de8c6"},{"author":{"_account_id":6167,"name":"Ken\u0027ichi Ohmichi","email":"ken1ohmichi@gmail.com","username":"oomichi"},"change_message_id":"d3523917932914ffa53796c1461bf19c2a7931db","unresolved":false,"context_lines":[{"line_number":83,"context_line":""},{"line_number":84,"context_line":"* check-tempest-dsvm-nova-v21-full (minimum, already exists)"},{"line_number":85,"context_line":"* check-tempest-dsvm-nova-v23-full (interesting)"},{"line_number":86,"context_line":"* check-tempest-dsvm-nova-maximum-mv-full (maximum)"},{"line_number":87,"context_line":""},{"line_number":88,"context_line":"These jobs should run on Nova and Tempest gates only."},{"line_number":89,"context_line":""}],"source_content_type":"text/x-rst","patch_set":3,"id":"ba15a1d1_4ea66911","line":86,"in_reply_to":"1a4dcd0f_61666e52","updated":"2015-09-15 23:16:17.000000000","message":"By using \"maximum\" word, we can avoid many changes of project-config when implementing a micro version test on tempest side.","commit_id":"5c82682627599a136e728142b9d84824f99de8c6"},{"author":{"_account_id":8556,"name":"Ghanshyam Maan","display_name":"Ghanshyam Maan","email":"gmaan.os14@gmail.com","username":"ghanshyam"},"change_message_id":"0c62cb2297eabfd89ab73672ca214f74008894d4","unresolved":false,"context_lines":[{"line_number":83,"context_line":""},{"line_number":84,"context_line":"* check-tempest-dsvm-nova-v21-full (minimum, already exists)"},{"line_number":85,"context_line":"* check-tempest-dsvm-nova-v23-full (interesting)"},{"line_number":86,"context_line":"* check-tempest-dsvm-nova-maximum-mv-full (maximum)"},{"line_number":87,"context_line":""},{"line_number":88,"context_line":"These jobs should run on Nova and Tempest gates only."},{"line_number":89,"context_line":""}],"source_content_type":"text/x-rst","patch_set":3,"id":"3a29b11f_bc0d1e40","line":86,"in_reply_to":"ba15a1d1_4ea66911","updated":"2015-10-20 00:40:48.000000000","message":"Yea, agree with that, we should avoid infra changes as much as possible. As discussed, one way is to have tempest supported max version and on gate job name we can have generic name as you suggested.","commit_id":"5c82682627599a136e728142b9d84824f99de8c6"},{"author":{"_account_id":8556,"name":"Ghanshyam Maan","display_name":"Ghanshyam Maan","email":"gmaan.os14@gmail.com","username":"ghanshyam"},"change_message_id":"94fb353560f55dcaf53b77737cb2030219dab390","unresolved":false,"context_lines":[{"line_number":75,"context_line":"  New configration options also are min_microversion and max_microversion,"},{"line_number":76,"context_line":"  and the test classes will be selected like the following::"},{"line_number":77,"context_line":""},{"line_number":78,"context_line":"    TestClass A: min_microversion \u003d None,  max_microversion \u003d \u0027latest\u0027"},{"line_number":79,"context_line":"    TestClass B: min_microversion \u003d None,  max_microversion \u003d \u00272.2\u0027"},{"line_number":80,"context_line":"    TestClass C: min_microversion \u003d \u00272.3\u0027, max_microversion \u003d \u0027latest\u0027"},{"line_number":81,"context_line":"    TestClass D: min_microversion \u003d \u00272.5\u0027, max_microversion \u003d \u00272.10\u0027"}],"source_content_type":"text/x-rst","patch_set":6,"id":"fa80f949_048ce933","line":78,"updated":"2015-11-05 09:55:52.000000000","message":"This is really nice to run all current existing tests with any configured micro version. Variation of response among versions will be taken care by response schema versioning But we might need to adjust some tests to handle all versions if they are verifying some attributes which has been changed in some version. \n\nI think that can be handle again by limiting max_microversion for that specific test class.","commit_id":"a93b635a7e90e3206f82dde77c99444760bf1824"},{"author":{"_account_id":6167,"name":"Ken\u0027ichi Ohmichi","email":"ken1ohmichi@gmail.com","username":"oomichi"},"change_message_id":"f6c7b7cfeba3c3caea12a17a7bdcf67c834a51c4","unresolved":false,"context_lines":[{"line_number":75,"context_line":"  New configration options also are min_microversion and max_microversion,"},{"line_number":76,"context_line":"  and the test classes will be selected like the following::"},{"line_number":77,"context_line":""},{"line_number":78,"context_line":"    TestClass A: min_microversion \u003d None,  max_microversion \u003d \u0027latest\u0027"},{"line_number":79,"context_line":"    TestClass B: min_microversion \u003d None,  max_microversion \u003d \u00272.2\u0027"},{"line_number":80,"context_line":"    TestClass C: min_microversion \u003d \u00272.3\u0027, max_microversion \u003d \u0027latest\u0027"},{"line_number":81,"context_line":"    TestClass D: min_microversion \u003d \u00272.5\u0027, max_microversion \u003d \u00272.10\u0027"}],"source_content_type":"text/x-rst","patch_set":6,"id":"fa80f949_11ce4682","line":78,"in_reply_to":"fa80f949_048ce933","updated":"2015-11-06 01:52:16.000000000","message":"In my idea, test classes need to pass microversion to service client for sending it on api header, and service client needs to switch json-schema based on the microversion.\nSo I think we don\u0027t need to consider the switching on the test class side.","commit_id":"a93b635a7e90e3206f82dde77c99444760bf1824"},{"author":{"_account_id":8556,"name":"Ghanshyam Maan","display_name":"Ghanshyam Maan","email":"gmaan.os14@gmail.com","username":"ghanshyam"},"change_message_id":"94fb353560f55dcaf53b77737cb2030219dab390","unresolved":false,"context_lines":[{"line_number":129,"context_line":"  feature. So we need to define the responses for each microversions and"},{"line_number":130,"context_line":"  Tempest needs to switch the response definition of JSON-Schema by the"},{"line_number":131,"context_line":"  microversion."},{"line_number":132,"context_line":"  Now the responses are defined under tempest/api_schema/response/compute/"},{"line_number":133,"context_line":"  and the one of the base microversion v2.1 is defined under ./v2_1 ."},{"line_number":134,"context_line":"  Each microversion is a little different from the previous one and it is"},{"line_number":135,"context_line":"  necessary to define the difference under ./v2_2, ./v2_3, etc."}],"source_content_type":"text/x-rst","patch_set":6,"id":"fa80f949_8478d92c","line":132,"updated":"2015-11-05 09:55:52.000000000","message":"now they are in lib, you might want to change this path :)","commit_id":"a93b635a7e90e3206f82dde77c99444760bf1824"},{"author":{"_account_id":6167,"name":"Ken\u0027ichi Ohmichi","email":"ken1ohmichi@gmail.com","username":"oomichi"},"change_message_id":"f6c7b7cfeba3c3caea12a17a7bdcf67c834a51c4","unresolved":false,"context_lines":[{"line_number":129,"context_line":"  feature. So we need to define the responses for each microversions and"},{"line_number":130,"context_line":"  Tempest needs to switch the response definition of JSON-Schema by the"},{"line_number":131,"context_line":"  microversion."},{"line_number":132,"context_line":"  Now the responses are defined under tempest/api_schema/response/compute/"},{"line_number":133,"context_line":"  and the one of the base microversion v2.1 is defined under ./v2_1 ."},{"line_number":134,"context_line":"  Each microversion is a little different from the previous one and it is"},{"line_number":135,"context_line":"  necessary to define the difference under ./v2_2, ./v2_3, etc."}],"source_content_type":"text/x-rst","patch_set":6,"id":"fa80f949_71a312b3","line":132,"in_reply_to":"fa80f949_8478d92c","updated":"2015-11-06 01:52:16.000000000","message":"Nice catch, I will update it:-)","commit_id":"a93b635a7e90e3206f82dde77c99444760bf1824"},{"author":{"_account_id":8556,"name":"Ghanshyam Maan","display_name":"Ghanshyam Maan","email":"gmaan.os14@gmail.com","username":"ghanshyam"},"change_message_id":"94fb353560f55dcaf53b77737cb2030219dab390","unresolved":false,"context_lines":[{"line_number":158,"context_line":"----------"},{"line_number":159,"context_line":""},{"line_number":160,"context_line":"Target Milestone for completion:"},{"line_number":161,"context_line":"  Mitaka-1"},{"line_number":162,"context_line":""},{"line_number":163,"context_line":"Work Items"},{"line_number":164,"context_line":"----------"}],"source_content_type":"text/x-rst","patch_set":6,"id":"fa80f949_e4dd2515","line":161,"updated":"2015-11-05 09:55:52.000000000","message":"Including schema versioning, there might be lot of work. Should we change it to M-2?","commit_id":"a93b635a7e90e3206f82dde77c99444760bf1824"},{"author":{"_account_id":6167,"name":"Ken\u0027ichi Ohmichi","email":"ken1ohmichi@gmail.com","username":"oomichi"},"change_message_id":"f6c7b7cfeba3c3caea12a17a7bdcf67c834a51c4","unresolved":false,"context_lines":[{"line_number":158,"context_line":"----------"},{"line_number":159,"context_line":""},{"line_number":160,"context_line":"Target Milestone for completion:"},{"line_number":161,"context_line":"  Mitaka-1"},{"line_number":162,"context_line":""},{"line_number":163,"context_line":"Work Items"},{"line_number":164,"context_line":"----------"}],"source_content_type":"text/x-rst","patch_set":6,"id":"fa80f949_91bd56b9","line":161,"in_reply_to":"fa80f949_e4dd2515","updated":"2015-11-06 01:52:16.000000000","message":"json-schema thing is for each microversion test, and I think we can implement them after this spec work because microversion continues implementing on nova/ironic side.\n\nSo it is enough to cover base part and a single microversion test (as sample) on this spec work IMO.","commit_id":"a93b635a7e90e3206f82dde77c99444760bf1824"},{"author":{"_account_id":8556,"name":"Ghanshyam Maan","display_name":"Ghanshyam Maan","email":"gmaan.os14@gmail.com","username":"ghanshyam"},"change_message_id":"94fb353560f55dcaf53b77737cb2030219dab390","unresolved":false,"context_lines":[{"line_number":162,"context_line":""},{"line_number":163,"context_line":"Work Items"},{"line_number":164,"context_line":"----------"},{"line_number":165,"context_line":""},{"line_number":166,"context_line":"* Implement base test classe for microversions"},{"line_number":167,"context_line":"* Add test classes for each microversion"},{"line_number":168,"context_line":"* Change the configrations on openstack-infra/project-config for master"}],"source_content_type":"text/x-rst","patch_set":6,"id":"fa80f949_04086990","line":165,"updated":"2015-11-05 09:55:52.000000000","message":"Also version the schemas in lib and support of passing micro version in req header for all request.","commit_id":"a93b635a7e90e3206f82dde77c99444760bf1824"},{"author":{"_account_id":6167,"name":"Ken\u0027ichi Ohmichi","email":"ken1ohmichi@gmail.com","username":"oomichi"},"change_message_id":"f6c7b7cfeba3c3caea12a17a7bdcf67c834a51c4","unresolved":false,"context_lines":[{"line_number":162,"context_line":""},{"line_number":163,"context_line":"Work Items"},{"line_number":164,"context_line":"----------"},{"line_number":165,"context_line":""},{"line_number":166,"context_line":"* Implement base test classe for microversions"},{"line_number":167,"context_line":"* Add test classes for each microversion"},{"line_number":168,"context_line":"* Change the configrations on openstack-infra/project-config for master"}],"source_content_type":"text/x-rst","patch_set":6,"id":"fa80f949_d1bf5ea8","line":165,"in_reply_to":"fa80f949_04086990","updated":"2015-11-06 01:52:16.000000000","message":"nice point. I will add working item for passing microversion for service clients.\n\nthe scehama thing is fine to be out of scope as the first step as I said on the above.","commit_id":"a93b635a7e90e3206f82dde77c99444760bf1824"},{"author":{"_account_id":5196,"name":"Matthew Treinish","email":"mtreinish@kortar.org","username":"treinish"},"change_message_id":"c83db4a40de78ca90fee2a483dbdcf1b7d36a921","unresolved":false,"context_lines":[{"line_number":84,"context_line":"  | Configration       | Test classes                                     |"},{"line_number":85,"context_line":"  | (min,    max)      | (Passed microversion)                            |"},{"line_number":86,"context_line":"  +\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d+\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d+"},{"line_number":87,"context_line":"  | None,     None     | A(Not passed), B(Not passed)                     |"},{"line_number":88,"context_line":"  +--------------------+--------------------------------------------------+"},{"line_number":89,"context_line":"  | None,     \u00272.3\u0027    | A(Not passed), B(Not passed), C(\u00272.3\u0027)           |"},{"line_number":90,"context_line":"  +--------------------+--------------------------------------------------+"}],"source_content_type":"text/x-rst","patch_set":7,"id":"fa80f949_29f99ec7","line":87,"updated":"2015-11-06 04:22:34.000000000","message":"nit: we probably should say c and d are skipped","commit_id":"426ffac20811ccd2e87da597f17303da01ae893f"},{"author":{"_account_id":8556,"name":"Ghanshyam Maan","display_name":"Ghanshyam Maan","email":"gmaan.os14@gmail.com","username":"ghanshyam"},"change_message_id":"bfac011528eabd1dbd60c80d68c5dee167056226","unresolved":false,"context_lines":[{"line_number":84,"context_line":"  | Configration       | Test classes                                     |"},{"line_number":85,"context_line":"  | (min,    max)      | (Passed microversion)                            |"},{"line_number":86,"context_line":"  +\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d+\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d+"},{"line_number":87,"context_line":"  | None,     None     | A(Not passed), B(Not passed)                     |"},{"line_number":88,"context_line":"  +--------------------+--------------------------------------------------+"},{"line_number":89,"context_line":"  | None,     \u00272.3\u0027    | A(Not passed), B(Not passed), C(\u00272.3\u0027)           |"},{"line_number":90,"context_line":"  +--------------------+--------------------------------------------------+"}],"source_content_type":"text/x-rst","patch_set":7,"id":"da85f559_261dcc66","line":87,"in_reply_to":"fa80f949_29f99ec7","updated":"2015-11-07 02:02:09.000000000","message":"Done","commit_id":"426ffac20811ccd2e87da597f17303da01ae893f"},{"author":{"_account_id":5196,"name":"Matthew Treinish","email":"mtreinish@kortar.org","username":"treinish"},"change_message_id":"c83db4a40de78ca90fee2a483dbdcf1b7d36a921","unresolved":false,"context_lines":[{"line_number":86,"context_line":"  +\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d+\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d+"},{"line_number":87,"context_line":"  | None,     None     | A(Not passed), B(Not passed)                     |"},{"line_number":88,"context_line":"  +--------------------+--------------------------------------------------+"},{"line_number":89,"context_line":"  | None,     \u00272.3\u0027    | A(Not passed), B(Not passed), C(\u00272.3\u0027)           |"},{"line_number":90,"context_line":"  +--------------------+--------------------------------------------------+"},{"line_number":91,"context_line":"  | \u00272.2\u0027,    \u0027latest\u0027 | A(\u00272.2\u0027), B(\u00272.2\u0027), C(\u00272.3\u0027), D(\u00272.5\u0027)           |"},{"line_number":92,"context_line":"  +--------------------+--------------------------------------------------+"}],"source_content_type":"text/x-rst","patch_set":7,"id":"fa80f949_c901f2b0","line":89,"updated":"2015-11-06 04:22:34.000000000","message":"nit d is skipped right?","commit_id":"426ffac20811ccd2e87da597f17303da01ae893f"},{"author":{"_account_id":8556,"name":"Ghanshyam Maan","display_name":"Ghanshyam Maan","email":"gmaan.os14@gmail.com","username":"ghanshyam"},"change_message_id":"bfac011528eabd1dbd60c80d68c5dee167056226","unresolved":false,"context_lines":[{"line_number":86,"context_line":"  +\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d+\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d+"},{"line_number":87,"context_line":"  | None,     None     | A(Not passed), B(Not passed)                     |"},{"line_number":88,"context_line":"  +--------------------+--------------------------------------------------+"},{"line_number":89,"context_line":"  | None,     \u00272.3\u0027    | A(Not passed), B(Not passed), C(\u00272.3\u0027)           |"},{"line_number":90,"context_line":"  +--------------------+--------------------------------------------------+"},{"line_number":91,"context_line":"  | \u00272.2\u0027,    \u0027latest\u0027 | A(\u00272.2\u0027), B(\u00272.2\u0027), C(\u00272.3\u0027), D(\u00272.5\u0027)           |"},{"line_number":92,"context_line":"  +--------------------+--------------------------------------------------+"}],"source_content_type":"text/x-rst","patch_set":7,"id":"da85f559_461a5070","line":89,"in_reply_to":"fa80f949_c901f2b0","updated":"2015-11-07 02:02:09.000000000","message":"Done","commit_id":"426ffac20811ccd2e87da597f17303da01ae893f"},{"author":{"_account_id":5196,"name":"Matthew Treinish","email":"mtreinish@kortar.org","username":"treinish"},"change_message_id":"c83db4a40de78ca90fee2a483dbdcf1b7d36a921","unresolved":false,"context_lines":[{"line_number":92,"context_line":"  +--------------------+--------------------------------------------------+"},{"line_number":93,"context_line":"  | \u00272.2\u0027,    \u00272.3\u0027    | A(\u00272.2\u0027), B(\u00272.2\u0027), C(\u00272.3\u0027)                     |"},{"line_number":94,"context_line":"  +--------------------+--------------------------------------------------+"},{"line_number":95,"context_line":"  | \u00272.10\u0027,   \u00272.10\u0027   | A(\u00272.10\u0027), C(\u00272.10\u0027), D(\u00272.10\u0027)                  |"},{"line_number":96,"context_line":"  +--------------------+--------------------------------------------------+"},{"line_number":97,"context_line":"  | None,     \u0027latest\u0027 | A(Not passed), B(Not passed), C(\u00272.3\u0027), D(\u00272.5\u0027) |"},{"line_number":98,"context_line":"  +--------------------+--------------------------------------------------+"}],"source_content_type":"text/x-rst","patch_set":7,"id":"fa80f949_c95332e1","line":95,"updated":"2015-11-06 04:22:34.000000000","message":"nit: I think we should be clear that we\u0027re skipping B here","commit_id":"426ffac20811ccd2e87da597f17303da01ae893f"},{"author":{"_account_id":8556,"name":"Ghanshyam Maan","display_name":"Ghanshyam Maan","email":"gmaan.os14@gmail.com","username":"ghanshyam"},"change_message_id":"bfac011528eabd1dbd60c80d68c5dee167056226","unresolved":false,"context_lines":[{"line_number":92,"context_line":"  +--------------------+--------------------------------------------------+"},{"line_number":93,"context_line":"  | \u00272.2\u0027,    \u00272.3\u0027    | A(\u00272.2\u0027), B(\u00272.2\u0027), C(\u00272.3\u0027)                     |"},{"line_number":94,"context_line":"  +--------------------+--------------------------------------------------+"},{"line_number":95,"context_line":"  | \u00272.10\u0027,   \u00272.10\u0027   | A(\u00272.10\u0027), C(\u00272.10\u0027), D(\u00272.10\u0027)                  |"},{"line_number":96,"context_line":"  +--------------------+--------------------------------------------------+"},{"line_number":97,"context_line":"  | None,     \u0027latest\u0027 | A(Not passed), B(Not passed), C(\u00272.3\u0027), D(\u00272.5\u0027) |"},{"line_number":98,"context_line":"  +--------------------+--------------------------------------------------+"}],"source_content_type":"text/x-rst","patch_set":7,"id":"da85f559_e62584ae","line":95,"in_reply_to":"fa80f949_c95332e1","updated":"2015-11-07 02:02:09.000000000","message":"Done","commit_id":"426ffac20811ccd2e87da597f17303da01ae893f"},{"author":{"_account_id":5196,"name":"Matthew Treinish","email":"mtreinish@kortar.org","username":"treinish"},"change_message_id":"c83db4a40de78ca90fee2a483dbdcf1b7d36a921","unresolved":false,"context_lines":[{"line_number":96,"context_line":"  +--------------------+--------------------------------------------------+"},{"line_number":97,"context_line":"  | None,     \u0027latest\u0027 | A(Not passed), B(Not passed), C(\u00272.3\u0027), D(\u00272.5\u0027) |"},{"line_number":98,"context_line":"  +--------------------+--------------------------------------------------+"},{"line_number":99,"context_line":"  | \u0027latest\u0027, \u0027latest\u0027 | A(\u0027latest\u0027), C(\u0027latest\u0027)                         |"},{"line_number":100,"context_line":"  +--------------------+--------------------------------------------------+"},{"line_number":101,"context_line":""},{"line_number":102,"context_line":"  So basically the configration min_microversion value is passed on the"}],"source_content_type":"text/x-rst","patch_set":7,"id":"fa80f949_e95076d6","line":99,"updated":"2015-11-06 04:22:34.000000000","message":"nit: skipping b and d","commit_id":"426ffac20811ccd2e87da597f17303da01ae893f"},{"author":{"_account_id":8556,"name":"Ghanshyam Maan","display_name":"Ghanshyam Maan","email":"gmaan.os14@gmail.com","username":"ghanshyam"},"change_message_id":"bfac011528eabd1dbd60c80d68c5dee167056226","unresolved":false,"context_lines":[{"line_number":96,"context_line":"  +--------------------+--------------------------------------------------+"},{"line_number":97,"context_line":"  | None,     \u0027latest\u0027 | A(Not passed), B(Not passed), C(\u00272.3\u0027), D(\u00272.5\u0027) |"},{"line_number":98,"context_line":"  +--------------------+--------------------------------------------------+"},{"line_number":99,"context_line":"  | \u0027latest\u0027, \u0027latest\u0027 | A(\u0027latest\u0027), C(\u0027latest\u0027)                         |"},{"line_number":100,"context_line":"  +--------------------+--------------------------------------------------+"},{"line_number":101,"context_line":""},{"line_number":102,"context_line":"  So basically the configration min_microversion value is passed on the"}],"source_content_type":"text/x-rst","patch_set":7,"id":"da85f559_a61f7c5a","line":99,"in_reply_to":"fa80f949_e95076d6","updated":"2015-11-07 02:02:09.000000000","message":"Done","commit_id":"426ffac20811ccd2e87da597f17303da01ae893f"},{"author":{"_account_id":5196,"name":"Matthew Treinish","email":"mtreinish@kortar.org","username":"treinish"},"change_message_id":"c83db4a40de78ca90fee2a483dbdcf1b7d36a921","unresolved":false,"context_lines":[{"line_number":104,"context_line":"  is bigger, the class\u0027 min_microversion is passed instead."},{"line_number":105,"context_line":"  If you want to pass the maximum supported microversion for all tests,"},{"line_number":106,"context_line":"  you need to specify the microversion as the configration min_microversion"},{"line_number":107,"context_line":"  like the above 5th example."},{"line_number":108,"context_line":"  The default configuration values should be (None, None) like 1st example"},{"line_number":109,"context_line":"  for running on the existing clouds which don\u0027t support microversions."},{"line_number":110,"context_line":"  So we need to change the configuration values with openstack-dev/devstack"}],"source_content_type":"text/x-rst","patch_set":7,"id":"fa80f949_490d02e7","line":107,"updated":"2015-11-06 04:22:34.000000000","message":"nit: I think the wording here is a bit akward. Maybe something like:\n\nif you\u0027d like to always send the maxmim micoversion you need to set the max_microversion and the min_microversion to be the same value, like the 5th example above.","commit_id":"426ffac20811ccd2e87da597f17303da01ae893f"},{"author":{"_account_id":8556,"name":"Ghanshyam Maan","display_name":"Ghanshyam Maan","email":"gmaan.os14@gmail.com","username":"ghanshyam"},"change_message_id":"bfac011528eabd1dbd60c80d68c5dee167056226","unresolved":false,"context_lines":[{"line_number":104,"context_line":"  is bigger, the class\u0027 min_microversion is passed instead."},{"line_number":105,"context_line":"  If you want to pass the maximum supported microversion for all tests,"},{"line_number":106,"context_line":"  you need to specify the microversion as the configration min_microversion"},{"line_number":107,"context_line":"  like the above 5th example."},{"line_number":108,"context_line":"  The default configuration values should be (None, None) like 1st example"},{"line_number":109,"context_line":"  for running on the existing clouds which don\u0027t support microversions."},{"line_number":110,"context_line":"  So we need to change the configuration values with openstack-dev/devstack"}],"source_content_type":"text/x-rst","patch_set":7,"id":"da85f559_c62400b1","line":107,"in_reply_to":"fa80f949_490d02e7","updated":"2015-11-07 02:02:09.000000000","message":"Done","commit_id":"426ffac20811ccd2e87da597f17303da01ae893f"},{"author":{"_account_id":6167,"name":"Ken\u0027ichi Ohmichi","email":"ken1ohmichi@gmail.com","username":"oomichi"},"change_message_id":"aa487e882fc483d55bec1d13735f53acfbab7b58","unresolved":false,"context_lines":[{"line_number":148,"context_line":"     Which includes skipping methods etc for microversion tests based on"},{"line_number":149,"context_line":"     provided configuration."},{"line_number":150,"context_line":""},{"line_number":151,"context_line":"     Tempest, test it on gate tehn migrate to Tempest-lib."},{"line_number":152,"context_line":"  #. Implement base framework for service clients to pass microversion"},{"line_number":153,"context_line":"     in request header."},{"line_number":154,"context_line":""}],"source_content_type":"text/x-rst","patch_set":10,"id":"da85f559_a17433fd","line":151,"updated":"2015-11-09 00:39:41.000000000","message":"typo: tehn","commit_id":"d32f64585a1b3efeb939958af03577cf37ddcd18"},{"author":{"_account_id":8556,"name":"Ghanshyam Maan","display_name":"Ghanshyam Maan","email":"gmaan.os14@gmail.com","username":"ghanshyam"},"change_message_id":"1b690588b563c804d908616874f39a42be25924a","unresolved":false,"context_lines":[{"line_number":71,"context_line":"  We need to specify test target microversions because the supported"},{"line_number":72,"context_line":"  microversions are different between OpenStack clouds. For operating"},{"line_number":73,"context_line":"  multiple microversion tests in a single Tempest operation, configration"},{"line_number":74,"context_line":"  options should represent the range of test target microversions."},{"line_number":75,"context_line":"  New configration options also are min_microversion and max_microversion,"},{"line_number":76,"context_line":"  and the test classes will be selected like the following::"},{"line_number":77,"context_line":""}],"source_content_type":"text/x-rst","patch_set":11,"id":"da85f559_1c5d66e9","line":74,"updated":"2015-11-09 01:18:17.000000000","message":"1 more scenario- how we are going to handle if Cloud support different range of microversion for different service.\n\nFor example - \nNova latest microversion is - 2.10\nIronic latest microversion is  -2.9\n\n-Tempest has tests for test_nova-2.10 (min-2.1, max-2.10) and test_ironic-2.9 (min-2.8, max-2.9)\n\n-Cloud support max version Nova-2.10 and Ironic-2.7\n\n-If User set config option as Min_version \u003d 2.1, max_version \u003d 2.10 (as per Nova version in Cloud)\n\nIn this case  test_ironic-2.9 will fail as Tempest will pass ironic 2.8 microversion in req for this test case.\n\nBasically here Tempest would not be able to handle different project supported microversion with single global configuration options?","commit_id":"afbe372610785e592c7617f86bad459427b2eef7"},{"author":{"_account_id":6167,"name":"Ken\u0027ichi Ohmichi","email":"ken1ohmichi@gmail.com","username":"oomichi"},"change_message_id":"2de1673a75c1c7301183d22dbe5b4b2e38124bc1","unresolved":false,"context_lines":[{"line_number":71,"context_line":"  We need to specify test target microversions because the supported"},{"line_number":72,"context_line":"  microversions are different between OpenStack clouds. For operating"},{"line_number":73,"context_line":"  multiple microversion tests in a single Tempest operation, configration"},{"line_number":74,"context_line":"  options should represent the range of test target microversions."},{"line_number":75,"context_line":"  New configration options also are min_microversion and max_microversion,"},{"line_number":76,"context_line":"  and the test classes will be selected like the following::"},{"line_number":77,"context_line":""}],"source_content_type":"text/x-rst","patch_set":11,"id":"da85f559_1cec86c6","line":74,"in_reply_to":"da85f559_1c5d66e9","updated":"2015-11-09 01:33:01.000000000","message":"Nice point, these options should be separated for each project. I will update the spec for clarifying.","commit_id":"afbe372610785e592c7617f86bad459427b2eef7"},{"author":{"_account_id":2750,"name":"Sean Dague","email":"sean@dague.net","username":"sdague"},"change_message_id":"cd8c7b761a70d5305ac786f322b46143e7fd488d","unresolved":false,"context_lines":[{"line_number":71,"context_line":"  We need to specify test target microversions because the supported"},{"line_number":72,"context_line":"  microversions are different between OpenStack clouds. For operating"},{"line_number":73,"context_line":"  multiple microversion tests in a single Tempest operation, configration"},{"line_number":74,"context_line":"  options should represent the range of test target microversions."},{"line_number":75,"context_line":"  New configration options also are min_microversion and max_microversion,"},{"line_number":76,"context_line":"  and the test classes will be selected like the following::"},{"line_number":77,"context_line":""}],"source_content_type":"text/x-rst","patch_set":11,"id":"da85f559_5107c2c9","line":74,"in_reply_to":"da85f559_1cec86c6","updated":"2015-11-11 16:29:50.000000000","message":"yes, these need to be per service. That was somewhat assumed. But it\u0027s fine to be explicit about it.","commit_id":"afbe372610785e592c7617f86bad459427b2eef7"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"af31cc3dab6bd5016572cc13d2f7353b68498b7e","unresolved":false,"context_lines":[{"line_number":19,"context_line":"Problem description"},{"line_number":20,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"},{"line_number":21,"context_line":""},{"line_number":22,"context_line":"On microversions mechanism, each microversion change is very small."},{"line_number":23,"context_line":"For example, version 2.2\u0027s change is::"},{"line_number":24,"context_line":""},{"line_number":25,"context_line":" Add \u0027type\u0027 to os-keypairs response"}],"source_content_type":"text/x-rst","patch_set":12,"id":"da85f559_40dfe60c","line":22,"updated":"2015-11-11 18:10:03.000000000","message":"Not every change is small, like 2.6 renamed an API for consoles:\n\nhttp://docs.openstack.org/developer/nova/api_microversion_history.html\n\nThat had impacts to novaclient because novaclient was blindly defaulting to 2.latest which would then break if you just tried getting vnc consoles from novaclient.  We\u0027ve since fixed that in novaclient such that our upper limit on the API request is the highest version we know we support in the client.","commit_id":"50f1294427391b7a3d7a5021b8444cf847e19e64"},{"author":{"_account_id":8556,"name":"Ghanshyam Maan","display_name":"Ghanshyam Maan","email":"gmaan.os14@gmail.com","username":"ghanshyam"},"change_message_id":"1d3e40ced876000f38174984e7cbda9b122bb333","unresolved":false,"context_lines":[{"line_number":19,"context_line":"Problem description"},{"line_number":20,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"},{"line_number":21,"context_line":""},{"line_number":22,"context_line":"On microversions mechanism, each microversion change is very small."},{"line_number":23,"context_line":"For example, version 2.2\u0027s change is::"},{"line_number":24,"context_line":""},{"line_number":25,"context_line":" Add \u0027type\u0027 to os-keypairs response"}],"source_content_type":"text/x-rst","patch_set":12,"id":"da85f559_d3f4045c","line":22,"in_reply_to":"da85f559_40dfe60c","updated":"2015-11-12 00:08:51.000000000","message":"Yea, that\u0027s right.\n\nFor \"latest\" thing, Tempest will be fine as long as min and max both version in configuration are latest (Then only \"latest\" will be sent to Nova).\n\nOtherwise we will send either configured min or test class min version.\n\nGate will have experimental/periodic job which will send always \u0027latest\u0027 and check what all version testing missing in Tempest.","commit_id":"50f1294427391b7a3d7a5021b8444cf847e19e64"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"af31cc3dab6bd5016572cc13d2f7353b68498b7e","unresolved":false,"context_lines":[{"line_number":51,"context_line":"        [..]"},{"line_number":52,"context_line":""},{"line_number":53,"context_line":"  as os-keypairs test class. In the above case, max_microversion is not"},{"line_number":54,"context_line":"  contained. That means unlimited as max_microversion. If we change the API"},{"line_number":55,"context_line":"  again with microversion \u00272.100\u0027 as an example, the test class will be::"},{"line_number":56,"context_line":""},{"line_number":57,"context_line":"    class KeyPairsV22Test(base.BaseKeypairTest):"}],"source_content_type":"text/x-rst","patch_set":12,"id":"da85f559_f46b3fb7","line":54,"updated":"2015-11-11 18:10:03.000000000","message":"So I guess if nova changes the API in some way, a dependency of the change in nova is going to be capping the max version for the test in Tempest. Makes sense, plus it would/should be obvious given the nova change would be failing the test in tempest.","commit_id":"50f1294427391b7a3d7a5021b8444cf847e19e64"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"af31cc3dab6bd5016572cc13d2f7353b68498b7e","unresolved":false,"context_lines":[{"line_number":121,"context_line":"  to specify it as experimental job to know we need to update Tempest for"},{"line_number":122,"context_line":"  supporting the latest microversion."},{"line_number":123,"context_line":""},{"line_number":124,"context_line":"  These configration options should be added for each project(Nova, Ironic,"},{"line_number":125,"context_line":"  etc.) because the microversion numbers are different between projects."},{"line_number":126,"context_line":""},{"line_number":127,"context_line":"JSON-Schema for each microversion (Nova specific)"}],"source_content_type":"text/x-rst","patch_set":12,"id":"da85f559_b44db73f","line":124,"updated":"2015-11-11 18:10:03.000000000","message":"An example of what this would actually look like in tempest.conf would be helpful.  I\u0027m not really following how the above table and description translates to how this looks in a config file.","commit_id":"50f1294427391b7a3d7a5021b8444cf847e19e64"},{"author":{"_account_id":8556,"name":"Ghanshyam Maan","display_name":"Ghanshyam Maan","email":"gmaan.os14@gmail.com","username":"ghanshyam"},"change_message_id":"1d3e40ced876000f38174984e7cbda9b122bb333","unresolved":false,"context_lines":[{"line_number":121,"context_line":"  to specify it as experimental job to know we need to update Tempest for"},{"line_number":122,"context_line":"  supporting the latest microversion."},{"line_number":123,"context_line":""},{"line_number":124,"context_line":"  These configration options should be added for each project(Nova, Ironic,"},{"line_number":125,"context_line":"  etc.) because the microversion numbers are different between projects."},{"line_number":126,"context_line":""},{"line_number":127,"context_line":"JSON-Schema for each microversion (Nova specific)"}],"source_content_type":"text/x-rst","patch_set":12,"id":"da85f559_ee611bd4","line":124,"in_reply_to":"da85f559_b44db73f","updated":"2015-11-12 00:08:51.000000000","message":"In conf we will have only 2 value min and max version. like  - https://review.openstack.org/#/c/242296/4/tempest/config.py\n\nBased on those each test class will be selected as skip or run depends on what valid range of version they are for.\n\nIf tests is for above version 2.8 and configured one is less than that then skip it.\n\nThat Table shows each combination of possibilities between configured versions and tests class one.","commit_id":"50f1294427391b7a3d7a5021b8444cf847e19e64"},{"author":{"_account_id":8556,"name":"Ghanshyam Maan","display_name":"Ghanshyam Maan","email":"gmaan.os14@gmail.com","username":"ghanshyam"},"change_message_id":"bbddb3357a2065945c99c4cc72b49c1649812cee","unresolved":false,"context_lines":[{"line_number":121,"context_line":"  to specify it as experimental job to know we need to update Tempest for"},{"line_number":122,"context_line":"  supporting the latest microversion."},{"line_number":123,"context_line":""},{"line_number":124,"context_line":"  These configration options should be added for each project(Nova, Ironic,"},{"line_number":125,"context_line":"  etc.) because the microversion numbers are different between projects."},{"line_number":126,"context_line":""},{"line_number":127,"context_line":"JSON-Schema for each microversion (Nova specific)"}],"source_content_type":"text/x-rst","patch_set":12,"id":"da85f559_24d091f5","line":124,"in_reply_to":"da85f559_ec0b5ed4","updated":"2015-11-12 10:01:34.000000000","message":"Nice. Done","commit_id":"50f1294427391b7a3d7a5021b8444cf847e19e64"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"388c3cef42a852c1266e1906d9ee7f8cad7e2a23","unresolved":false,"context_lines":[{"line_number":121,"context_line":"  to specify it as experimental job to know we need to update Tempest for"},{"line_number":122,"context_line":"  supporting the latest microversion."},{"line_number":123,"context_line":""},{"line_number":124,"context_line":"  These configration options should be added for each project(Nova, Ironic,"},{"line_number":125,"context_line":"  etc.) because the microversion numbers are different between projects."},{"line_number":126,"context_line":""},{"line_number":127,"context_line":"JSON-Schema for each microversion (Nova specific)"}],"source_content_type":"text/x-rst","patch_set":12,"id":"da85f559_ec0b5ed4","line":124,"in_reply_to":"da85f559_ee611bd4","updated":"2015-11-12 03:09:20.000000000","message":"OK, that\u0027s more clear then. It might be helpful for people that aren\u0027t working on this but use Tempest to give that example, or link to that review as a reference.","commit_id":"50f1294427391b7a3d7a5021b8444cf847e19e64"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"af31cc3dab6bd5016572cc13d2f7353b68498b7e","unresolved":false,"context_lines":[{"line_number":164,"context_line":"External consumption"},{"line_number":165,"context_line":"--------------------"},{"line_number":166,"context_line":""},{"line_number":167,"context_line":"Once all frameworks are migrated to Tempest-lib, other projects can"},{"line_number":168,"context_line":"use the same for their microversion testing."},{"line_number":169,"context_line":"Document needs to be updated how to consume the microversion testing"},{"line_number":170,"context_line":"framework with some example."}],"source_content_type":"text/x-rst","patch_set":12,"id":"da85f559_b4d05720","line":167,"updated":"2015-11-11 18:10:03.000000000","message":"So the idea is this would eventually be used in the nova functional API samples tests?  Not the actual tempest tests, but the framework for determining what microversions the tests are hitting.","commit_id":"50f1294427391b7a3d7a5021b8444cf847e19e64"},{"author":{"_account_id":5196,"name":"Matthew Treinish","email":"mtreinish@kortar.org","username":"treinish"},"change_message_id":"6b452cad688b33f838f8181cc845fcb857a01425","unresolved":false,"context_lines":[{"line_number":164,"context_line":"External consumption"},{"line_number":165,"context_line":"--------------------"},{"line_number":166,"context_line":""},{"line_number":167,"context_line":"Once all frameworks are migrated to Tempest-lib, other projects can"},{"line_number":168,"context_line":"use the same for their microversion testing."},{"line_number":169,"context_line":"Document needs to be updated how to consume the microversion testing"},{"line_number":170,"context_line":"framework with some example."}],"source_content_type":"text/x-rst","patch_set":12,"id":"da85f559_f7fb4179","line":167,"in_reply_to":"da85f559_b4d05720","updated":"2015-11-11 18:32:10.000000000","message":"I believe the idea is for external projects that use microversions to leverage this in their tempest plugins. For example, manila has microversions but no way to test them with it\u0027s tempest plugin.\n\nIf you want to leverage it in a nova functional test that works too, I just don\u0027t think that was the intent going into it.","commit_id":"50f1294427391b7a3d7a5021b8444cf847e19e64"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"7c4a707e9d886293988b2c76f6a2c7e2ab538a4d","unresolved":false,"context_lines":[{"line_number":164,"context_line":"External consumption"},{"line_number":165,"context_line":"--------------------"},{"line_number":166,"context_line":""},{"line_number":167,"context_line":"Once all frameworks are migrated to Tempest-lib, other projects can"},{"line_number":168,"context_line":"use the same for their microversion testing."},{"line_number":169,"context_line":"Document needs to be updated how to consume the microversion testing"},{"line_number":170,"context_line":"framework with some example."}],"source_content_type":"text/x-rst","patch_set":12,"id":"da85f559_6a1e84bc","line":167,"in_reply_to":"da85f559_f7fb4179","updated":"2015-11-11 18:55:23.000000000","message":"OK, so I guess tempest plugins are implied, I missed that.","commit_id":"50f1294427391b7a3d7a5021b8444cf847e19e64"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"af31cc3dab6bd5016572cc13d2f7353b68498b7e","unresolved":false,"context_lines":[{"line_number":191,"context_line":"----------"},{"line_number":192,"context_line":""},{"line_number":193,"context_line":"Target Milestone for completion:"},{"line_number":194,"context_line":"  Mitaka-1"},{"line_number":195,"context_line":""},{"line_number":196,"context_line":"Work Items"},{"line_number":197,"context_line":"----------"}],"source_content_type":"text/x-rst","patch_set":12,"id":"da85f559_54a80b8d","line":194,"updated":"2015-11-11 18:10:03.000000000","message":"This seems very aggressive unless there is already POC code.","commit_id":"50f1294427391b7a3d7a5021b8444cf847e19e64"},{"author":{"_account_id":8556,"name":"Ghanshyam Maan","display_name":"Ghanshyam Maan","email":"gmaan.os14@gmail.com","username":"ghanshyam"},"change_message_id":"1d3e40ced876000f38174984e7cbda9b122bb333","unresolved":false,"context_lines":[{"line_number":191,"context_line":"----------"},{"line_number":192,"context_line":""},{"line_number":193,"context_line":"Target Milestone for completion:"},{"line_number":194,"context_line":"  Mitaka-1"},{"line_number":195,"context_line":""},{"line_number":196,"context_line":"Work Items"},{"line_number":197,"context_line":"----------"}],"source_content_type":"text/x-rst","patch_set":12,"id":"da85f559_0e181f74","line":194,"in_reply_to":"da85f559_54a80b8d","updated":"2015-11-12 00:08:51.000000000","message":":). There are some POC/base patches out to check how this will looks like - https://review.openstack.org/#/q/status:open+project:openstack/tempest+branch:master+topic:bp/nova-microversions-tests,n,z \n\nTarget for M-1 will be to have testing framework ready and implement 1 or 2 microversion tests (for Nova, Iroinic etc).  Once those (M-1) are ready we can keep adding tests for other versions too.","commit_id":"50f1294427391b7a3d7a5021b8444cf847e19e64"},{"author":{"_account_id":10239,"name":"Dmitry Tantsur","email":"dtantsur@protonmail.com","username":"dtantsur"},"change_message_id":"94dc4d802a713e39a7a3e8e1f604463c54a2a949","unresolved":false,"context_lines":[{"line_number":6,"context_line":".."},{"line_number":7,"context_line":""},{"line_number":8,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"},{"line_number":9,"context_line":"Add microversions tests for Nova"},{"line_number":10,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"},{"line_number":11,"context_line":""},{"line_number":12,"context_line":"https://blueprints.launchpad.net/tempest/+spec/nova-microversions-tests"}],"source_content_type":"text/x-rst","patch_set":13,"id":"da85f559_751f24b7","line":9,"updated":"2015-11-12 11:16:30.000000000","message":"If you create another revision, could you please drop \"nova\" from all the names? This spec lays basis for all the versioning testing work, including ironic","commit_id":"c08a920a3a15241abb9921e708df2ed77d0bbd86"},{"author":{"_account_id":8556,"name":"Ghanshyam Maan","display_name":"Ghanshyam Maan","email":"gmaan.os14@gmail.com","username":"ghanshyam"},"change_message_id":"243ba7784921103f7e3baa83f50451b075f4dabe","unresolved":false,"context_lines":[{"line_number":6,"context_line":".."},{"line_number":7,"context_line":""},{"line_number":8,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"},{"line_number":9,"context_line":"Add microversions tests for Nova"},{"line_number":10,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"},{"line_number":11,"context_line":""},{"line_number":12,"context_line":"https://blueprints.launchpad.net/tempest/+spec/nova-microversions-tests"}],"source_content_type":"text/x-rst","patch_set":13,"id":"da85f559_33f18b96","line":9,"in_reply_to":"da85f559_228f0576","updated":"2015-11-13 00:10:13.000000000","message":"Done","commit_id":"c08a920a3a15241abb9921e708df2ed77d0bbd86"},{"author":{"_account_id":7173,"name":"Scott DAngelo","email":"scott.dangelo@gmail.com","username":"scottda"},"change_message_id":"13384ab201ca2e43182874c99f1ecd633ec65332","unresolved":false,"context_lines":[{"line_number":6,"context_line":".."},{"line_number":7,"context_line":""},{"line_number":8,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"},{"line_number":9,"context_line":"Add microversions tests for Nova"},{"line_number":10,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"},{"line_number":11,"context_line":""},{"line_number":12,"context_line":"https://blueprints.launchpad.net/tempest/+spec/nova-microversions-tests"}],"source_content_type":"text/x-rst","patch_set":13,"id":"da85f559_228f0576","line":9,"in_reply_to":"da85f559_751f24b7","updated":"2015-11-12 17:20:15.000000000","message":"Yes, where possible, making this spec more generic would be great. Cinder expects to land api-microversions in Mitaka and will need this Tempest framework as well.","commit_id":"c08a920a3a15241abb9921e708df2ed77d0bbd86"},{"author":{"_account_id":10239,"name":"Dmitry Tantsur","email":"dtantsur@protonmail.com","username":"dtantsur"},"change_message_id":"94dc4d802a713e39a7a3e8e1f604463c54a2a949","unresolved":false,"context_lines":[{"line_number":144,"context_line":""},{"line_number":145,"context_line":"Tempest-lib migration plan"},{"line_number":146,"context_line":"--------------------------"},{"line_number":147,"context_line":""},{"line_number":148,"context_line":"* Steps:"},{"line_number":149,"context_line":""},{"line_number":150,"context_line":"  #. Implement the microversion testing framework in Tempest."}],"source_content_type":"text/x-rst","patch_set":13,"id":"da85f559_b58cacda","line":147,"updated":"2015-11-12 11:16:30.000000000","message":"would be nice to explain why we can\u0027t start with tempest-lib from the beginning","commit_id":"c08a920a3a15241abb9921e708df2ed77d0bbd86"},{"author":{"_account_id":8556,"name":"Ghanshyam Maan","display_name":"Ghanshyam Maan","email":"gmaan.os14@gmail.com","username":"ghanshyam"},"change_message_id":"243ba7784921103f7e3baa83f50451b075f4dabe","unresolved":false,"context_lines":[{"line_number":144,"context_line":""},{"line_number":145,"context_line":"Tempest-lib migration plan"},{"line_number":146,"context_line":"--------------------------"},{"line_number":147,"context_line":""},{"line_number":148,"context_line":"* Steps:"},{"line_number":149,"context_line":""},{"line_number":150,"context_line":"  #. Implement the microversion testing framework in Tempest."}],"source_content_type":"text/x-rst","patch_set":13,"id":"da85f559_1399e71f","line":147,"in_reply_to":"da85f559_624e6d3c","updated":"2015-11-13 00:10:13.000000000","message":"For most of interface, Tempest implement those and tests to make sure they are much stable before going to lib side.\n\nMicroversion framework case will be tested with microversion test case on gate then we will migrate to lib. As unit tests cannot cover all scenarios and it might be risky to directly exposing interface without much stability.\n\nAnother point is that lib release takes time (gr update etc), so waiting for that to tests framework might not be good idea.","commit_id":"c08a920a3a15241abb9921e708df2ed77d0bbd86"},{"author":{"_account_id":7173,"name":"Scott DAngelo","email":"scott.dangelo@gmail.com","username":"scottda"},"change_message_id":"13384ab201ca2e43182874c99f1ecd633ec65332","unresolved":false,"context_lines":[{"line_number":144,"context_line":""},{"line_number":145,"context_line":"Tempest-lib migration plan"},{"line_number":146,"context_line":"--------------------------"},{"line_number":147,"context_line":""},{"line_number":148,"context_line":"* Steps:"},{"line_number":149,"context_line":""},{"line_number":150,"context_line":"  #. Implement the microversion testing framework in Tempest."}],"source_content_type":"text/x-rst","patch_set":13,"id":"da85f559_624e6d3c","line":147,"in_reply_to":"da85f559_b58cacda","updated":"2015-11-12 17:20:15.000000000","message":"I agree. Cinder will land microversions in Mitaka and I would like to start down the path of Tempest testing A.S.A.P.\nI would like to avoid duplicate work as much as possible.","commit_id":"c08a920a3a15241abb9921e708df2ed77d0bbd86"},{"author":{"_account_id":10239,"name":"Dmitry Tantsur","email":"dtantsur@protonmail.com","username":"dtantsur"},"change_message_id":"94dc4d802a713e39a7a3e8e1f604463c54a2a949","unresolved":false,"context_lines":[{"line_number":200,"context_line":"* Pass a test target microversion to service clients"},{"line_number":201,"context_line":"* Add a test class for a single microversion(as sample)"},{"line_number":202,"context_line":"* Migrate tested microversion testing framework to Tempest-lib"},{"line_number":203,"context_line":"* Consume those interface from Temepst-lib and remove from Tempest"},{"line_number":204,"context_line":"* Change the configrations on openstack-infra/project-config for master"},{"line_number":205,"context_line":""},{"line_number":206,"context_line":"Dependencies"}],"source_content_type":"text/x-rst","patch_set":13,"id":"da85f559_b5058c46","line":203,"updated":"2015-11-12 11:16:30.000000000","message":"typo in Tempest-lib","commit_id":"c08a920a3a15241abb9921e708df2ed77d0bbd86"},{"author":{"_account_id":8556,"name":"Ghanshyam Maan","display_name":"Ghanshyam Maan","email":"gmaan.os14@gmail.com","username":"ghanshyam"},"change_message_id":"243ba7784921103f7e3baa83f50451b075f4dabe","unresolved":false,"context_lines":[{"line_number":200,"context_line":"* Pass a test target microversion to service clients"},{"line_number":201,"context_line":"* Add a test class for a single microversion(as sample)"},{"line_number":202,"context_line":"* Migrate tested microversion testing framework to Tempest-lib"},{"line_number":203,"context_line":"* Consume those interface from Temepst-lib and remove from Tempest"},{"line_number":204,"context_line":"* Change the configrations on openstack-infra/project-config for master"},{"line_number":205,"context_line":""},{"line_number":206,"context_line":"Dependencies"}],"source_content_type":"text/x-rst","patch_set":13,"id":"da85f559_738873c7","line":203,"in_reply_to":"da85f559_b5058c46","updated":"2015-11-13 00:10:13.000000000","message":"Done","commit_id":"c08a920a3a15241abb9921e708df2ed77d0bbd86"}]}