)]}'
{"doc/source/microversion_testing.rst":[{"author":{"_account_id":8556,"name":"Ghanshyam Maan","display_name":"Ghanshyam Maan","email":"gmaan.os14@gmail.com","username":"ghanshyam"},"change_message_id":"36cfbacf41f2e56a6af53d76a0ea88cca61ba461","unresolved":false,"context_lines":[{"line_number":167,"context_line":"Some of the compute Microversion tests have been already implemented"},{"line_number":168,"context_line":"with the Microversion testing framework. So for further tests only"},{"line_number":169,"context_line":"step 4 is needed."},{"line_number":170,"context_line":""},{"line_number":171,"context_line":"Along with that JSON response schema might need versioning if needed."},{"line_number":172,"context_line":""},{"line_number":173,"context_line":"Compute service clients strictly validate the response against defined JSON"}],"source_content_type":"text/x-rst","patch_set":1,"id":"dada55a8_d369fcbf","line":170,"range":{"start_line":170,"start_character":0,"end_line":170,"end_character":0},"updated":"2016-07-26 05:25:52.000000000","message":"As we concluded about microversion tests scope in Tempest as in ML- http://lists.openstack.org/pipermail/openstack-dev/2016-July/099860.html,\nwe can explicitly mentioned about that.\n\nmay be note with-\n- Tempest tests not needed for microversion which changing only API/DB layer and no change in response schema.\n Those can be covered in Nova only.\n- All other kind of microversion changes are good to test via Tempest tests.","commit_id":"d5129ac1a9db34d2b007438dc66224a06e220741"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"9972942e4b4840b7d6197716d9719ed5cbd5c3fc","unresolved":false,"context_lines":[{"line_number":167,"context_line":"Some of the compute Microversion tests have been already implemented"},{"line_number":168,"context_line":"with the Microversion testing framework. So for further tests only"},{"line_number":169,"context_line":"step 4 is needed."},{"line_number":170,"context_line":""},{"line_number":171,"context_line":"Along with that JSON response schema might need versioning if needed."},{"line_number":172,"context_line":""},{"line_number":173,"context_line":"Compute service clients strictly validate the response against defined JSON"}],"source_content_type":"text/x-rst","patch_set":1,"id":"dada55a8_d50548c2","line":170,"range":{"start_line":170,"start_character":0,"end_line":170,"end_character":0},"in_reply_to":"dada55a8_d369fcbf","updated":"2016-07-26 13:54:09.000000000","message":"That\u0027s what \"All response schema changes should be tested even if they are added\n   to just fill a gap in order to test the latest microversion\" below is for.","commit_id":"d5129ac1a9db34d2b007438dc66224a06e220741"},{"author":{"_account_id":8556,"name":"Ghanshyam Maan","display_name":"Ghanshyam Maan","email":"gmaan.os14@gmail.com","username":"ghanshyam"},"change_message_id":"282b9840b26d207971d05ea039c5e5bf1f3d7ca9","unresolved":false,"context_lines":[{"line_number":167,"context_line":"Some of the compute Microversion tests have been already implemented"},{"line_number":168,"context_line":"with the Microversion testing framework. So for further tests only"},{"line_number":169,"context_line":"step 4 is needed."},{"line_number":170,"context_line":""},{"line_number":171,"context_line":"Along with that JSON response schema might need versioning if needed."},{"line_number":172,"context_line":""},{"line_number":173,"context_line":"Compute service clients strictly validate the response against defined JSON"}],"source_content_type":"text/x-rst","patch_set":1,"id":"dada55a8_fbd02055","line":170,"range":{"start_line":170,"start_character":0,"end_line":170,"end_character":0},"in_reply_to":"dada55a8_d50548c2","updated":"2016-07-27 00:34:26.000000000","message":"Yea i agree, but i saw many review comments about what all microversion tests are allowed or not.\n\nSo it will be better and clear if we mention about what type of tests are not allowed.\nWhich will avoid same discussion on reviews in future.","commit_id":"d5129ac1a9db34d2b007438dc66224a06e220741"},{"author":{"_account_id":7350,"name":"Jordan Pittier","email":"jordan.pittier@gmail.com","username":"JordanP"},"change_message_id":"9ede4d33267bb46ab8c25d04325bd9710bd95fb0","unresolved":false,"context_lines":[{"line_number":185,"context_line":".. note:: All response schema changes should be tested even if they are added"},{"line_number":186,"context_line":"   to just fill a gap in order to test the latest microversion. Furthermore, to"},{"line_number":187,"context_line":"   avoid gaps in microversion schema response validation coverage, missing"},{"line_number":188,"context_line":"   schema changes should be added after feature freeze each release."},{"line_number":189,"context_line":""},{"line_number":190,"context_line":"For Example:"},{"line_number":191,"context_line":"    Implementing the v2.20 Microversion tests before v2.9 and 2.19-"}],"source_content_type":"text/x-rst","patch_set":1,"id":"dada55a8_67efa187","line":188,"updated":"2016-07-26 09:34:32.000000000","message":"A word seems to be missing. \"feature freeze *of* each release\" maybe.","commit_id":"d5129ac1a9db34d2b007438dc66224a06e220741"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"9972942e4b4840b7d6197716d9719ed5cbd5c3fc","unresolved":false,"context_lines":[{"line_number":185,"context_line":".. note:: All response schema changes should be tested even if they are added"},{"line_number":186,"context_line":"   to just fill a gap in order to test the latest microversion. Furthermore, to"},{"line_number":187,"context_line":"   avoid gaps in microversion schema response validation coverage, missing"},{"line_number":188,"context_line":"   schema changes should be added after feature freeze each release."},{"line_number":189,"context_line":""},{"line_number":190,"context_line":"For Example:"},{"line_number":191,"context_line":"    Implementing the v2.20 Microversion tests before v2.9 and 2.19-"}],"source_content_type":"text/x-rst","patch_set":1,"id":"dada55a8_75fd7cb0","line":188,"in_reply_to":"dada55a8_67efa187","updated":"2016-07-26 13:54:09.000000000","message":"I can add that.","commit_id":"d5129ac1a9db34d2b007438dc66224a06e220741"},{"author":{"_account_id":5196,"name":"Matthew Treinish","email":"mtreinish@kortar.org","username":"treinish"},"change_message_id":"b352e63cb43eb6500f8490807f8719bfac389d45","unresolved":false,"context_lines":[{"line_number":185,"context_line":".. note:: All response schema changes should be tested even if they are added"},{"line_number":186,"context_line":"   to just fill a gap in order to test the latest microversion. Furthermore, to"},{"line_number":187,"context_line":"   avoid gaps in microversion schema response validation coverage, missing"},{"line_number":188,"context_line":"   schema changes should be added after feature freeze of each release."},{"line_number":189,"context_line":""},{"line_number":190,"context_line":"For Example:"},{"line_number":191,"context_line":"    Implementing the v2.20 Microversion tests before v2.9 and 2.19-"}],"source_content_type":"text/x-rst","patch_set":2,"id":"dada55a8_6fe58e07","line":188,"updated":"2016-07-26 22:28:17.000000000","message":"I think there is some confusion here from some people, at least based on what I\u0027ve seen in a couple of reviews. We\u0027re saying that test adds for things that are normally not allowed are ok to fill a schema gap at the end of the cycle. But during the normal case we rely on tests for microversions to backfill the gaps as things get added during the middle of the cycle. The way this is worded makes it sound like we\u0027ll accept any patch that adds a test for a missing microversion schema.\n\nI think we should add a sentence saying something like that just because a microversion schema change was made doesn\u0027t necessarily justify a new test being added. (I\u0027m thinking of the 2.9, or whatever it was, microversion test add review)","commit_id":"edf6dd80f5593f76c49d6af2a26c0a5500ec6931"},{"author":{"_account_id":6167,"name":"Ken\u0027ichi Ohmichi","email":"ken1ohmichi@gmail.com","username":"oomichi"},"change_message_id":"594c98e2c1b9561c404f72ae74e2ba439b600e5e","unresolved":false,"context_lines":[{"line_number":185,"context_line":".. note:: All response schema changes should be tested even if they are added"},{"line_number":186,"context_line":"   to just fill a gap in order to test the latest microversion. Furthermore, to"},{"line_number":187,"context_line":"   avoid gaps in microversion schema response validation coverage, missing"},{"line_number":188,"context_line":"   schema changes should be added after feature freeze of each release."},{"line_number":189,"context_line":""},{"line_number":190,"context_line":"For Example:"},{"line_number":191,"context_line":"    Implementing the v2.20 Microversion tests before v2.9 and 2.19-"}],"source_content_type":"text/x-rst","patch_set":2,"id":"bacf61ea_7e9727ec","line":188,"range":{"start_line":188,"start_character":34,"end_line":188,"end_character":71},"updated":"2016-07-30 00:58:45.000000000","message":"Why should we wait for the feature freeze to implement the schemas? I could not find any merit for doing that.","commit_id":"edf6dd80f5593f76c49d6af2a26c0a5500ec6931"},{"author":{"_account_id":5689,"name":"Masayuki Igawa","email":"masayuki@igawa.io","username":"igawa"},"change_message_id":"481f0f5fb6dabe47dd940fa60e8f6b9aaf93dff3","unresolved":false,"context_lines":[{"line_number":185,"context_line":".. note:: All response schema changes should be tested even if they are added"},{"line_number":186,"context_line":"   to just fill a gap in order to test the latest microversion. Furthermore, to"},{"line_number":187,"context_line":"   avoid gaps in microversion schema response validation coverage, missing"},{"line_number":188,"context_line":"   schema changes should be added after feature freeze of each release."},{"line_number":189,"context_line":""},{"line_number":190,"context_line":"For Example:"},{"line_number":191,"context_line":"    Implementing the v2.20 Microversion tests before v2.9 and 2.19-"}],"source_content_type":"text/x-rst","patch_set":2,"id":"bacf61ea_fa1823d5","line":188,"range":{"start_line":188,"start_character":34,"end_line":188,"end_character":71},"in_reply_to":"bacf61ea_60500bd5","updated":"2016-08-01 09:13:18.000000000","message":"I think we need to add tests for each schema if we take the separated schema way as Ghanshyam saying.\n\nIn a different view point, how about adding the link to the Nova code review and microversion doc?[1][2] (It was already surprisingly merged, though.)\nI feel it\u0027s good docs to understand the context.\n\n[1] http://docs.openstack.org/developer/nova/code-review.html#microversion-api\n[2] http://docs.openstack.org/developer/nova/api_microversion_dev.html#other-necessary-changes","commit_id":"edf6dd80f5593f76c49d6af2a26c0a5500ec6931"},{"author":{"_account_id":8556,"name":"Ghanshyam Maan","display_name":"Ghanshyam Maan","email":"gmaan.os14@gmail.com","username":"ghanshyam"},"change_message_id":"d09531fbf34a37c497594b56d89f85ed6a147047","unresolved":false,"context_lines":[{"line_number":185,"context_line":".. note:: All response schema changes should be tested even if they are added"},{"line_number":186,"context_line":"   to just fill a gap in order to test the latest microversion. Furthermore, to"},{"line_number":187,"context_line":"   avoid gaps in microversion schema response validation coverage, missing"},{"line_number":188,"context_line":"   schema changes should be added after feature freeze of each release."},{"line_number":189,"context_line":""},{"line_number":190,"context_line":"For Example:"},{"line_number":191,"context_line":"    Implementing the v2.20 Microversion tests before v2.9 and 2.19-"}],"source_content_type":"text/x-rst","patch_set":2,"id":"bacf61ea_60500bd5","line":188,"range":{"start_line":188,"start_character":34,"end_line":188,"end_character":71},"in_reply_to":"bacf61ea_7e9727ec","updated":"2016-08-01 01:42:00.000000000","message":"Another point to consider here is this - https://review.openstack.org/#/c/306245.\n\nIf we separate the each schema then lower version schema cannot be covered under higher microversion tests (because we do not use deepcopy and each schema will be independent in seprate file).\nIn that case we should add tests for each schema to test those on gate.","commit_id":"edf6dd80f5593f76c49d6af2a26c0a5500ec6931"},{"author":{"_account_id":8556,"name":"Ghanshyam Maan","display_name":"Ghanshyam Maan","email":"gmaan.os14@gmail.com","username":"ghanshyam"},"change_message_id":"e08706e9d9905c38b036cd6fcd6a796c707bb667","unresolved":false,"context_lines":[{"line_number":185,"context_line":".. note:: All response schema changes should be tested even if they are added"},{"line_number":186,"context_line":"   to just fill a gap in order to test the latest microversion. Furthermore, to"},{"line_number":187,"context_line":"   avoid gaps in microversion schema response validation coverage, missing"},{"line_number":188,"context_line":"   schema changes should be added after feature freeze of each release."},{"line_number":189,"context_line":""},{"line_number":190,"context_line":"For Example:"},{"line_number":191,"context_line":"    Implementing the v2.20 Microversion tests before v2.9 and 2.19-"}],"source_content_type":"text/x-rst","patch_set":2,"id":"bacf61ea_bda43eb9","line":188,"in_reply_to":"dada55a8_0064a11f","updated":"2016-07-28 23:55:01.000000000","message":"We discussed this in yesterday qa meeting and i am on side of adding the tests for microversion changing schema at the same time with nova changes. Not wait till the end.\n\nThis will give another merit to tests each schema as independent instead of testing in a bunch with some higher microversion tests. \nExample, 2.3 schema added with 2.25 tests and we say no 2.3 tests as schema is covered. \nBut 2.3 schema alone is not tests means we do not tests that anything more is returned in 2.3 than expected.","commit_id":"edf6dd80f5593f76c49d6af2a26c0a5500ec6931"},{"author":{"_account_id":8556,"name":"Ghanshyam Maan","display_name":"Ghanshyam Maan","email":"gmaan.os14@gmail.com","username":"ghanshyam"},"change_message_id":"64d6d90beb0604c3d0f7612dc63dd4d2d2510459","unresolved":false,"context_lines":[{"line_number":185,"context_line":".. note:: All response schema changes should be tested even if they are added"},{"line_number":186,"context_line":"   to just fill a gap in order to test the latest microversion. Furthermore, to"},{"line_number":187,"context_line":"   avoid gaps in microversion schema response validation coverage, missing"},{"line_number":188,"context_line":"   schema changes should be added after feature freeze of each release."},{"line_number":189,"context_line":""},{"line_number":190,"context_line":"For Example:"},{"line_number":191,"context_line":"    Implementing the v2.20 Microversion tests before v2.9 and 2.19-"}],"source_content_type":"text/x-rst","patch_set":2,"id":"dada55a8_0064a11f","line":188,"in_reply_to":"dada55a8_6fe58e07","updated":"2016-07-27 00:34:46.000000000","message":"Yea, I am ok on both way, \n1. to filling the gap at end of cycle(as mentioned in ML) and let valid microversion tests fill the same as much as possible during the cycle. \n\n2. + allow microversion tests which just tests schema changes as compare to Nova tests.\n\nFrom Nova doc patch also it seems, every schema changes must have tempest tests.\n- https://review.openstack.org/#/c/346080/2/doc/source/code-review.rst\n\nSo i was taking the same on tempest side too but anyways both side doc should be consistent with either approach.","commit_id":"edf6dd80f5593f76c49d6af2a26c0a5500ec6931"}]}