)]}'
{"specs/newton/approved/associate-test-result-to-product.rst":[{"author":{"_account_id":9312,"name":"Alexandre Levine","email":"alexandrelevine@gmail.com","username":"alevine"},"change_message_id":"a11b4c182b6e1512b263e930e1ba30b842ce87d4","unresolved":false,"context_lines":[{"line_number":25,"context_line":""},{"line_number":26,"context_line":"* Add two new columns to the \"test\" table in the database:"},{"line_number":27,"context_line":""},{"line_number":28,"context_line":"  * **product_uuid**:  this field stores the product UUID that the test run is"},{"line_number":29,"context_line":"    associated to."},{"line_number":30,"context_line":""},{"line_number":31,"context_line":"  * **certification**: this is a boolean that indicates whether the test record"}],"source_content_type":"text/x-rst","patch_set":1,"id":"3aaa91ec_1ff0fe8f","line":28,"range":{"start_line":28,"start_character":4,"end_line":28,"end_character":20},"updated":"2016-06-22 13:20:12.000000000","message":"I don\u0027t mind having PRODUCT_ID here BUT:\n1. It should be named the same way we name it in the PRODUCT table, i.e. PRODUCT_ID or PRODUCT_REF_ID if we rename it this way.\n2. It should contain the same value held in the PRODUCT table for this particular cloud (product) and the same one reported by the tests, i.e. cpid or cloudid for any given cloud. So for clouds it\u0027ll alway be equal to cpid which is already in metadata as I understand.","commit_id":"1edc4427e8b41709c55b2b76b946ab460ce73992"},{"author":{"_account_id":9312,"name":"Alexandre Levine","email":"alexandrelevine@gmail.com","username":"alevine"},"change_message_id":"46fd20429d896a82ad13c249c9be7f1722c258e8","unresolved":false,"context_lines":[{"line_number":25,"context_line":""},{"line_number":26,"context_line":"* Add two new columns to the \"test\" table in the database:"},{"line_number":27,"context_line":""},{"line_number":28,"context_line":"  * **product_uuid**:  this field stores the product UUID that the test run is"},{"line_number":29,"context_line":"    associated to."},{"line_number":30,"context_line":""},{"line_number":31,"context_line":"  * **certification**: this is a boolean that indicates whether the test record"}],"source_content_type":"text/x-rst","patch_set":1,"id":"1aa78d24_6a1eee0c","line":28,"range":{"start_line":28,"start_character":4,"end_line":28,"end_character":20},"in_reply_to":"1aa78d24_144dd199","updated":"2016-07-07 15:46:43.000000000","message":"I don\u0027t quite understand that statement. I mean the meaning of it. Definitely each user can have many products.  You most probably should meant it in relation to some of my comments above but I don\u0027t quite understand to which.","commit_id":"1edc4427e8b41709c55b2b76b946ab460ce73992"},{"author":{"_account_id":9312,"name":"Alexandre Levine","email":"alexandrelevine@gmail.com","username":"alevine"},"change_message_id":"46fd20429d896a82ad13c249c9be7f1722c258e8","unresolved":false,"context_lines":[{"line_number":25,"context_line":""},{"line_number":26,"context_line":"* Add two new columns to the \"test\" table in the database:"},{"line_number":27,"context_line":""},{"line_number":28,"context_line":"  * **product_uuid**:  this field stores the product UUID that the test run is"},{"line_number":29,"context_line":"    associated to."},{"line_number":30,"context_line":""},{"line_number":31,"context_line":"  * **certification**: this is a boolean that indicates whether the test record"}],"source_content_type":"text/x-rst","patch_set":1,"id":"1aa78d24_ea949e37","line":28,"range":{"start_line":28,"start_character":4,"end_line":28,"end_character":20},"in_reply_to":"1aa78d24_14667129","updated":"2016-07-07 15:46:43.000000000","message":"What do you mean?\nIn the design I presumed that each cloud absolutely must to be identified by some automatically determinable CloudID like a mac-address of a computer.\nWithout it how can we ensure that we speak about the same Cloud if a CloudID is volatile?\nWhat are the scenarios for such instability?","commit_id":"1edc4427e8b41709c55b2b76b946ab460ce73992"},{"author":{"_account_id":7822,"name":"Chris Hoge (not involved in upstream development)","email":"chris@hogepodge.com","username":"hogepodge"},"change_message_id":"c473bcb140bfa5b0604751eec3ca1a31c5ab4366","unresolved":false,"context_lines":[{"line_number":25,"context_line":""},{"line_number":26,"context_line":"* Add two new columns to the \"test\" table in the database:"},{"line_number":27,"context_line":""},{"line_number":28,"context_line":"  * **product_uuid**:  this field stores the product UUID that the test run is"},{"line_number":29,"context_line":"    associated to."},{"line_number":30,"context_line":""},{"line_number":31,"context_line":"  * **certification**: this is a boolean that indicates whether the test record"}],"source_content_type":"text/x-rst","patch_set":1,"id":"1aa78d24_8999a77b","line":28,"range":{"start_line":28,"start_character":4,"end_line":28,"end_character":20},"in_reply_to":"1aa78d24_45404507","updated":"2016-07-07 16:01:28.000000000","message":"It\u0027s absolutely dependent on trust. It was designed to be that way. It is also designed to allow users to verify. That is why DefCore doesn\u0027t require multiple accounts, or admin accounts. In fact, I would like to add an option to voluntarily upload tempest subunit data.\n\nWith regards to 3, depending on how cpid is generated it\u0027s likely that it is not stable. I outlined some real world cases, so I\u0027m confused as to why you refuse to accept them. The internet is built off of abstracting away underlying components, for example with load balancers. Some clouds use test beds that are separately run, but otherwise identical to production clouds. By definition, distributions will have a multitude of deployments, each with a different CPID.\n\nPlus, we have absolutely no way of enforcing CPID stability.","commit_id":"1edc4427e8b41709c55b2b76b946ab460ce73992"},{"author":{"_account_id":9312,"name":"Alexandre Levine","email":"alexandrelevine@gmail.com","username":"alevine"},"change_message_id":"46fd20429d896a82ad13c249c9be7f1722c258e8","unresolved":false,"context_lines":[{"line_number":25,"context_line":""},{"line_number":26,"context_line":"* Add two new columns to the \"test\" table in the database:"},{"line_number":27,"context_line":""},{"line_number":28,"context_line":"  * **product_uuid**:  this field stores the product UUID that the test run is"},{"line_number":29,"context_line":"    associated to."},{"line_number":30,"context_line":""},{"line_number":31,"context_line":"  * **certification**: this is a boolean that indicates whether the test record"}],"source_content_type":"text/x-rst","patch_set":1,"id":"1aa78d24_45404507","line":28,"range":{"start_line":28,"start_character":4,"end_line":28,"end_character":20},"in_reply_to":"1aa78d24_6f7a8025","updated":"2016-07-07 15:46:43.000000000","message":"1. What is ownership then? Association of results to some VendorID?\nWe\u0027ve got a gap in use-cases and scenarios. Right now it is possible for different Vendors (Vendor admins) to register products for the same cloud (means with the same CloudID). In case a User is an admin in both and uploads test results for the Cloud, what ownership in terms of Vendor do we have? Both Vendors seem to have claims to this tests.\n\n2. See above. I presumed those would be different clouds because otherwise we cannot ensure validity and relation of test results. The whole system becomes very \"advisory\" and depending on trust. It is not to use this kind of mechanisms for official certification purposes. Do we really want to keep it this way?\n\n3. I don\u0027t quite understand this answer. Would be great to see the offline answers and to understand your real-life scenarios now and in future (I presume we\u0027d want to improve some).","commit_id":"1edc4427e8b41709c55b2b76b946ab460ce73992"},{"author":{"_account_id":9312,"name":"Alexandre Levine","email":"alexandrelevine@gmail.com","username":"alevine"},"change_message_id":"678d74a490881ecca43f16ac4379a25ea19952e5","unresolved":false,"context_lines":[{"line_number":25,"context_line":""},{"line_number":26,"context_line":"* Add two new columns to the \"test\" table in the database:"},{"line_number":27,"context_line":""},{"line_number":28,"context_line":"  * **product_uuid**:  this field stores the product UUID that the test run is"},{"line_number":29,"context_line":"    associated to."},{"line_number":30,"context_line":""},{"line_number":31,"context_line":"  * **certification**: this is a boolean that indicates whether the test record"}],"source_content_type":"text/x-rst","patch_set":1,"id":"1aa78d24_897b67bd","line":28,"range":{"start_line":28,"start_character":4,"end_line":28,"end_character":20},"in_reply_to":"1aa78d24_8999a77b","updated":"2016-07-07 16:06:56.000000000","message":"Ok, then the design doc is incorrect in this relation and need to be changed.\nFor that I need to understand the scenarios of handling results and Clouds. Because without automatic ways of doing things everything should be done \"somehow manually\" - association of results, clouds, etc. It needs to be defined and thought over again.\nWhen I initially worked on the doc I couldn\u0027t find any of place with such complete set of scenarios so I created the list, supposing that it made sense.\nApparently I was wrong in some of my presumptions. Let\u0027s change the use-case list and make it complete and reflecting this \"other\" real situation.","commit_id":"1edc4427e8b41709c55b2b76b946ab460ce73992"},{"author":{"_account_id":10526,"name":"Catherine Diep","email":"cdiep@us.ibm.com","username":"cdiep"},"change_message_id":"9452e3d5cb3cecc271c76058a8fbc29e27ec3251","unresolved":false,"context_lines":[{"line_number":25,"context_line":""},{"line_number":26,"context_line":"* Add two new columns to the \"test\" table in the database:"},{"line_number":27,"context_line":""},{"line_number":28,"context_line":"  * **product_uuid**:  this field stores the product UUID that the test run is"},{"line_number":29,"context_line":"    associated to."},{"line_number":30,"context_line":""},{"line_number":31,"context_line":"  * **certification**: this is a boolean that indicates whether the test record"}],"source_content_type":"text/x-rst","patch_set":1,"id":"3aaa91ec_ee7a1b23","line":28,"range":{"start_line":28,"start_character":4,"end_line":28,"end_character":20},"in_reply_to":"3aaa91ec_158068bf","updated":"2016-06-24 18:20:17.000000000","message":"1) Creator, i.e. User which did the uploading of the results never changes.\n[Comment] Correct. But if the creator associates the test results to a product of a vendor. The ownership of the test runs now transfers to the vendor.\n\n2) The use-case of association (8) shouldn\u0027t exist as it\u0027s written in the doc - CloudID for the test run result unambiguously defines which Cloud this test is applicable to. So in fact it is applicable to the cloud with CloudID, i.e. for any record in Product table with the PRODUCT_REF_ID\u003d\u003dCloudID. \n[Comment] Use case (8) describes the association of test results to clouds and not to products.  The different is, for a vendor, each product should be unique and can be associated to many clouds.  Furthermore, I don’t think CloudIDs (in this case PRODUCT_ID, PRODUCT_REF_ID) are unambiguously unique for public clouds. Please review my CloudID (PRODUCT_ID, PRODUCT_REF_ID) uniqueness concerns listed in the alternative section in this document.\n\n3) The test is being run for the Cloud (CloudID) by a User (UserID) and that\u0027s all the facts we need to record \"forever\" to be ok with any further handling. \n[Comment] This is true if the user does not associate the results to a vendor. Once the creator associated the results to a vendor, the creator forfeits the privilege to the test records unless he/she is an admin of the vendor.","commit_id":"1edc4427e8b41709c55b2b76b946ab460ce73992"},{"author":{"_account_id":10526,"name":"Catherine Diep","email":"cdiep@us.ibm.com","username":"cdiep"},"change_message_id":"d4416c74809881475354edd891ff96719d91baf5","unresolved":false,"context_lines":[{"line_number":25,"context_line":""},{"line_number":26,"context_line":"* Add two new columns to the \"test\" table in the database:"},{"line_number":27,"context_line":""},{"line_number":28,"context_line":"  * **product_uuid**:  this field stores the product UUID that the test run is"},{"line_number":29,"context_line":"    associated to."},{"line_number":30,"context_line":""},{"line_number":31,"context_line":"  * **certification**: this is a boolean that indicates whether the test record"}],"source_content_type":"text/x-rst","patch_set":1,"id":"3aaa91ec_f57c1fed","line":28,"range":{"start_line":28,"start_character":4,"end_line":28,"end_character":20},"in_reply_to":"3aaa91ec_1ff0fe8f","updated":"2016-06-22 19:12:57.000000000","message":"Alex, Thanks for your review!\n\nThis is exactly the reason why I would like us to rename the PRODUCT_ID to a new name (PRODUCT_REF_ID) in the \"product\" table.  \n\nAs you have indicated, for clouds, the PRODUCT_ID (soon to be PRODUCT_REF_ID) should be the same as the CPID field in the \"test\" table.  So there is really no need to add a field to store the duplicated value.\n\nThe PRODUCT_UUID that I suggest to add here is to store the \"ID field of the PRODUCT table\" (not the PRODUCT_ID field).  The reason why I think we would use the ID field and not the PRODUCT_ID (PRODUCT_REF_ID) is described in the Alternatives session of this document. Could you please take a look!","commit_id":"1edc4427e8b41709c55b2b76b946ab460ce73992"},{"author":{"_account_id":7822,"name":"Chris Hoge (not involved in upstream development)","email":"chris@hogepodge.com","username":"hogepodge"},"change_message_id":"0c277b228b2a69f4614e92a9ef391894045afb92","unresolved":false,"context_lines":[{"line_number":25,"context_line":""},{"line_number":26,"context_line":"* Add two new columns to the \"test\" table in the database:"},{"line_number":27,"context_line":""},{"line_number":28,"context_line":"  * **product_uuid**:  this field stores the product UUID that the test run is"},{"line_number":29,"context_line":"    associated to."},{"line_number":30,"context_line":""},{"line_number":31,"context_line":"  * **certification**: this is a boolean that indicates whether the test record"}],"source_content_type":"text/x-rst","patch_set":1,"id":"1aa78d24_14667129","line":28,"range":{"start_line":28,"start_character":4,"end_line":28,"end_character":20},"in_reply_to":"3aaa91ec_1ff0fe8f","updated":"2016-07-07 15:27:04.000000000","message":"CPID is not guaranteed to be stable.","commit_id":"1edc4427e8b41709c55b2b76b946ab460ce73992"},{"author":{"_account_id":7822,"name":"Chris Hoge (not involved in upstream development)","email":"chris@hogepodge.com","username":"hogepodge"},"change_message_id":"0c277b228b2a69f4614e92a9ef391894045afb92","unresolved":false,"context_lines":[{"line_number":25,"context_line":""},{"line_number":26,"context_line":"* Add two new columns to the \"test\" table in the database:"},{"line_number":27,"context_line":""},{"line_number":28,"context_line":"  * **product_uuid**:  this field stores the product UUID that the test run is"},{"line_number":29,"context_line":"    associated to."},{"line_number":30,"context_line":""},{"line_number":31,"context_line":"  * **certification**: this is a boolean that indicates whether the test record"}],"source_content_type":"text/x-rst","patch_set":1,"id":"1aa78d24_144dd199","line":28,"range":{"start_line":28,"start_character":4,"end_line":28,"end_character":20},"in_reply_to":"3aaa91ec_2493f150","updated":"2016-07-07 15:27:04.000000000","message":"Users and products are not 1:1","commit_id":"1edc4427e8b41709c55b2b76b946ab460ce73992"},{"author":{"_account_id":10526,"name":"Catherine Diep","email":"cdiep@us.ibm.com","username":"cdiep"},"change_message_id":"0fb0c261b2a389bc24df8de22b5a384304a8d6c6","unresolved":false,"context_lines":[{"line_number":25,"context_line":""},{"line_number":26,"context_line":"* Add two new columns to the \"test\" table in the database:"},{"line_number":27,"context_line":""},{"line_number":28,"context_line":"  * **product_uuid**:  this field stores the product UUID that the test run is"},{"line_number":29,"context_line":"    associated to."},{"line_number":30,"context_line":""},{"line_number":31,"context_line":"  * **certification**: this is a boolean that indicates whether the test record"}],"source_content_type":"text/x-rst","patch_set":1,"id":"3aaa91ec_f1e6b013","line":28,"range":{"start_line":28,"start_character":4,"end_line":28,"end_character":20},"in_reply_to":"3aaa91ec_2493f150","updated":"2016-06-23 23:02:35.000000000","message":"We can’t use USER_ID to associate data to a product.  Because the user would lose his/her ownership to the data once the data is associate to a product. Think about the following situation:\n\n1) John is an admin of \"Vendor A\" which has \"Product A\".\n2) John uploads data with signature to RefStack.  At this point, John is the sole owner of the data.\n3) John being an owner of the data and an admin of the vendor owning \"Product A\", John can associate the data to \"Product A\" by initiating to update \"PRODUCT_UUID\" field with the ID of \"Product A\".\n4) The Data is now owned by the Vendor, any admins (not only John) of this vendor can now manage the data.\n5) John quits \"Vendor A\" and now works for \"Vendor B\".  John\u0027s user ID should be removed from the admin group of \"Vendor A\" and subsequently John will lose the privilege to manage the data.","commit_id":"1edc4427e8b41709c55b2b76b946ab460ce73992"},{"author":{"_account_id":9312,"name":"Alexandre Levine","email":"alexandrelevine@gmail.com","username":"alevine"},"change_message_id":"e30ddc60845cc5405d1857dbbeebb64821d7f47c","unresolved":false,"context_lines":[{"line_number":25,"context_line":""},{"line_number":26,"context_line":"* Add two new columns to the \"test\" table in the database:"},{"line_number":27,"context_line":""},{"line_number":28,"context_line":"  * **product_uuid**:  this field stores the product UUID that the test run is"},{"line_number":29,"context_line":"    associated to."},{"line_number":30,"context_line":""},{"line_number":31,"context_line":"  * **certification**: this is a boolean that indicates whether the test record"}],"source_content_type":"text/x-rst","patch_set":1,"id":"3aaa91ec_cd73f04c","line":28,"range":{"start_line":28,"start_character":4,"end_line":28,"end_character":20},"in_reply_to":"3aaa91ec_694c8942","updated":"2016-06-30 11:53:14.000000000","message":"4.a) That doesn\u0027t answer the question\n4.b) Let\u0027s not use the terminology which is not defined. As I said - I don\u0027t understand what \"transfer of ownership\" is. It can be defined if motivated by some business use-case which should be clarified first and yesterday at the meeting we almost clarified a couple of existing business use-cases but they do not require ownership transfer, one of them required setting up Foundation as owner for anonymous test results - that\u0027s what I heard.\n4.c) As are the other questions (maybe not as perfect)\n4.d) That doesn\u0027t answer the question. And a Product is an instance of a Cloud. Cloud is a real thing in the real world defined by its CloudID, and Product is its replica per user in our Product table where PRODUCT_REF_ID reflects CloudID and PRODUCT_UUID (or PRODUCT_ID) is the replica\u0027s ID.\n4.e) Then there is no need for association mechanism at all because Vendor has already access and is associated with test results if one of its admins uploaded them and there would be no other way to associate non-vendor-admin user\u0027s results for the same cloud with this Vendor. At least that comes from what you\u0027re saying.\n4.f) That\u0027s doesn\u0027t answer the question. The same as in 4.e - if only Vendor admins\u0027 uploaded results can be used for the certification of Vendor\u0027s  cloud then there is no need in association of any kind.\n\n6) Sorry I was unaware of that during last Summit and couldn\u0027t help.","commit_id":"1edc4427e8b41709c55b2b76b946ab460ce73992"},{"author":{"_account_id":10526,"name":"Catherine Diep","email":"cdiep@us.ibm.com","username":"cdiep"},"change_message_id":"1df0da204e1f45aee5dc150ec2670ffefcc4ed02","unresolved":false,"context_lines":[{"line_number":25,"context_line":""},{"line_number":26,"context_line":"* Add two new columns to the \"test\" table in the database:"},{"line_number":27,"context_line":""},{"line_number":28,"context_line":"  * **product_uuid**:  this field stores the product UUID that the test run is"},{"line_number":29,"context_line":"    associated to."},{"line_number":30,"context_line":""},{"line_number":31,"context_line":"  * **certification**: this is a boolean that indicates whether the test record"}],"source_content_type":"text/x-rst","patch_set":1,"id":"3aaa91ec_694c8942","line":28,"range":{"start_line":28,"start_character":4,"end_line":28,"end_character":20},"in_reply_to":"3aaa91ec_92593138","updated":"2016-06-29 23:21:54.000000000","message":"Again, Chris Hoge (OpenStack Foundation ) should be the right person for questions related to OpenStack certification.  Following answers are my personal opinions.\n\n4.a) OpenStack Logo is required for commercial use only.  If a cloud is built based on a certified product, and that cloud will be used for commercial purposes (like a public cloud), the cloud needs to be re-certified.  On the other hand, if the cloud is used as a private or on premise cloud for a customer, I do not think the cloud needs to be certified.\n\n4.b) To protect ownership of the data creator, only the creator of the data and being an admin of a vendor can initiate the transferring of data ownership to that vendor.  In the use case where some consulting company doing testing and validation of all known public clouds and wants to retain ownership of the data, the public clouds owners cannot use these results for certification.  However, nothing would stop the vendors to use these data as independent data for marketing purposes (by providing the links to the publicly viewable data ).\n \n4.c) This is a perfect question for the Foundation regarding the certification.  From RefStack point of view, RefStack provides data to help customers to make impactful decisions against the clouds. \n\n4.d) See answers in question 4.a) and 4.b).  A cloud is an instance of a product.  The relationship can be one product to one cloud or one product to many clouds.\n\n4.e) See answers in question 4.a) and 4.b).  User cannot associate his results to a vendor unless he is one of the admins of the vendor.\n\n4.f) Vendor is the one who adds a user to its admin list.  The vendor should know the identity of the users.  Typically, vendor admins should be someone employed by the vendor.\n\n5) I guess I will have to wait for more questions.\n\n6) These features were actually targeted for the last (Mitaka) cycle.  I would like to have them implemented in the Newton cycle.","commit_id":"1edc4427e8b41709c55b2b76b946ab460ce73992"},{"author":{"_account_id":7822,"name":"Chris Hoge (not involved in upstream development)","email":"chris@hogepodge.com","username":"hogepodge"},"change_message_id":"0c277b228b2a69f4614e92a9ef391894045afb92","unresolved":false,"context_lines":[{"line_number":25,"context_line":""},{"line_number":26,"context_line":"* Add two new columns to the \"test\" table in the database:"},{"line_number":27,"context_line":""},{"line_number":28,"context_line":"  * **product_uuid**:  this field stores the product UUID that the test run is"},{"line_number":29,"context_line":"    associated to."},{"line_number":30,"context_line":""},{"line_number":31,"context_line":"  * **certification**: this is a boolean that indicates whether the test record"}],"source_content_type":"text/x-rst","patch_set":1,"id":"1aa78d24_6f7a8025","line":28,"range":{"start_line":28,"start_character":4,"end_line":28,"end_character":20},"in_reply_to":"3aaa91ec_92593138","updated":"2016-07-07 15:27:04.000000000","message":"There are many reasons to transfer ownership. Results used by OSF are just one reason. Employees leaving a company are another. Associating anonymous results is a third.\n\nCloud IDs are not guaranteed to be stable. Upgrades, database changes, use of temporary test clouds, are all things that can disrupt table CPID. One cloud can have many CPIDs.\n\nAll of your questions for 4 are cases handled administratively by the OSF in implementation of the Powered program. I can take answers to those offline, but they don\u0027t impact this design process. At a high level, much of the program is trust based and special cases (like c and d) are handled on a case-by-case basis.","commit_id":"1edc4427e8b41709c55b2b76b946ab460ce73992"},{"author":{"_account_id":9312,"name":"Alexandre Levine","email":"alexandrelevine@gmail.com","username":"alevine"},"change_message_id":"ef0a47df4f4ab2fa228a8e6385983ecc2d4e6ca4","unresolved":false,"context_lines":[{"line_number":25,"context_line":""},{"line_number":26,"context_line":"* Add two new columns to the \"test\" table in the database:"},{"line_number":27,"context_line":""},{"line_number":28,"context_line":"  * **product_uuid**:  this field stores the product UUID that the test run is"},{"line_number":29,"context_line":"    associated to."},{"line_number":30,"context_line":""},{"line_number":31,"context_line":"  * **certification**: this is a boolean that indicates whether the test record"}],"source_content_type":"text/x-rst","patch_set":1,"id":"3aaa91ec_92593138","line":28,"range":{"start_line":28,"start_character":4,"end_line":28,"end_character":20},"in_reply_to":"3aaa91ec_9df17ad0","updated":"2016-06-29 15:19:15.000000000","message":"1) Ownership transfer is not a business goal. It should be used for something like certification action. Unless we figure out the certification ownership transfer is not needed at all.\nAs for the certification - once again if you used your test results to certify your Product which happens to be the same Cloud as I have registered as my Product, then does it mean that my Product got certified for OpenStack logo as well? Because in reality - it\u0027s exactly the same Cloud and it did pass the tests.\n\n2) Come on. It sounds like let\u0027s not use GUID because we might rethink its format later.  It doesn\u0027t work like that. We HAVE to have some unique ID for each cloud. In case we rethink cloudID algorithm later we\u0027ll have to support both algorithms and prevent new IDs to look like old ones in order for them not to mix or in the worst case we\u0027ll have to devise some translation. But in any case we can and we should give out and use unique CloudIDs.\nOr maybe I just didn\u0027t get what you\u0027re saying.\n\n3) Again. Right now test results are characterized by a) cloudID b) userID. There is no any other association notion of any kind in our design. Also there is no notion of test results ownership besides its relation to the creator userID in the design and use-cases at the moment.\nSo right now without clarification of the top-level use-cases which CAN potentially lead to the notions of association or ownership, I just don\u0027t understand what you mean. So let\u0027s clarify the top level first.\nWe shouldn\u0027t \"wag the dog\", if you know what I mean :)\n\n4) To get into details we need to answer the following questions:\n\na) I create a Product for your Cloud which already have OpenStack Logo. Do I get my Product certified right away as well?\n\nb) There is a set of published test results made by some consulting company doing testing and validation of all known clouds. The results are 100% PASSED. Can I as a Vendor for this Cloud use these results for its certification if current test results creator (the consulting company) wants to retain ownership because it\u0027s their business.\n\nc) What happens if your Cloud is certified with the OpenStack Logo and I test it from my account and find that it doesn\u0027t pass the tests - what happens with the current certification? Obviously I don\u0027t get one for your Cloud.\n\nd) What happens if I have an account for some Vendor\u0027s Cloud and Vendor didn\u0027t certify it and I run the tests and they pass and I apply for the certification - am I being rejected because I\u0027m not the Vendor? Only Vendor can certify its Cloud? (and I\u0027m saying that because we decided to allow different replicas of the same Cloud (namely Product) for different users.\n\ne) How can user associate his results with the Vendor\u0027s Cloud? Should he apply for it and gets Vendor\u0027s approval?\n\nf) How does Vendor check that the results are valid and actual if they are not produced by his trusted admins (which would\u0027ve not required late association of results)\n\n5) There are more questions like above to be answered and when they get answered I\u0027ll produce even more :)\n\n6) Bottomline: I believe that\u0027s a subject for a real-life discussion during next summit. Until then I\u0027d suggest to not add certification process into RefStack in any way because later we\u0027ll have to rethink and redo it. Let current model work some more time and let\u0027s concentrate on other infrastructural changes to get finished.","commit_id":"1edc4427e8b41709c55b2b76b946ab460ce73992"},{"author":{"_account_id":10526,"name":"Catherine Diep","email":"cdiep@us.ibm.com","username":"cdiep"},"change_message_id":"e14f826ae579ef6ed342c36dc78339c625e95640","unresolved":false,"context_lines":[{"line_number":25,"context_line":""},{"line_number":26,"context_line":"* Add two new columns to the \"test\" table in the database:"},{"line_number":27,"context_line":""},{"line_number":28,"context_line":"  * **product_uuid**:  this field stores the product UUID that the test run is"},{"line_number":29,"context_line":"    associated to."},{"line_number":30,"context_line":""},{"line_number":31,"context_line":"  * **certification**: this is a boolean that indicates whether the test record"}],"source_content_type":"text/x-rst","patch_set":1,"id":"3aaa91ec_9df17ad0","line":28,"range":{"start_line":28,"start_character":4,"end_line":28,"end_character":20},"in_reply_to":"3aaa91ec_ab4402b7","updated":"2016-06-28 18:13:13.000000000","message":"1) If one is not the owner (by being the creator or vendor admins) of the test results, nothing he/she can do (manage) on the results.   So, you cannot really use someone else’s data for your certification purposes.  And, this is what we want to prevent.\nIn this section, the main focus is the mechanism of transferring the data ownership from a creator to a vendor not the certification process.  I will ask Chris for advises regarding OpenStack certification.\n\n2) CloudID is an ID that the tests are run against whether it is a private cloud, a hosted private cloud or a public cloud.  Relying on a different algorithm in the future to resolve the current uniqueness issues of public CloudID is the exact reason why I believe we should not use it as an identifier for a product. \n\n3) Test results should not be associated directly to a vendor.  Test results are associated to products which belong to vendors.  Therefor, there is no need for a vendor ID in a test result.","commit_id":"1edc4427e8b41709c55b2b76b946ab460ce73992"},{"author":{"_account_id":9312,"name":"Alexandre Levine","email":"alexandrelevine@gmail.com","username":"alevine"},"change_message_id":"6684431075e3b61711d3db571f6f0908f3259c18","unresolved":false,"context_lines":[{"line_number":25,"context_line":""},{"line_number":26,"context_line":"* Add two new columns to the \"test\" table in the database:"},{"line_number":27,"context_line":""},{"line_number":28,"context_line":"  * **product_uuid**:  this field stores the product UUID that the test run is"},{"line_number":29,"context_line":"    associated to."},{"line_number":30,"context_line":""},{"line_number":31,"context_line":"  * **certification**: this is a boolean that indicates whether the test record"}],"source_content_type":"text/x-rst","patch_set":1,"id":"3aaa91ec_ab4402b7","line":28,"range":{"start_line":28,"start_character":4,"end_line":28,"end_character":20},"in_reply_to":"3aaa91ec_ee7a1b23","updated":"2016-06-28 13:45:41.000000000","message":"1) This is actually a tricky part and it concerns certification use-case. I understand that you suggest that only owned test-results can be used for validation or certification. But! Let\u0027s suppose my replica of cloud gets OS Logo. It means that your replica of the same cloud is also certified and has OS Logo, isn\u0027t it? Thus a question: can I use your test-results for certification of the same cloud if you made them public but haven\u0027t used them for cloud certification yet? Especially it\u0027s interesting in case I want to validate for some external Guidlines which use the same test-base but include advisory tests or concentrate in some other area (say, block-storage)? You already have uploaded the Tempest results and published them and they already include all I need and I want to reuse your results to get myself a Badge for block-storage certification. Can I?\n\n2. CloudID SHOULD BE unambigiously unique - that\u0027s what we base our design on and that\u0027s the only means for at least some verification of the results. So the cloud with given CloudID is the same cloud in any conditions. If CloudID is different, then it\u0027s a different cloud. How to determine CloudID for all of the cases, especially Amazon and other public clouds - that\u0027s a different story. But once the algorithm defined, all of the records in our tables referring to particular CloudID (PRODUCT_REF_ID) refer to exactly the same cloud.\n\n3. We do not have use-case of test-results association with Vendor. And we do not have VendorID in test results now. So I\u0027m not sure what are we talking about. If such a use-case should exist it should be defined and discussed first. I sort-of understand what you\u0027re aiming for but that\u0027s the same questions as in 1)","commit_id":"1edc4427e8b41709c55b2b76b946ab460ce73992"},{"author":{"_account_id":9312,"name":"Alexandre Levine","email":"alexandrelevine@gmail.com","username":"alevine"},"change_message_id":"05d3625c97e1c4f5fd81bf882ff17589b2dfdd2c","unresolved":false,"context_lines":[{"line_number":25,"context_line":""},{"line_number":26,"context_line":"* Add two new columns to the \"test\" table in the database:"},{"line_number":27,"context_line":""},{"line_number":28,"context_line":"  * **product_uuid**:  this field stores the product UUID that the test run is"},{"line_number":29,"context_line":"    associated to."},{"line_number":30,"context_line":""},{"line_number":31,"context_line":"  * **certification**: this is a boolean that indicates whether the test record"}],"source_content_type":"text/x-rst","patch_set":1,"id":"3aaa91ec_158068bf","line":28,"range":{"start_line":28,"start_character":4,"end_line":28,"end_character":20},"in_reply_to":"3aaa91ec_f1e6b013","updated":"2016-06-24 10:28:29.000000000","message":"1. Creator, i.e. User which did the uploading of the results never changes.\n2. The use-case of association (8) shouldn\u0027t exist as it\u0027s written in the doc - CloudID for the test run result unambiguously defines which Cloud this test is applicable to. So in fact it is applicable to the cloud with CloudID, i.e. for any record in Product table with the PRODUCT_REF_ID\u003d\u003dCloudID\n3. The test is being run for the Cloud (CloudID) by a User (UserID) and that\u0027s all the facts we need to record \"forever\" to be ok with any further handling.","commit_id":"1edc4427e8b41709c55b2b76b946ab460ce73992"},{"author":{"_account_id":9312,"name":"Alexandre Levine","email":"alexandrelevine@gmail.com","username":"alevine"},"change_message_id":"86226fc541ced4f0356349218d65640007c3fd54","unresolved":false,"context_lines":[{"line_number":25,"context_line":""},{"line_number":26,"context_line":"* Add two new columns to the \"test\" table in the database:"},{"line_number":27,"context_line":""},{"line_number":28,"context_line":"  * **product_uuid**:  this field stores the product UUID that the test run is"},{"line_number":29,"context_line":"    associated to."},{"line_number":30,"context_line":""},{"line_number":31,"context_line":"  * **certification**: this is a boolean that indicates whether the test record"}],"source_content_type":"text/x-rst","patch_set":1,"id":"3aaa91ec_2493f150","line":28,"range":{"start_line":28,"start_character":4,"end_line":28,"end_character":20},"in_reply_to":"3aaa91ec_f57c1fed","updated":"2016-06-23 13:24:40.000000000","message":"We already have USER_ID field here I guess so no need to indirectly store it again in the PRODUCT_UUID (which actually constitutes a pair PRODUCT_REF_ID and USER_ID). Also for the certification purposes it doesn\u0027t matter which user did the testing so the one and only unique PRODUCT_REF_ID correctly reflects which cloud (not which registered by some user replica of this cloud) the tests are run for.","commit_id":"1edc4427e8b41709c55b2b76b946ab460ce73992"},{"author":{"_account_id":9312,"name":"Alexandre Levine","email":"alexandrelevine@gmail.com","username":"alevine"},"change_message_id":"a11b4c182b6e1512b263e930e1ba30b842ce87d4","unresolved":false,"context_lines":[{"line_number":28,"context_line":"  * **product_uuid**:  this field stores the product UUID that the test run is"},{"line_number":29,"context_line":"    associated to."},{"line_number":30,"context_line":""},{"line_number":31,"context_line":"  * **certification**: this is a boolean that indicates whether the test record"},{"line_number":32,"context_line":"    was used for OpenStack certification."},{"line_number":33,"context_line":""},{"line_number":34,"context_line":"* Associate a test result to a product"}],"source_content_type":"text/x-rst","patch_set":1,"id":"3aaa91ec_e23e39b7","line":31,"range":{"start_line":31,"start_character":6,"end_line":31,"end_character":19},"updated":"2016-06-22 13:20:12.000000000","message":"I don\u0027t understand this. The certification use-case is right now absent and should be spelled over (maybe added to the doc) before we can create this.\nWhen and how foundation admin can use what for certification? Obviously whatever it is it should pass some criteria for certification. What are the mechanisms for that? Is there a UI in RefStack where foundation admin is presented with the list of Clouds which has test runs passing the criteria and then he marks the Cloud as certified for the Logo and thus this test run is set responsible for this certification?\nAnyways, we need to start from the use-case, not from the DB.","commit_id":"1edc4427e8b41709c55b2b76b946ab460ce73992"},{"author":{"_account_id":9312,"name":"Alexandre Levine","email":"alexandrelevine@gmail.com","username":"alevine"},"change_message_id":"46fd20429d896a82ad13c249c9be7f1722c258e8","unresolved":false,"context_lines":[{"line_number":28,"context_line":"  * **product_uuid**:  this field stores the product UUID that the test run is"},{"line_number":29,"context_line":"    associated to."},{"line_number":30,"context_line":""},{"line_number":31,"context_line":"  * **certification**: this is a boolean that indicates whether the test record"},{"line_number":32,"context_line":"    was used for OpenStack certification."},{"line_number":33,"context_line":""},{"line_number":34,"context_line":"* Associate a test result to a product"}],"source_content_type":"text/x-rst","patch_set":1,"id":"1aa78d24_8f61ac07","line":31,"range":{"start_line":31,"start_character":6,"end_line":31,"end_character":19},"in_reply_to":"1aa78d24_0f7c9cea","updated":"2016-07-07 15:46:43.000000000","message":"It would have to be moved then. We\u0027ll need to define a place for that though.","commit_id":"1edc4427e8b41709c55b2b76b946ab460ce73992"},{"author":{"_account_id":9312,"name":"Alexandre Levine","email":"alexandrelevine@gmail.com","username":"alevine"},"change_message_id":"46fd20429d896a82ad13c249c9be7f1722c258e8","unresolved":false,"context_lines":[{"line_number":28,"context_line":"  * **product_uuid**:  this field stores the product UUID that the test run is"},{"line_number":29,"context_line":"    associated to."},{"line_number":30,"context_line":""},{"line_number":31,"context_line":"  * **certification**: this is a boolean that indicates whether the test record"},{"line_number":32,"context_line":"    was used for OpenStack certification."},{"line_number":33,"context_line":""},{"line_number":34,"context_line":"* Associate a test result to a product"}],"source_content_type":"text/x-rst","patch_set":1,"id":"1aa78d24_8ae07a2a","line":31,"range":{"start_line":31,"start_character":6,"end_line":31,"end_character":19},"in_reply_to":"1aa78d24_2fb238e5","updated":"2016-07-07 15:46:43.000000000","message":"Anonymously - means no UserID in the test results?\nDo you want to keep it that way?\nHow do you achieve security and validity of results with such a process?\nIsn\u0027t it necessary for certification process to have some utility (refstack_client) to run tests, collect UserID and CloudID, package all of it with results (probably sign it) and later or right away send it to RefStack?","commit_id":"1edc4427e8b41709c55b2b76b946ab460ce73992"},{"author":{"_account_id":9312,"name":"Alexandre Levine","email":"alexandrelevine@gmail.com","username":"alevine"},"change_message_id":"6684431075e3b61711d3db571f6f0908f3259c18","unresolved":false,"context_lines":[{"line_number":28,"context_line":"  * **product_uuid**:  this field stores the product UUID that the test run is"},{"line_number":29,"context_line":"    associated to."},{"line_number":30,"context_line":""},{"line_number":31,"context_line":"  * **certification**: this is a boolean that indicates whether the test record"},{"line_number":32,"context_line":"    was used for OpenStack certification."},{"line_number":33,"context_line":""},{"line_number":34,"context_line":"* Associate a test result to a product"}],"source_content_type":"text/x-rst","patch_set":1,"id":"3aaa91ec_4b0c7e94","line":31,"range":{"start_line":31,"start_character":6,"end_line":31,"end_character":19},"in_reply_to":"3aaa91ec_0e104756","updated":"2016-06-28 13:45:41.000000000","message":"Done","commit_id":"1edc4427e8b41709c55b2b76b946ab460ce73992"},{"author":{"_account_id":10526,"name":"Catherine Diep","email":"cdiep@us.ibm.com","username":"cdiep"},"change_message_id":"0fb0c261b2a389bc24df8de22b5a384304a8d6c6","unresolved":false,"context_lines":[{"line_number":28,"context_line":"  * **product_uuid**:  this field stores the product UUID that the test run is"},{"line_number":29,"context_line":"    associated to."},{"line_number":30,"context_line":""},{"line_number":31,"context_line":"  * **certification**: this is a boolean that indicates whether the test record"},{"line_number":32,"context_line":"    was used for OpenStack certification."},{"line_number":33,"context_line":""},{"line_number":34,"context_line":"* Associate a test result to a product"}],"source_content_type":"text/x-rst","patch_set":1,"id":"3aaa91ec_f4885ec7","line":31,"range":{"start_line":31,"start_character":6,"end_line":31,"end_character":19},"in_reply_to":"3aaa91ec_17d42545","updated":"2016-06-23 23:02:35.000000000","message":"I agree that keeping all of our requirements and use-cases on one place can be beneficial. I have added the certification use case to your Google document.\n\nHowever, I also would like the RefStack team to start documenting in blueprints and specs whenever we implement new major features.\n\nAgree that the next step would be providing UI similar to the Vendor UI.\n\nCurrent spec is to add a column (certification) in the test table for the purpose of identifying test-runs that are used for certification.\n\nOnce this spec is implemented we can proceed to UI implementation.","commit_id":"1edc4427e8b41709c55b2b76b946ab460ce73992"},{"author":{"_account_id":9312,"name":"Alexandre Levine","email":"alexandrelevine@gmail.com","username":"alevine"},"change_message_id":"86226fc541ced4f0356349218d65640007c3fd54","unresolved":false,"context_lines":[{"line_number":28,"context_line":"  * **product_uuid**:  this field stores the product UUID that the test run is"},{"line_number":29,"context_line":"    associated to."},{"line_number":30,"context_line":""},{"line_number":31,"context_line":"  * **certification**: this is a boolean that indicates whether the test record"},{"line_number":32,"context_line":"    was used for OpenStack certification."},{"line_number":33,"context_line":""},{"line_number":34,"context_line":"* Associate a test result to a product"}],"source_content_type":"text/x-rst","patch_set":1,"id":"3aaa91ec_17d42545","line":31,"range":{"start_line":31,"start_character":6,"end_line":31,"end_character":19},"in_reply_to":"3aaa91ec_30ecf58e","updated":"2016-06-23 13:24:40.000000000","message":"Thank you.\nWouldn\u0027t it be better to keep all of our requirements and use-cases and design in one place? If my doc isn\u0027t the correct place for this we can copy its text someplace else. Or you can just add your use-cases there.\n\nI\u0027m against spreading the requirements and use-cases wide in various blueprints - it\u0027s unmanageable and unreadable. If some guidelines or rules in OpenStack require us to have such blueprints for new features, we can copy corresponding parts from the wholesome doc when it\u0027s agreed and ready in regard to this parts. This will allow us at any given moment to let any interested party outside to easily get acquainted with our product, requirements and design. The same goes for ourselves (like Monday when we used some of the text during our discussion)\n\nIf you know any other way how to achieve the same simplicity with various separate pieces please let me know.\n\nAs for the problem itself:\nI guess the approach should be the same we employ in RefStack already for Vendor registration. User supplies an application to become Vendor and Foundation admin approves or rejects it and have nice UI to see current applications.\nThe same here: User supplies a request for some of its test-runs to be used as a base for certification. Foundation admin approves or rejects it in the similar UI.","commit_id":"1edc4427e8b41709c55b2b76b946ab460ce73992"},{"author":{"_account_id":10526,"name":"Catherine Diep","email":"cdiep@us.ibm.com","username":"cdiep"},"change_message_id":"9452e3d5cb3cecc271c76058a8fbc29e27ec3251","unresolved":false,"context_lines":[{"line_number":28,"context_line":"  * **product_uuid**:  this field stores the product UUID that the test run is"},{"line_number":29,"context_line":"    associated to."},{"line_number":30,"context_line":""},{"line_number":31,"context_line":"  * **certification**: this is a boolean that indicates whether the test record"},{"line_number":32,"context_line":"    was used for OpenStack certification."},{"line_number":33,"context_line":""},{"line_number":34,"context_line":"* Associate a test result to a product"}],"source_content_type":"text/x-rst","patch_set":1,"id":"3aaa91ec_0e104756","line":31,"range":{"start_line":31,"start_character":6,"end_line":31,"end_character":19},"in_reply_to":"3aaa91ec_d8b9f716","updated":"2016-06-24 18:20:17.000000000","message":"Thanks, I responded to your comments in the document.\nI did define the UI flows in the doc in the \"Mark a test record (result) as “used for OpenStack certification” section.  Please review!","commit_id":"1edc4427e8b41709c55b2b76b946ab460ce73992"},{"author":{"_account_id":10526,"name":"Catherine Diep","email":"cdiep@us.ibm.com","username":"cdiep"},"change_message_id":"d4416c74809881475354edd891ff96719d91baf5","unresolved":false,"context_lines":[{"line_number":28,"context_line":"  * **product_uuid**:  this field stores the product UUID that the test run is"},{"line_number":29,"context_line":"    associated to."},{"line_number":30,"context_line":""},{"line_number":31,"context_line":"  * **certification**: this is a boolean that indicates whether the test record"},{"line_number":32,"context_line":"    was used for OpenStack certification."},{"line_number":33,"context_line":""},{"line_number":34,"context_line":"* Associate a test result to a product"}],"source_content_type":"text/x-rst","patch_set":1,"id":"3aaa91ec_30ecf58e","line":31,"range":{"start_line":31,"start_character":6,"end_line":31,"end_character":19},"in_reply_to":"3aaa91ec_e23e39b7","updated":"2016-06-22 19:12:57.000000000","message":"Good suggestion,  I will add a blueprint for the certification use case.\n\nMay be certification is not a good name for the column.  The intention of this column is to identify the records that have been used by the Foundation for OpenStack Powered Logo applications. These records should remain in the RefStack database and only Foundation admins can manage them (delete, un-certify, etc)  \n\nHere are the answers to your questions.\n\n1) When and how foundation admin can use what for certification?\n[Answer] This process is defined by the Foundation in the \"How to Run the Test\" section of [1]\n\n2) Obviously whatever it is it should pass some criteria for certification. What are the mechanisms for that?\n[Answer] See criteria and mechanism in [1].\n\n3) Is there a UI in RefStack where foundation admin is presented with the list of Clouds which has test runs passing the criteria and then he marks the Cloud as certified for the Logo and thus this test run is set responsible for this certification?\n[Answer] As described in the procedure in [1], the link of the data record will be send to the Foundation by the OpenStack Powered Logo applicant.  There is no UI for such function in RefStack, though this would be a good feature to add to RefStack.\n\n4) Anyways, we need to start from the use-case, not from the DB.\n[Answer] I will document the use case in a blueprint.  \n\n[1] http://www.openstack.org/brand/interop/.","commit_id":"1edc4427e8b41709c55b2b76b946ab460ce73992"},{"author":{"_account_id":7822,"name":"Chris Hoge (not involved in upstream development)","email":"chris@hogepodge.com","username":"hogepodge"},"change_message_id":"0c277b228b2a69f4614e92a9ef391894045afb92","unresolved":false,"context_lines":[{"line_number":28,"context_line":"  * **product_uuid**:  this field stores the product UUID that the test run is"},{"line_number":29,"context_line":"    associated to."},{"line_number":30,"context_line":""},{"line_number":31,"context_line":"  * **certification**: this is a boolean that indicates whether the test record"},{"line_number":32,"context_line":"    was used for OpenStack certification."},{"line_number":33,"context_line":""},{"line_number":34,"context_line":"* Associate a test result to a product"}],"source_content_type":"text/x-rst","patch_set":1,"id":"1aa78d24_2fb238e5","line":31,"range":{"start_line":31,"start_character":6,"end_line":31,"end_character":19},"in_reply_to":"3aaa91ec_e23e39b7","updated":"2016-07-07 15:27:04.000000000","message":"Vendors send me a link to the refstack results. Currently, they are usually anonymously uploaded.","commit_id":"1edc4427e8b41709c55b2b76b946ab460ce73992"},{"author":{"_account_id":7822,"name":"Chris Hoge (not involved in upstream development)","email":"chris@hogepodge.com","username":"hogepodge"},"change_message_id":"0c277b228b2a69f4614e92a9ef391894045afb92","unresolved":false,"context_lines":[{"line_number":28,"context_line":"  * **product_uuid**:  this field stores the product UUID that the test run is"},{"line_number":29,"context_line":"    associated to."},{"line_number":30,"context_line":""},{"line_number":31,"context_line":"  * **certification**: this is a boolean that indicates whether the test record"},{"line_number":32,"context_line":"    was used for OpenStack certification."},{"line_number":33,"context_line":""},{"line_number":34,"context_line":"* Associate a test result to a product"}],"source_content_type":"text/x-rst","patch_set":1,"id":"1aa78d24_0f7c9cea","line":31,"range":{"start_line":31,"start_character":6,"end_line":31,"end_character":19},"in_reply_to":"3aaa91ec_f4885ec7","updated":"2016-07-07 15:27:04.000000000","message":"A google doc is not the appropriate place to keep design documents. These things need to be in the git repository.","commit_id":"1edc4427e8b41709c55b2b76b946ab460ce73992"},{"author":{"_account_id":9312,"name":"Alexandre Levine","email":"alexandrelevine@gmail.com","username":"alevine"},"change_message_id":"05d3625c97e1c4f5fd81bf882ff17589b2dfdd2c","unresolved":false,"context_lines":[{"line_number":28,"context_line":"  * **product_uuid**:  this field stores the product UUID that the test run is"},{"line_number":29,"context_line":"    associated to."},{"line_number":30,"context_line":""},{"line_number":31,"context_line":"  * **certification**: this is a boolean that indicates whether the test record"},{"line_number":32,"context_line":"    was used for OpenStack certification."},{"line_number":33,"context_line":""},{"line_number":34,"context_line":"* Associate a test result to a product"}],"source_content_type":"text/x-rst","patch_set":1,"id":"3aaa91ec_d8b9f716","line":31,"range":{"start_line":31,"start_character":6,"end_line":31,"end_character":19},"in_reply_to":"3aaa91ec_f4885ec7","updated":"2016-06-24 10:28:29.000000000","message":"I commented in the doc.\nAnd we need to define UI and the flow first, before approving the DB, because the second (DB) is dictated by the first.","commit_id":"1edc4427e8b41709c55b2b76b946ab460ce73992"},{"author":{"_account_id":17716,"name":"Luz Cazares","email":"luz.cazares@intel.com","username":"luzcazares"},"change_message_id":"6110f2087093cf9125f737f3640171f1eb38e5bc","unresolved":false,"context_lines":[{"line_number":1,"context_line":"Launchpad blueprint:"},{"line_number":2,"context_line":""},{"line_number":3,"context_line":"* https://blueprints.launchpad.net/refstack/+spec/test-results-ownership"},{"line_number":4,"context_line":"* https://blueprints.launchpad.net/refstack/+spec/certification-test-record"}],"source_content_type":"text/x-rst","patch_set":4,"id":"9ad45d7e_bf27c6b4","line":1,"range":{"start_line":1,"start_character":0,"end_line":1,"end_character":2},"updated":"2016-08-10 22:47:28.000000000","message":"Add spec name","commit_id":"1b8f9201e96339aac3169b69fe7564dca31f1448"},{"author":{"_account_id":17716,"name":"Luz Cazares","email":"luz.cazares@intel.com","username":"luzcazares"},"change_message_id":"3b25096a2232ddb9875f125cba682943501f944a","unresolved":false,"context_lines":[{"line_number":105,"context_line":""},{"line_number":106,"context_line":"None."},{"line_number":107,"context_line":""},{"line_number":108,"context_line":"Other end user impact"},{"line_number":109,"context_line":"---------------------"},{"line_number":110,"context_line":""},{"line_number":111,"context_line":"None"}],"source_content_type":"text/x-rst","patch_set":4,"id":"3aaa91ec_27c2a585","line":108,"range":{"start_line":108,"start_character":0,"end_line":108,"end_character":5},"updated":"2016-06-30 02:48:39.000000000","message":"Refstack-client has a --upload flag, is the client impacted? should the upload method be also updated?","commit_id":"1b8f9201e96339aac3169b69fe7564dca31f1448"},{"author":{"_account_id":10526,"name":"Catherine Diep","email":"cdiep@us.ibm.com","username":"cdiep"},"change_message_id":"b16a27b4f3a826311e0ac74ff8424d0ddc5f882d","unresolved":false,"context_lines":[{"line_number":105,"context_line":""},{"line_number":106,"context_line":"None."},{"line_number":107,"context_line":""},{"line_number":108,"context_line":"Other end user impact"},{"line_number":109,"context_line":"---------------------"},{"line_number":110,"context_line":""},{"line_number":111,"context_line":"None"}],"source_content_type":"text/x-rst","patch_set":4,"id":"1aa78d24_ab49e990","line":108,"range":{"start_line":108,"start_character":0,"end_line":108,"end_character":5},"in_reply_to":"3aaa91ec_27c2a585","updated":"2016-07-01 22:39:09.000000000","message":"IMO, keeping the association of test results to product_ids at the server side makes sense.  My thinking is that the users can run as many test runs as they want on the cloud before associating it to a product.\n\nAs said, we need to first agree on the association of products to test runs should be done via product_ref_id (CPID) or product_id (product UUID).","commit_id":"1b8f9201e96339aac3169b69fe7564dca31f1448"},{"author":{"_account_id":10234,"name":"Andrey Pavlov","email":"andrey.mp@gmail.com","username":"andrey-mp"},"change_message_id":"f1c151f43df1a54dbb6814b8b325283a953b86b5","unresolved":false,"context_lines":[{"line_number":158,"context_line":"  .. parsed-literal::"},{"line_number":159,"context_line":"    {"},{"line_number":160,"context_line":"       {"},{"line_number":161,"context_line":"          \"verification_status\" : 1,"},{"line_number":162,"context_line":"          \"product_version_id\": \"85346866-307f-4052-ba31-ff6270635e14\","},{"line_number":163,"context_line":"          \"required\": []"},{"line_number":164,"context_line":"       }"}],"source_content_type":"text/x-rst","patch_set":8,"id":"9a89bdaa_6b913a52","line":161,"range":{"start_line":161,"start_character":11,"end_line":161,"end_character":30},"updated":"2016-09-06 10:35:01.000000000","message":"it\u0027s from another spec...","commit_id":"aa3c9ac45d0ca42d0325c6a3f278ffdeda19c5ed"},{"author":{"_account_id":10526,"name":"Catherine Diep","email":"cdiep@us.ibm.com","username":"cdiep"},"change_message_id":"828e163201d06f3160a70334e8b047d70896df4e","unresolved":false,"context_lines":[{"line_number":158,"context_line":"  .. parsed-literal::"},{"line_number":159,"context_line":"    {"},{"line_number":160,"context_line":"       {"},{"line_number":161,"context_line":"          \"verification_status\" : 1,"},{"line_number":162,"context_line":"          \"product_version_id\": \"85346866-307f-4052-ba31-ff6270635e14\","},{"line_number":163,"context_line":"          \"required\": []"},{"line_number":164,"context_line":"       }"}],"source_content_type":"text/x-rst","patch_set":8,"id":"9a89bdaa_d43bac36","line":161,"range":{"start_line":161,"start_character":11,"end_line":161,"end_character":30},"in_reply_to":"9a89bdaa_6b913a52","updated":"2016-09-06 17:20:30.000000000","message":"Yes, it is as it is described in the \"Note\" section below.","commit_id":"aa3c9ac45d0ca42d0325c6a3f278ffdeda19c5ed"},{"author":{"_account_id":10234,"name":"Andrey Pavlov","email":"andrey.mp@gmail.com","username":"andrey-mp"},"change_message_id":"f1c151f43df1a54dbb6814b8b325283a953b86b5","unresolved":false,"context_lines":[{"line_number":193,"context_line":"* JSON schema definition for the response data:"},{"line_number":194,"context_line":""},{"line_number":195,"context_line":"  Update to add product_version_id to the response body."},{"line_number":196,"context_line":""},{"line_number":197,"context_line":"  .. parsed-literal::"},{"line_number":198,"context_line":"    {"},{"line_number":199,"context_line":"       pagination: {"}],"source_content_type":"text/x-rst","patch_set":8,"id":"9a89bdaa_0bffce67","line":196,"updated":"2016-09-06 10:35:01.000000000","message":"1) what should be returned if no association in DB exists?\nabsent field or field with empty value?\n\n2) whether it is necessary to update the filters?","commit_id":"aa3c9ac45d0ca42d0325c6a3f278ffdeda19c5ed"},{"author":{"_account_id":10526,"name":"Catherine Diep","email":"cdiep@us.ibm.com","username":"cdiep"},"change_message_id":"828e163201d06f3160a70334e8b047d70896df4e","unresolved":false,"context_lines":[{"line_number":193,"context_line":"* JSON schema definition for the response data:"},{"line_number":194,"context_line":""},{"line_number":195,"context_line":"  Update to add product_version_id to the response body."},{"line_number":196,"context_line":""},{"line_number":197,"context_line":"  .. parsed-literal::"},{"line_number":198,"context_line":"    {"},{"line_number":199,"context_line":"       pagination: {"}],"source_content_type":"text/x-rst","patch_set":8,"id":"9a89bdaa_9f24ddaf","line":196,"in_reply_to":"9a89bdaa_0bffce67","updated":"2016-09-06 17:20:30.000000000","message":"1) If not association in DB it will return field with empty value.\n2) Yea, we should add filter.  I am debating where we should add the filter in this patch or subsequent patch.","commit_id":"aa3c9ac45d0ca42d0325c6a3f278ffdeda19c5ed"},{"author":{"_account_id":10526,"name":"Catherine Diep","email":"cdiep@us.ibm.com","username":"cdiep"},"change_message_id":"878ab2026d8a55205ed067e0e47483c4396d9b3d","unresolved":false,"context_lines":[{"line_number":193,"context_line":"* JSON schema definition for the response data:"},{"line_number":194,"context_line":""},{"line_number":195,"context_line":"  Update to add product_version_id to the response body."},{"line_number":196,"context_line":""},{"line_number":197,"context_line":"  .. parsed-literal::"},{"line_number":198,"context_line":"    {"},{"line_number":199,"context_line":"       pagination: {"}],"source_content_type":"text/x-rst","patch_set":8,"id":"9a89bdaa_fb740a37","line":196,"in_reply_to":"9a89bdaa_9f24ddaf","updated":"2016-09-07 03:15:24.000000000","message":"1) Update one of the example to show empty product_version_id field.\n2)Update filter in the parameter section.","commit_id":"aa3c9ac45d0ca42d0325c6a3f278ffdeda19c5ed"},{"author":{"_account_id":10234,"name":"Andrey Pavlov","email":"andrey.mp@gmail.com","username":"andrey-mp"},"change_message_id":"f1c151f43df1a54dbb6814b8b325283a953b86b5","unresolved":false,"context_lines":[{"line_number":208,"context_line":"             id: \"7943e04a-2b95-453c-b627-8a24b2c6faa0\","},{"line_number":209,"context_line":"             duration_seconds: 0,"},{"line_number":210,"context_line":"             verification_status : 0,"},{"line_number":211,"context_line":"             product_version_id; \"85346866-307f-4052-ba31-ff6270635e14\""},{"line_number":212,"context_line":"          },"},{"line_number":213,"context_line":"          {"},{"line_number":214,"context_line":"             url: \"https://refstack.openstack.org/#/results/91ae10c5-ecf5-4823-81d4-09836dc212cf\","}],"source_content_type":"text/x-rst","patch_set":8,"id":"9a89bdaa_cb8fa621","line":211,"range":{"start_line":211,"start_character":31,"end_line":211,"end_character":32},"updated":"2016-09-06 10:35:01.000000000","message":"\u0027;\u0027 -\u003e \u0027:\u0027","commit_id":"aa3c9ac45d0ca42d0325c6a3f278ffdeda19c5ed"},{"author":{"_account_id":10526,"name":"Catherine Diep","email":"cdiep@us.ibm.com","username":"cdiep"},"change_message_id":"878ab2026d8a55205ed067e0e47483c4396d9b3d","unresolved":false,"context_lines":[{"line_number":208,"context_line":"             id: \"7943e04a-2b95-453c-b627-8a24b2c6faa0\","},{"line_number":209,"context_line":"             duration_seconds: 0,"},{"line_number":210,"context_line":"             verification_status : 0,"},{"line_number":211,"context_line":"             product_version_id; \"85346866-307f-4052-ba31-ff6270635e14\""},{"line_number":212,"context_line":"          },"},{"line_number":213,"context_line":"          {"},{"line_number":214,"context_line":"             url: \"https://refstack.openstack.org/#/results/91ae10c5-ecf5-4823-81d4-09836dc212cf\","}],"source_content_type":"text/x-rst","patch_set":8,"id":"9a89bdaa_3b8af21b","line":211,"range":{"start_line":211,"start_character":31,"end_line":211,"end_character":32},"in_reply_to":"9a89bdaa_cb8fa621","updated":"2016-09-07 03:15:24.000000000","message":"Done","commit_id":"aa3c9ac45d0ca42d0325c6a3f278ffdeda19c5ed"}]}