)]}'
{"specs/pike/approved/upload-subunit-tests.rst":[{"author":{"_account_id":10526,"name":"Catherine Diep","email":"cdiep@us.ibm.com","username":"cdiep"},"change_message_id":"dc0ff16a5be2582282f9779abb9c1a7514daf113","unresolved":false,"context_lines":[{"line_number":10,"context_line":"functionality to allow for upload of the subunit data which corresponds"},{"line_number":11,"context_line":"to a given set of test results."},{"line_number":12,"context_line":""},{"line_number":13,"context_line":"Problem description"},{"line_number":14,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"},{"line_number":15,"context_line":""},{"line_number":16,"context_line":"In the past, all test results uploaded to the RefStack website have consisted"}],"source_content_type":"text/x-rst","patch_set":1,"id":"1f1a1f67_98032c74","line":13,"range":{"start_line":13,"start_character":0,"end_line":13,"end_character":19},"updated":"2017-07-14 17:35:20.000000000","message":"Please include a background paragraph (sample below) to explain why the uploading data is a JSON file with only passing tests.\n\nBackground:\n\nAt the start of the RefStack project, DefCore (now known as the Interop-WG) was very concerned about the private data that may be included in the subunit files.  Vendors may hesitate to upload data to RefStack for fear of unintentionally revealing vendor specific private data such as reasons for test failures, etc.  The unanimous agreement at the time was that for interop purposes, DefCore/RefStack should only care about identifying the passing tests and not the non-passing tests which may be due to tests failing or being skipped.  The concern of data tampering in a text JSON file was also discussed.  The conclusion at the time was that the end customers will be the ones who would find out and \"punish\" those who alter the test results.","commit_id":"2cef4864629f08d5192b78a1d618251374b58810"},{"author":{"_account_id":10526,"name":"Catherine Diep","email":"cdiep@us.ibm.com","username":"cdiep"},"change_message_id":"dc0ff16a5be2582282f9779abb9c1a7514daf113","unresolved":false,"context_lines":[{"line_number":20,"context_line":"interoperable than it actually is."},{"line_number":21,"context_line":""},{"line_number":22,"context_line":""},{"line_number":23,"context_line":"Proposed change"},{"line_number":24,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"},{"line_number":25,"context_line":""},{"line_number":26,"context_line":"In order to move towards having a more reliable and verifiable collection"}],"source_content_type":"text/x-rst","patch_set":1,"id":"1f1a1f67_28938e51","line":23,"range":{"start_line":23,"start_character":0,"end_line":23,"end_character":15},"updated":"2017-07-14 17:35:20.000000000","message":"The first design point that should be considered is to determine the format and location to save the subunit data in RefStack.\n\nExample options are:\n\n1) Save subunit files as-is in a file system.\n2) Save subunit data in the RefStack database. In this case, new RefStack data model design is needed.\n3) Save subunit data in a new database.  In this case, existing tools such as subunit2SQL should be considered.\n\nIn all cases, update of RefStack database should also be defined to link the newly added data to the existing data.","commit_id":"2cef4864629f08d5192b78a1d618251374b58810"},{"author":{"_account_id":17716,"name":"Luz Cazares","email":"luz.cazares@intel.com","username":"luzcazares"},"change_message_id":"e9c4eaa4b5d09f9f8fb7415241d800d657ae9192","unresolved":false,"context_lines":[{"line_number":46,"context_line":""},{"line_number":47,"context_line":"None previously discussed."},{"line_number":48,"context_line":""},{"line_number":49,"context_line":"Data model impact"},{"line_number":50,"context_line":"-----------------"},{"line_number":51,"context_line":"This change should impact RefStack\u0027s data model in that it will introduce"},{"line_number":52,"context_line":"the import of the subunit data into the db, as well as linking that data to"}],"source_content_type":"text/x-rst","patch_set":1,"id":"1f1a1f67_6149ca89","line":49,"range":{"start_line":49,"start_character":0,"end_line":49,"end_character":17},"updated":"2017-07-13 18:22:58.000000000","message":"subunit2sql creates its own DB, with its own model. Other tools can be used to access the info on that DB model.\n\nSee:\nhttps://github.com/openstack-infra/subunit2sql/\nhttps://github.com/openstack-infra/subunit2sql/blob/master/subunit2sql/db/models.py\nhttp://blog.kortar.org/?p\u003d212\n\nI think additional information is needed in regards of the implementation and expected results for using subunit2SQL. How is it going to affect current data flow on refstack.\nAlso if refstack table(s) are going to change (like addind a new field). Then add table schema to visualize the actual change).","commit_id":"2cef4864629f08d5192b78a1d618251374b58810"},{"author":{"_account_id":17716,"name":"Luz Cazares","email":"luz.cazares@intel.com","username":"luzcazares"},"change_message_id":"e9c4eaa4b5d09f9f8fb7415241d800d657ae9192","unresolved":false,"context_lines":[{"line_number":52,"context_line":"the import of the subunit data into the db, as well as linking that data to"},{"line_number":53,"context_line":"an existing table in the RefStack database."},{"line_number":54,"context_line":""},{"line_number":55,"context_line":"REST API impact"},{"line_number":56,"context_line":"---------------"},{"line_number":57,"context_line":""},{"line_number":58,"context_line":"None"},{"line_number":59,"context_line":""},{"line_number":60,"context_line":"Security impact"},{"line_number":61,"context_line":"---------------"}],"source_content_type":"text/x-rst","patch_set":1,"id":"1f1a1f67_8114e64f","line":58,"range":{"start_line":55,"start_character":0,"end_line":58,"end_character":4},"updated":"2017-07-13 18:22:58.000000000","message":"This will be impacted too right? So that it can handle the upload of a subunit file? \nNeed additional information about what needs to be changed","commit_id":"2cef4864629f08d5192b78a1d618251374b58810"},{"author":{"_account_id":17716,"name":"Luz Cazares","email":"luz.cazares@intel.com","username":"luzcazares"},"change_message_id":"e9c4eaa4b5d09f9f8fb7415241d800d657ae9192","unresolved":false,"context_lines":[{"line_number":57,"context_line":""},{"line_number":58,"context_line":"None"},{"line_number":59,"context_line":""},{"line_number":60,"context_line":"Security impact"},{"line_number":61,"context_line":"---------------"},{"line_number":62,"context_line":"It has been suggested that uploading the subunit data for tests may expose"},{"line_number":63,"context_line":"private data. However, it was determined in the 6/ /2017 RefStack meeting[1]"},{"line_number":64,"context_line":"that if any such data is revealed through this upload, it would be due to a"}],"source_content_type":"text/x-rst","patch_set":1,"id":"1f1a1f67_8409942d","line":61,"range":{"start_line":60,"start_character":0,"end_line":61,"end_character":15},"updated":"2017-07-13 18:22:58.000000000","message":"One concern might be preventing or handling malicious attepmts to upload files via the rest api?","commit_id":"2cef4864629f08d5192b78a1d618251374b58810"},{"author":{"_account_id":10526,"name":"Catherine Diep","email":"cdiep@us.ibm.com","username":"cdiep"},"change_message_id":"dc0ff16a5be2582282f9779abb9c1a7514daf113","unresolved":false,"context_lines":[{"line_number":60,"context_line":"Security impact"},{"line_number":61,"context_line":"---------------"},{"line_number":62,"context_line":"It has been suggested that uploading the subunit data for tests may expose"},{"line_number":63,"context_line":"private data. However, it was determined in the 6/ /2017 RefStack meeting[1]"},{"line_number":64,"context_line":"that if any such data is revealed through this upload, it would be due to a"},{"line_number":65,"context_line":"leak in tempest\u0027s logging procedures, not the upload of this new type of data."},{"line_number":66,"context_line":""}],"source_content_type":"text/x-rst","patch_set":1,"id":"1f1a1f67_d8f3e472","line":63,"range":{"start_line":63,"start_character":23,"end_line":63,"end_character":57},"updated":"2017-07-14 17:35:20.000000000","message":"Please also mention the Interop-WG IRC meeting and action plan documented in [1]\n\n[1] http://eavesdrop.openstack.org/meetings/interopwg/2017/interopwg.2017-06-28-16.00.log.html","commit_id":"2cef4864629f08d5192b78a1d618251374b58810"},{"author":{"_account_id":17716,"name":"Luz Cazares","email":"luz.cazares@intel.com","username":"luzcazares"},"change_message_id":"e9c4eaa4b5d09f9f8fb7415241d800d657ae9192","unresolved":false,"context_lines":[{"line_number":79,"context_line":""},{"line_number":80,"context_line":"None"},{"line_number":81,"context_line":""},{"line_number":82,"context_line":"Other deployer impact"},{"line_number":83,"context_line":"---------------------"},{"line_number":84,"context_line":""},{"line_number":85,"context_line":"None"},{"line_number":86,"context_line":""},{"line_number":87,"context_line":"Developer impact"},{"line_number":88,"context_line":"----------------"}],"source_content_type":"text/x-rst","patch_set":1,"id":"1f1a1f67_e45e0811","line":85,"range":{"start_line":82,"start_character":0,"end_line":85,"end_character":4},"updated":"2017-07-13 18:22:58.000000000","message":"Problably on refstack-client, it might need a way to upload the subunit file by consuming the new api feature?","commit_id":"2cef4864629f08d5192b78a1d618251374b58810"},{"author":{"_account_id":10526,"name":"Catherine Diep","email":"cdiep@us.ibm.com","username":"cdiep"},"change_message_id":"7e5e98a026d6e35dc35021366b3c9f16291acbfd","unresolved":false,"context_lines":[{"line_number":48,"context_line":""},{"line_number":49,"context_line":"Data model impact"},{"line_number":50,"context_line":"-----------------"},{"line_number":51,"context_line":"This change should impact RefStack\u0027s data model in that it will introduce"},{"line_number":52,"context_line":"the import of the subunit data into a new, separate db, as well as linking"},{"line_number":53,"context_line":"that data to an existing table in the RefStack database."},{"line_number":54,"context_line":""}],"source_content_type":"text/x-rst","patch_set":2,"id":"1f1a1f67_bc0ebc2f","line":51,"range":{"start_line":51,"start_character":25,"end_line":51,"end_character":48},"updated":"2017-07-14 18:55:06.000000000","message":"No RefStack data model impact if a new database is added to store the subunit data.","commit_id":"8a787daf7444fd65a72884896433ebb8d61300db"},{"author":{"_account_id":10526,"name":"Catherine Diep","email":"cdiep@us.ibm.com","username":"cdiep"},"change_message_id":"7e5e98a026d6e35dc35021366b3c9f16291acbfd","unresolved":false,"context_lines":[{"line_number":138,"context_line":"example, the table \"tests\" is linked to the class \"Test\"."},{"line_number":139,"context_line":""},{"line_number":140,"context_line":"Outside of needing to incorporate this addition of data into this new database"},{"line_number":141,"context_line":"into RefStack\u0027s data model, we will need to add a field to the $TABLE table"},{"line_number":142,"context_line":"in the built-in refstack database that contains the id of an entry in the"},{"line_number":143,"context_line":"\"test_runs\" table of the subunit db."},{"line_number":144,"context_line":""},{"line_number":145,"context_line":"REST API impact"},{"line_number":146,"context_line":"---------------"}],"source_content_type":"text/x-rst","patch_set":2,"id":"1f1a1f67_fc5d54e9","line":143,"range":{"start_line":141,"start_character":27,"end_line":143,"end_character":36},"updated":"2017-07-14 18:55:06.000000000","message":"Let\u0027s specify the table name (\"test\") and a new field name that should be added to this table.","commit_id":"8a787daf7444fd65a72884896433ebb8d61300db"},{"author":{"_account_id":10526,"name":"Catherine Diep","email":"cdiep@us.ibm.com","username":"cdiep"},"change_message_id":"55bd1872708ae0eba0bcbeb7f586673a59799c7e","unresolved":false,"context_lines":[{"line_number":13,"context_line":"Problem description"},{"line_number":14,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"},{"line_number":15,"context_line":""},{"line_number":16,"context_line":"In the past, all test results uploaded to the RefStack website have consisted"},{"line_number":17,"context_line":"of a json file containing the only the portion of the RefStack run pertaining"},{"line_number":18,"context_line":"to the passed tests. This limitation dates back to the the start of the"},{"line_number":19,"context_line":"RefStack project. At that time, Defcore (which is now known as interop-WG)"}],"source_content_type":"text/x-rst","patch_set":3,"id":"1f1a1f67_50af567c","line":16,"range":{"start_line":16,"start_character":0,"end_line":16,"end_character":11},"updated":"2017-07-18 23:50:01.000000000","message":"What being described in the paragraph followed is the current implementation.  Should \"In the past\" be \"Currently\"?","commit_id":"7dc3cde7e814c18894e2809fa6f2f9e34ac12d18"},{"author":{"_account_id":10526,"name":"Catherine Diep","email":"cdiep@us.ibm.com","username":"cdiep"},"change_message_id":"55bd1872708ae0eba0bcbeb7f586673a59799c7e","unresolved":false,"context_lines":[{"line_number":42,"context_line":"verification as well as debugging assistance. Upload of subunit data will not,"},{"line_number":43,"context_line":"for the time being, be required."},{"line_number":44,"context_line":""},{"line_number":45,"context_line":"The most user-visible part of this change would be the creation of a flag"},{"line_number":46,"context_line":"option which enables the upload of the subunit data, which would modify the"},{"line_number":47,"context_line":"existing procedure by attaching the subunit data to an existing RefStack"},{"line_number":48,"context_line":"entry in the database."},{"line_number":49,"context_line":""}],"source_content_type":"text/x-rst","patch_set":3,"id":"1f1a1f67_a3c87d73","line":46,"range":{"start_line":45,"start_character":67,"end_line":46,"end_character":6},"updated":"2017-07-18 23:50:01.000000000","message":"The flag option is implemented at the refstack-client side. At the RefStack side, a new REST API should be implemented for uploading of the subunit data.","commit_id":"7dc3cde7e814c18894e2809fa6f2f9e34ac12d18"},{"author":{"_account_id":10526,"name":"Catherine Diep","email":"cdiep@us.ibm.com","username":"cdiep"},"change_message_id":"55bd1872708ae0eba0bcbeb7f586673a59799c7e","unresolved":false,"context_lines":[{"line_number":55,"context_line":"One of the biggest design decisions to be made on this change have to do with"},{"line_number":56,"context_line":"how exactly to manage the subunit data. A few options available to us might be"},{"line_number":57,"context_line":"to:"},{"line_number":58,"context_line":"* Save subunit files as-is in a file system. This has the benefit of being the"},{"line_number":59,"context_line":"  least processing-intensive option for saving the data, as it would literally"},{"line_number":60,"context_line":"  just save the output into a file. It may, however, make subunit data upload"},{"line_number":61,"context_line":"  a bit less elegant, as well as being a deviation from the way test run data"},{"line_number":62,"context_line":"  is managed throughout the rest of RefStack."},{"line_number":63,"context_line":"* Save subunit data in a new database. This could be done using the existing"},{"line_number":64,"context_line":"  subunit2sql toolkit[3]. This option would do a lot of the heavy lifting for"},{"line_number":65,"context_line":"  us, as far as data import goes, as well as keeping the storage method of test"}],"source_content_type":"text/x-rst","patch_set":3,"id":"1f1a1f67_03117104","line":62,"range":{"start_line":58,"start_character":2,"end_line":62,"end_character":45},"updated":"2017-07-18 23:50:01.000000000","message":"Move this to the \"Alternatives\" section?","commit_id":"7dc3cde7e814c18894e2809fa6f2f9e34ac12d18"},{"author":{"_account_id":10526,"name":"Catherine Diep","email":"cdiep@us.ibm.com","username":"cdiep"},"change_message_id":"55bd1872708ae0eba0bcbeb7f586673a59799c7e","unresolved":false,"context_lines":[{"line_number":65,"context_line":"  us, as far as data import goes, as well as keeping the storage method of test"},{"line_number":66,"context_line":"  data consistent across the board, but keeping the data in a separate database"},{"line_number":67,"context_line":"  would carry with it some operational overhead."},{"line_number":68,"context_line":"* Save subunit data in the RefStack database. Like the option listed above,"},{"line_number":69,"context_line":"  this option keeps test run data stored consistently across refstack, which"},{"line_number":70,"context_line":"  would make the changes to the API more consistent as well. It would also"},{"line_number":71,"context_line":"  avoid the overhead that would result from using a separate database, as well"},{"line_number":72,"context_line":"  as any redundancies that resulted from using a second, separate database."},{"line_number":73,"context_line":"  However, any redundancy would be fairly minor due to the extremely limited"},{"line_number":74,"context_line":"  scope of the data we are currently storing from each test run, and this would"},{"line_number":75,"context_line":"  leave more of the implementation up to us, which, because of how well"},{"line_number":76,"context_line":"  subunit2sql\u0027s schema fulfills the needs of this change, may be wholly"},{"line_number":77,"context_line":"  unnecessary."},{"line_number":78,"context_line":""},{"line_number":79,"context_line":""},{"line_number":80,"context_line":"Possible toolsets to use:"}],"source_content_type":"text/x-rst","patch_set":3,"id":"1f1a1f67_c3f6d9d7","line":77,"range":{"start_line":68,"start_character":2,"end_line":77,"end_character":14},"updated":"2017-07-18 23:50:01.000000000","message":"ditto","commit_id":"7dc3cde7e814c18894e2809fa6f2f9e34ac12d18"},{"author":{"_account_id":10526,"name":"Catherine Diep","email":"cdiep@us.ibm.com","username":"cdiep"},"change_message_id":"55bd1872708ae0eba0bcbeb7f586673a59799c7e","unresolved":false,"context_lines":[{"line_number":86,"context_line":""},{"line_number":87,"context_line":"None previously discussed."},{"line_number":88,"context_line":""},{"line_number":89,"context_line":"Data model impact"},{"line_number":90,"context_line":"-----------------"},{"line_number":91,"context_line":"This change will impact RefStack\u0027s data model regardless of how data"},{"line_number":92,"context_line":"management ends up being implemented."}],"source_content_type":"text/x-rst","patch_set":3,"id":"1f1a1f67_83710147","line":89,"range":{"start_line":89,"start_character":0,"end_line":89,"end_character":17},"updated":"2017-07-18 23:50:01.000000000","message":"No change to the RefStack data model is needed. Linkage between the RefStack to the new subunit data will be done as a key-value pair in the RefStack meta data table for bow.","commit_id":"7dc3cde7e814c18894e2809fa6f2f9e34ac12d18"},{"author":{"_account_id":10526,"name":"Catherine Diep","email":"cdiep@us.ibm.com","username":"cdiep"},"change_message_id":"f34bed0ec31f7e7dbad3487a009e10e688ecfe85","unresolved":false,"context_lines":[{"line_number":42,"context_line":"verification as well as debugging assistance. Upload of subunit data will not,"},{"line_number":43,"context_line":"for the time being, be required."},{"line_number":44,"context_line":""},{"line_number":45,"context_line":"We will also need to add to the RefStack internal database, at a minimum, a"},{"line_number":46,"context_line":"new field in the \"results\" table. This field would link a test result to the"},{"line_number":47,"context_line":"corresponding subunit data. This could be done by linking the relevant subunit"},{"line_number":48,"context_line":"data via its ID."}],"source_content_type":"text/x-rst","patch_set":4,"id":"1f1a1f67_a6f8347c","line":45,"range":{"start_line":45,"start_character":21,"end_line":45,"end_character":24},"updated":"2017-07-20 23:56:28.000000000","message":"At the last IRC meeting[1], we discussed about adding a key-value pair into the existing meta data table to link the subunit data to the RefStack database.  Just want to confirm that you are suggesting to add a new field instead of using the meta data table.  \n\n[1] http://eavesdrop.openstack.org/meetings/refstack/2017/refstack.2017-07-18-19.00.log.txt","commit_id":"07876f479c36fa87ba55b73b2082926c5c80719e"},{"author":{"_account_id":24728,"name":"megan guiney","email":"meganmguiney@gmail.com","username":"mguiney"},"change_message_id":"75fe36c7f2a0dcd5c4ec2e8aaaf3fb1bfe57cb4b","unresolved":false,"context_lines":[{"line_number":42,"context_line":"verification as well as debugging assistance. Upload of subunit data will not,"},{"line_number":43,"context_line":"for the time being, be required."},{"line_number":44,"context_line":""},{"line_number":45,"context_line":"We will also need to add to the RefStack internal database, at a minimum, a"},{"line_number":46,"context_line":"new field in the \"results\" table. This field would link a test result to the"},{"line_number":47,"context_line":"corresponding subunit data. This could be done by linking the relevant subunit"},{"line_number":48,"context_line":"data via its ID."}],"source_content_type":"text/x-rst","patch_set":4,"id":"1f1a1f67_e3873961","line":45,"in_reply_to":"1f1a1f67_a6f8347c","updated":"2017-07-21 02:25:28.000000000","message":"Done","commit_id":"07876f479c36fa87ba55b73b2082926c5c80719e"},{"author":{"_account_id":10526,"name":"Catherine Diep","email":"cdiep@us.ibm.com","username":"cdiep"},"change_message_id":"f34bed0ec31f7e7dbad3487a009e10e688ecfe85","unresolved":false,"context_lines":[{"line_number":43,"context_line":"for the time being, be required."},{"line_number":44,"context_line":""},{"line_number":45,"context_line":"We will also need to add to the RefStack internal database, at a minimum, a"},{"line_number":46,"context_line":"new field in the \"results\" table. This field would link a test result to the"},{"line_number":47,"context_line":"corresponding subunit data. This could be done by linking the relevant subunit"},{"line_number":48,"context_line":"data via its ID."},{"line_number":49,"context_line":""}],"source_content_type":"text/x-rst","patch_set":4,"id":"1f1a1f67_66f21c9d","line":46,"range":{"start_line":46,"start_character":13,"end_line":46,"end_character":26},"updated":"2017-07-20 23:56:28.000000000","message":"If the decision is to add a new field, then the new field should be added to the RefStack \"test\" table, not the \"results\" table.","commit_id":"07876f479c36fa87ba55b73b2082926c5c80719e"},{"author":{"_account_id":24728,"name":"megan guiney","email":"meganmguiney@gmail.com","username":"mguiney"},"change_message_id":"75fe36c7f2a0dcd5c4ec2e8aaaf3fb1bfe57cb4b","unresolved":false,"context_lines":[{"line_number":43,"context_line":"for the time being, be required."},{"line_number":44,"context_line":""},{"line_number":45,"context_line":"We will also need to add to the RefStack internal database, at a minimum, a"},{"line_number":46,"context_line":"new field in the \"results\" table. This field would link a test result to the"},{"line_number":47,"context_line":"corresponding subunit data. This could be done by linking the relevant subunit"},{"line_number":48,"context_line":"data via its ID."},{"line_number":49,"context_line":""}],"source_content_type":"text/x-rst","patch_set":4,"id":"1f1a1f67_a381414d","line":46,"in_reply_to":"1f1a1f67_66f21c9d","updated":"2017-07-21 02:25:28.000000000","message":"Done","commit_id":"07876f479c36fa87ba55b73b2082926c5c80719e"},{"author":{"_account_id":10526,"name":"Catherine Diep","email":"cdiep@us.ibm.com","username":"cdiep"},"change_message_id":"f34bed0ec31f7e7dbad3487a009e10e688ecfe85","unresolved":false,"context_lines":[{"line_number":48,"context_line":"data via its ID."},{"line_number":49,"context_line":""},{"line_number":50,"context_line":"After discussing a number of data storage methods at the 7/18/17 RefStack"},{"line_number":51,"context_line":"meeting[3], we settled upon saving the newly usable subunit data in a new"},{"line_number":52,"context_line":"database. This could be done using the existing subunit2sql toolkit[4]. This"},{"line_number":53,"context_line":"option would do a lot of the heavy lifting for us, as far as data import goes,"},{"line_number":54,"context_line":"as well as keeping the storage method of test data consistent across the board,"},{"line_number":55,"context_line":"but keeping the data in a separate database would carry with it some"}],"source_content_type":"text/x-rst","patch_set":4,"id":"1f1a1f67_2661c43a","line":52,"range":{"start_line":51,"start_character":70,"end_line":52,"end_character":8},"updated":"2017-07-20 23:56:28.000000000","message":"I know this is the decision made in the meeting.  But, could you investigate whether the new tables created by the subunit2sql can be created in the existing RefStack database?  If so, this may be a better option.","commit_id":"07876f479c36fa87ba55b73b2082926c5c80719e"},{"author":{"_account_id":5196,"name":"Matthew Treinish","email":"mtreinish@kortar.org","username":"treinish"},"change_message_id":"92bde0b85c850de3be2922ee64587eae7c7d4a9a","unresolved":false,"context_lines":[{"line_number":48,"context_line":"data via its ID."},{"line_number":49,"context_line":""},{"line_number":50,"context_line":"After discussing a number of data storage methods at the 7/18/17 RefStack"},{"line_number":51,"context_line":"meeting[3], we settled upon saving the newly usable subunit data in a new"},{"line_number":52,"context_line":"database. This could be done using the existing subunit2sql toolkit[4]. This"},{"line_number":53,"context_line":"option would do a lot of the heavy lifting for us, as far as data import goes,"},{"line_number":54,"context_line":"as well as keeping the storage method of test data consistent across the board,"},{"line_number":55,"context_line":"but keeping the data in a separate database would carry with it some"}],"source_content_type":"text/x-rst","patch_set":4,"id":"1f1a1f67_d60879da","line":52,"range":{"start_line":51,"start_character":70,"end_line":52,"end_character":8},"in_reply_to":"1f1a1f67_2661c43a","updated":"2017-07-21 02:52:26.000000000","message":"As long as there is no overlap in table names (which if they both use alembic there will be) you could just point the subunit2sql-db-manage command at the same database. But you can\u0027t have 2 alembic managed schemas in the same db because the version table will collide.","commit_id":"07876f479c36fa87ba55b73b2082926c5c80719e"},{"author":{"_account_id":7822,"name":"Chris Hoge (not involved in upstream development)","email":"chris@hogepodge.com","username":"hogepodge"},"change_message_id":"dfe3ec4e66b3d02ca473e3951315cbab8f4db84f","unresolved":false,"context_lines":[{"line_number":53,"context_line":"option would do a lot of the heavy lifting for us, as far as data import goes,"},{"line_number":54,"context_line":"as well as keeping the storage method of test data consistent across the board,"},{"line_number":55,"context_line":"but keeping the data in a separate database would carry with it some"},{"line_number":56,"context_line":"operational overhead. For the time being, subunit data will be linked to the"},{"line_number":57,"context_line":"corresponding test results via a key value pair in the meta data table that"},{"line_number":58,"context_line":"is an existing part of the RefStack database."},{"line_number":59,"context_line":""},{"line_number":60,"context_line":"Possible toolsets to use:"},{"line_number":61,"context_line":""}],"source_content_type":"text/x-rst","patch_set":4,"id":"1f1a1f67_409dc364","line":58,"range":{"start_line":56,"start_character":22,"end_line":58,"end_character":45},"updated":"2017-07-21 01:43:44.000000000","message":"As Catherine commented earlier, this should be reconciled with the design described above.","commit_id":"07876f479c36fa87ba55b73b2082926c5c80719e"},{"author":{"_account_id":24728,"name":"megan guiney","email":"meganmguiney@gmail.com","username":"mguiney"},"change_message_id":"75fe36c7f2a0dcd5c4ec2e8aaaf3fb1bfe57cb4b","unresolved":false,"context_lines":[{"line_number":55,"context_line":"but keeping the data in a separate database would carry with it some"},{"line_number":56,"context_line":"operational overhead. For the time being, subunit data will be linked to the"},{"line_number":57,"context_line":"corresponding test results via a key value pair in the meta data table that"},{"line_number":58,"context_line":"is an existing part of the RefStack database."},{"line_number":59,"context_line":""},{"line_number":60,"context_line":"Possible toolsets to use:"},{"line_number":61,"context_line":""}],"source_content_type":"text/x-rst","patch_set":4,"id":"1f1a1f67_a32b812e","line":58,"in_reply_to":"1f1a1f67_409dc364","updated":"2017-07-21 02:25:28.000000000","message":"Done","commit_id":"07876f479c36fa87ba55b73b2082926c5c80719e"},{"author":{"_account_id":5196,"name":"Matthew Treinish","email":"mtreinish@kortar.org","username":"treinish"},"change_message_id":"92bde0b85c850de3be2922ee64587eae7c7d4a9a","unresolved":false,"context_lines":[{"line_number":55,"context_line":"but keeping the data in a separate database would carry with it some"},{"line_number":56,"context_line":"operational overhead. For the time being, subunit data will be linked to the"},{"line_number":57,"context_line":"corresponding test results via a key value pair in the meta data table that"},{"line_number":58,"context_line":"is an existing part of the RefStack database."},{"line_number":59,"context_line":""},{"line_number":60,"context_line":"Possible toolsets to use:"},{"line_number":61,"context_line":""}],"source_content_type":"text/x-rst","patch_set":4,"id":"1f1a1f67_969c415e","line":58,"in_reply_to":"1f1a1f67_a32b812e","updated":"2017-07-21 02:52:26.000000000","message":"FWIW, all the data in the refstack db can likely easily be stored in the subunit2sql schema. You just use the run_metadata to track the owner information, and then the runs table counts successes already. It would also be very easy to port the json parsing code to just use the subunit2sql db api to create runs entries. This way you could avoid the dual db overhead. I can help you out with this if you want.\n\nAlthough I think it\u0027s totally fine to work towards that incrementally and weigh your options. You always have to take care when making changes to a production system.","commit_id":"07876f479c36fa87ba55b73b2082926c5c80719e"},{"author":{"_account_id":7822,"name":"Chris Hoge (not involved in upstream development)","email":"chris@hogepodge.com","username":"hogepodge"},"change_message_id":"dfe3ec4e66b3d02ca473e3951315cbab8f4db84f","unresolved":false,"context_lines":[{"line_number":57,"context_line":"corresponding test results via a key value pair in the meta data table that"},{"line_number":58,"context_line":"is an existing part of the RefStack database."},{"line_number":59,"context_line":""},{"line_number":60,"context_line":"Possible toolsets to use:"},{"line_number":61,"context_line":""},{"line_number":62,"context_line":"subunit2sql"},{"line_number":63,"context_line":""}],"source_content_type":"text/x-rst","patch_set":4,"id":"1f1a1f67_c082f300","line":60,"range":{"start_line":60,"start_character":0,"end_line":60,"end_character":10},"updated":"2017-07-21 01:43:44.000000000","message":"Drop possible, just call it Toolset to use:","commit_id":"07876f479c36fa87ba55b73b2082926c5c80719e"},{"author":{"_account_id":24728,"name":"megan guiney","email":"meganmguiney@gmail.com","username":"mguiney"},"change_message_id":"75fe36c7f2a0dcd5c4ec2e8aaaf3fb1bfe57cb4b","unresolved":false,"context_lines":[{"line_number":57,"context_line":"corresponding test results via a key value pair in the meta data table that"},{"line_number":58,"context_line":"is an existing part of the RefStack database."},{"line_number":59,"context_line":""},{"line_number":60,"context_line":"Possible toolsets to use:"},{"line_number":61,"context_line":""},{"line_number":62,"context_line":"subunit2sql"},{"line_number":63,"context_line":""}],"source_content_type":"text/x-rst","patch_set":4,"id":"1f1a1f67_c3267d69","line":60,"in_reply_to":"1f1a1f67_c082f300","updated":"2017-07-21 02:25:28.000000000","message":"Done","commit_id":"07876f479c36fa87ba55b73b2082926c5c80719e"},{"author":{"_account_id":10526,"name":"Catherine Diep","email":"cdiep@us.ibm.com","username":"cdiep"},"change_message_id":"f34bed0ec31f7e7dbad3487a009e10e688ecfe85","unresolved":false,"context_lines":[{"line_number":86,"context_line":"Data model impact"},{"line_number":87,"context_line":"-----------------"},{"line_number":88,"context_line":""},{"line_number":89,"context_line":"None"},{"line_number":90,"context_line":""},{"line_number":91,"context_line":"REST API impact"},{"line_number":92,"context_line":"---------------"}],"source_content_type":"text/x-rst","patch_set":4,"id":"1f1a1f67_66fc5c3e","line":89,"range":{"start_line":89,"start_character":0,"end_line":89,"end_character":4},"updated":"2017-07-20 23:56:28.000000000","message":"Although there will be no impact to RefStack data models.  But, I think it is very helpful to include the information about the tables that are created by the subunit2sql tool. Especially, you have done the research and have them already included in the last version.","commit_id":"07876f479c36fa87ba55b73b2082926c5c80719e"},{"author":{"_account_id":7822,"name":"Chris Hoge (not involved in upstream development)","email":"chris@hogepodge.com","username":"hogepodge"},"change_message_id":"dfe3ec4e66b3d02ca473e3951315cbab8f4db84f","unresolved":false,"context_lines":[{"line_number":86,"context_line":"Data model impact"},{"line_number":87,"context_line":"-----------------"},{"line_number":88,"context_line":""},{"line_number":89,"context_line":"None"},{"line_number":90,"context_line":""},{"line_number":91,"context_line":"REST API impact"},{"line_number":92,"context_line":"---------------"}],"source_content_type":"text/x-rst","patch_set":4,"id":"1f1a1f67_a09c975d","line":89,"range":{"start_line":89,"start_character":0,"end_line":89,"end_character":4},"in_reply_to":"1f1a1f67_66fc5c3e","updated":"2017-07-21 01:43:44.000000000","message":"We should mention the inclusion of new metadata information, particularly that the metadata is not (as far as I know) strongly checked. The implications of the model we\u0027ve chosen should be documented here.","commit_id":"07876f479c36fa87ba55b73b2082926c5c80719e"},{"author":{"_account_id":5196,"name":"Matthew Treinish","email":"mtreinish@kortar.org","username":"treinish"},"change_message_id":"92bde0b85c850de3be2922ee64587eae7c7d4a9a","unresolved":false,"context_lines":[{"line_number":86,"context_line":"Data model impact"},{"line_number":87,"context_line":"-----------------"},{"line_number":88,"context_line":""},{"line_number":89,"context_line":"None"},{"line_number":90,"context_line":""},{"line_number":91,"context_line":"REST API impact"},{"line_number":92,"context_line":"---------------"}],"source_content_type":"text/x-rst","patch_set":4,"id":"1f1a1f67_b6ca9d9e","line":89,"range":{"start_line":89,"start_character":0,"end_line":89,"end_character":4},"in_reply_to":"1f1a1f67_a09c975d","updated":"2017-07-21 02:52:26.000000000","message":"Are you looking for: https://docs.openstack.org/subunit2sql/latest/data_model.html","commit_id":"07876f479c36fa87ba55b73b2082926c5c80719e"},{"author":{"_account_id":7822,"name":"Chris Hoge (not involved in upstream development)","email":"chris@hogepodge.com","username":"hogepodge"},"change_message_id":"dfe3ec4e66b3d02ca473e3951315cbab8f4db84f","unresolved":false,"context_lines":[{"line_number":96,"context_line":""},{"line_number":97,"context_line":""},{"line_number":98,"context_line":""},{"line_number":99,"context_line":"Security impact"},{"line_number":100,"context_line":"---------------"},{"line_number":101,"context_line":"It has been suggested that uploading the subunit data for tests may expose"},{"line_number":102,"context_line":"private data. However, it was determined in the 6/27/2017 RefStack meeting[1]"}],"source_content_type":"text/x-rst","patch_set":4,"id":"1f1a1f67_c0fe736a","line":99,"range":{"start_line":99,"start_character":0,"end_line":99,"end_character":15},"updated":"2017-07-21 01:43:44.000000000","message":"My only addition to the security impact would be in a database injection attack could be performed with maliciously crafted subunit data. We should check with mtreinish about the security of the tool itself, and protections against sql injection.","commit_id":"07876f479c36fa87ba55b73b2082926c5c80719e"},{"author":{"_account_id":5196,"name":"Matthew Treinish","email":"mtreinish@kortar.org","username":"treinish"},"change_message_id":"92bde0b85c850de3be2922ee64587eae7c7d4a9a","unresolved":false,"context_lines":[{"line_number":96,"context_line":""},{"line_number":97,"context_line":""},{"line_number":98,"context_line":""},{"line_number":99,"context_line":"Security impact"},{"line_number":100,"context_line":"---------------"},{"line_number":101,"context_line":"It has been suggested that uploading the subunit data for tests may expose"},{"line_number":102,"context_line":"private data. However, it was determined in the 6/27/2017 RefStack meeting[1]"}],"source_content_type":"text/x-rst","patch_set":4,"id":"1f1a1f67_168dd17a","line":99,"range":{"start_line":99,"start_character":0,"end_line":99,"end_character":15},"in_reply_to":"1f1a1f67_c0fe736a","updated":"2017-07-21 02:52:26.000000000","message":"On the security front I haven\u0027t done a intense audit. But thinking critically about it for a second the majority of the data written to the database comes after processing the subunit data and doesn\u0027t directly write raw strings into the database. (with a couple exceptions) But, even when it does it\u0027s all done in python with sqlalchemy, which I think should escape everything properly. (but we\u0027d have to double check)\n\nAssuming you use the sql2subunit tool directly (or the methods in the module) the places where things are directly used from the subunit into the database are limited to 3 or 4 places. There are test_ids themselves which is done here:\n\nhttps://github.com/openstack-infra/subunit2sql/blob/master/subunit2sql/db/api.py#L112-L142\n\nthen the tags and attrs, which are written to the db with this function:\n\nhttps://github.com/openstack-infra/subunit2sql/blob/master/subunit2sql/db/api.py#L363-L389\n\nand read from the subunit here:\n\nhttps://github.com/openstack-infra/subunit2sql/blob/master/subunit2sql/read_subunit.py#L82-L100\n\nand then the attachments which contain the logfiles, stdout, stdin, and anything else. These are an opt in thing because they\u0027re just BLOB columns in mysql and it\u0027s just like storing a file in the DB. Those are written into the db here:\n\nhttps://github.com/openstack-infra/subunit2sql/blob/master/subunit2sql/db/api.py#L1686-L1714\n\nYou can see all of those places we\u0027re passing the python object to sqlachemy to do it\u0027s ORM magic, but I\u0027d hope it knows how to handle string inputs named: \"Robert\u0027); DROP TABLE Students;\" properly. But I haven\u0027t ever had to test that because this will be the first time that I\u0027m aware subunit2sql is using publically submitted data. The only other users I know of are generating test results data (not always subunit fwiw) from trusted sources. I\u0027ll dig into some more tomorrow.","commit_id":"07876f479c36fa87ba55b73b2082926c5c80719e"},{"author":{"_account_id":10526,"name":"Catherine Diep","email":"cdiep@us.ibm.com","username":"cdiep"},"change_message_id":"f34bed0ec31f7e7dbad3487a009e10e688ecfe85","unresolved":false,"context_lines":[{"line_number":139,"context_line":"We will also need to adjust refstack-client to be able to consume the new API"},{"line_number":140,"context_line":"feature while uploading subunit data."},{"line_number":141,"context_line":""},{"line_number":142,"context_line":"The most user-visible part of this change would be the creation of a flag"},{"line_number":143,"context_line":"option which enables the upload of the subunit data, which would modify the"},{"line_number":144,"context_line":"existing procedure by attaching the subunit data to an existing RefStack"},{"line_number":145,"context_line":"entry in the database."},{"line_number":146,"context_line":""}],"source_content_type":"text/x-rst","patch_set":4,"id":"1f1a1f67_e612ac62","line":143,"range":{"start_line":142,"start_character":67,"end_line":143,"end_character":6},"updated":"2017-07-20 23:56:28.000000000","message":"I am not familiar with subunit2sql.  I am not sure whether the subunit2sql tool should be used at the server-side (to create the REST API) or the client side ( refstack-client).","commit_id":"07876f479c36fa87ba55b73b2082926c5c80719e"},{"author":{"_account_id":24728,"name":"megan guiney","email":"meganmguiney@gmail.com","username":"mguiney"},"change_message_id":"75fe36c7f2a0dcd5c4ec2e8aaaf3fb1bfe57cb4b","unresolved":false,"context_lines":[{"line_number":140,"context_line":"feature while uploading subunit data."},{"line_number":141,"context_line":""},{"line_number":142,"context_line":"The most user-visible part of this change would be the creation of a flag"},{"line_number":143,"context_line":"option which enables the upload of the subunit data, which would modify the"},{"line_number":144,"context_line":"existing procedure by attaching the subunit data to an existing RefStack"},{"line_number":145,"context_line":"entry in the database."},{"line_number":146,"context_line":""}],"source_content_type":"text/x-rst","patch_set":4,"id":"1f1a1f67_23f9b14c","line":143,"in_reply_to":"1f1a1f67_40c4a340","updated":"2017-07-21 02:25:28.000000000","message":"yes, but would we not want to have the option to upload the subunit data to the RefStack site as well?","commit_id":"07876f479c36fa87ba55b73b2082926c5c80719e"},{"author":{"_account_id":7822,"name":"Chris Hoge (not involved in upstream development)","email":"chris@hogepodge.com","username":"hogepodge"},"change_message_id":"dfe3ec4e66b3d02ca473e3951315cbab8f4db84f","unresolved":false,"context_lines":[{"line_number":139,"context_line":"We will also need to adjust refstack-client to be able to consume the new API"},{"line_number":140,"context_line":"feature while uploading subunit data."},{"line_number":141,"context_line":""},{"line_number":142,"context_line":"The most user-visible part of this change would be the creation of a flag"},{"line_number":143,"context_line":"option which enables the upload of the subunit data, which would modify the"},{"line_number":144,"context_line":"existing procedure by attaching the subunit data to an existing RefStack"},{"line_number":145,"context_line":"entry in the database."},{"line_number":146,"context_line":""}],"source_content_type":"text/x-rst","patch_set":4,"id":"1f1a1f67_40c4a340","line":143,"range":{"start_line":142,"start_character":67,"end_line":143,"end_character":6},"in_reply_to":"1f1a1f67_e612ac62","updated":"2017-07-21 01:43:44.000000000","message":"Should be done server side. Send the subunit data and import with subunit to sql","commit_id":"07876f479c36fa87ba55b73b2082926c5c80719e"},{"author":{"_account_id":10526,"name":"Catherine Diep","email":"cdiep@us.ibm.com","username":"cdiep"},"change_message_id":"3d10b2a5d1ca448ad3615577d75fa546ae11d865","unresolved":false,"context_lines":[{"line_number":88,"context_line":"  just save the output into a file. It may, however, make subunit data upload"},{"line_number":89,"context_line":"  a bit less elegant, as well as being a deviation from the way test run data"},{"line_number":90,"context_line":"  is managed throughout the rest of RefStack."},{"line_number":91,"context_line":"* Save subunit data in the RefStack database by building in the functionality"},{"line_number":92,"context_line":"  required to save and manage it. Like the option listed above, this option"},{"line_number":93,"context_line":"  keeps test run data stored consistently across refstack, which would make"},{"line_number":94,"context_line":"  the changes to the API more consistent as well. It would also avoid the"}],"source_content_type":"text/x-rst","patch_set":6,"id":"ff346bd7_e99c560f","line":91,"range":{"start_line":91,"start_character":44,"end_line":91,"end_character":45},"updated":"2017-07-26 20:26:43.000000000","message":"and tables","commit_id":"5f56d3385dff8b6b92f37f5be5c56b9f54e93179"},{"author":{"_account_id":10526,"name":"Catherine Diep","email":"cdiep@us.ibm.com","username":"cdiep"},"change_message_id":"3d10b2a5d1ca448ad3615577d75fa546ae11d865","unresolved":false,"context_lines":[{"line_number":104,"context_line":"-----------------"},{"line_number":105,"context_line":""},{"line_number":106,"context_line":"We may be able to use the tables created by subunit2sql within the RefStack"},{"line_number":107,"context_line":"database. These tables (for reference) are mapped out below:"},{"line_number":108,"context_line":""},{"line_number":109,"context_line":"--------------------------------------"},{"line_number":110,"context_line":"|               tests                |"}],"source_content_type":"text/x-rst","patch_set":6,"id":"ff346bd7_30d99a7e","line":107,"range":{"start_line":107,"start_character":10,"end_line":107,"end_character":58},"updated":"2017-07-26 20:26:43.000000000","message":"++","commit_id":"5f56d3385dff8b6b92f37f5be5c56b9f54e93179"},{"author":{"_account_id":10526,"name":"Catherine Diep","email":"cdiep@us.ibm.com","username":"cdiep"},"change_message_id":"3d10b2a5d1ca448ad3615577d75fa546ae11d865","unresolved":false,"context_lines":[{"line_number":197,"context_line":"---------------"},{"line_number":198,"context_line":""},{"line_number":199,"context_line":"We will need to implement a new REST API for the  upload of subunit data"},{"line_number":200,"context_line":"from the subunit2sql database."},{"line_number":201,"context_line":""},{"line_number":202,"context_line":""},{"line_number":203,"context_line":"Security impact"}],"source_content_type":"text/x-rst","patch_set":6,"id":"ff346bd7_e9e7b6ab","line":200,"range":{"start_line":200,"start_character":0,"end_line":200,"end_character":30},"updated":"2017-07-26 20:26:43.000000000","message":"from the client and then use the subunit2sql toolkit to process and save the data into the database.","commit_id":"5f56d3385dff8b6b92f37f5be5c56b9f54e93179"},{"author":{"_account_id":7822,"name":"Chris Hoge (not involved in upstream development)","email":"chris@hogepodge.com","username":"hogepodge"},"change_message_id":"9dbb0af4e10f47466868807ceb337fb5b8c071f8","unresolved":false,"context_lines":[{"line_number":29,"context_line":"order to make your OpenStack instance appear to be more interoperable than"},{"line_number":30,"context_line":"it actually is. This too, was discussed at the time, and Defcore eventually"},{"line_number":31,"context_line":"arrived at the conclusion that, in the end, customers would become aware of"},{"line_number":32,"context_line":"and \"punish\" those who had used altered test results."},{"line_number":33,"context_line":""},{"line_number":34,"context_line":"Proposed change"},{"line_number":35,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"}],"source_content_type":"text/x-rst","patch_set":8,"id":"9f436f4f_1629cc99","line":32,"range":{"start_line":32,"start_character":0,"end_line":32,"end_character":12},"updated":"2017-08-14 23:47:19.000000000","message":"Not sure of this language. We could say something about the \"trust but verify\" philosophy. That is, we would trust vendor results but, users (and the Foundation) would always have the opportunity to verify independently. It is nice in theory, but is not necessarily something we do in practice. Additionally, many vendors seek support in troubleshooting their cloud or test configurations, and this feature allows them to securely send subunit data for review by Foundation staff.","commit_id":"0e9b41104d8ca115383c730eaec9a22747a87358"}]}