)]}'
{"specs/website-stats.rst":[{"author":{"_account_id":5263,"name":"Jeremy Stanley","display_name":"fungi","email":"fungi@yuggoth.org","username":"fungi","status":"missing, presumed fed"},"change_message_id":"a1d87e485f0c973fa2ccdaaf9b8894509670b682","unresolved":false,"context_lines":[{"line_number":7,"context_line":"  http://creativecommons.org/licenses/by/3.0/legalcode"},{"line_number":8,"context_line":""},{"line_number":9,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"},{"line_number":10,"context_line":"Website Stat Tracking"},{"line_number":11,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"},{"line_number":12,"context_line":""},{"line_number":13,"context_line":"https://storyboard.openstack.org/TODO"}],"source_content_type":"text/x-rst","patch_set":1,"id":"3fa7e38b_73008e04","line":10,"updated":"2020-02-22 05:44:00.000000000","message":"Not to be the word police, but I\u0027d like to find a way to avoid using the term \"tracking\" when we talk about this. I explicitly don\u0027t want us tracking users\u0027 or visitors\u0027 PII, but folks might misunderstand if we\u0027re not careful in how we describe it. Maybe we can say \"activity metrics\" or something along those lines?","commit_id":"b80ad31d9a7df1e6ad19fd9b57d2c8b94cc7bb9e"},{"author":{"_account_id":5263,"name":"Jeremy Stanley","display_name":"fungi","email":"fungi@yuggoth.org","username":"fungi","status":"missing, presumed fed"},"change_message_id":"a1d87e485f0c973fa2ccdaaf9b8894509670b682","unresolved":false,"context_lines":[{"line_number":32,"context_line":"that allows web admins to properly manage sites without leaking personal"},{"line_number":33,"context_line":"information."},{"line_number":34,"context_line":""},{"line_number":35,"context_line":"In particular we don\u0027t want to leak IP Addresses as these are considered PII"},{"line_number":36,"context_line":"and can be traced back to individuals. We also wants to avoid publishing"},{"line_number":37,"context_line":"referer information as this can be used to infer who users are as well."},{"line_number":38,"context_line":""}],"source_content_type":"text/x-rst","patch_set":1,"id":"3fa7e38b_93fbca14","line":35,"updated":"2020-02-22 05:44:00.000000000","message":"Raw networks can also be risky in this regard.","commit_id":"b80ad31d9a7df1e6ad19fd9b57d2c8b94cc7bb9e"},{"author":{"_account_id":5263,"name":"Jeremy Stanley","display_name":"fungi","email":"fungi@yuggoth.org","username":"fungi","status":"missing, presumed fed"},"change_message_id":"a1d87e485f0c973fa2ccdaaf9b8894509670b682","unresolved":false,"context_lines":[{"line_number":34,"context_line":""},{"line_number":35,"context_line":"In particular we don\u0027t want to leak IP Addresses as these are considered PII"},{"line_number":36,"context_line":"and can be traced back to individuals. We also wants to avoid publishing"},{"line_number":37,"context_line":"referer information as this can be used to infer who users are as well."},{"line_number":38,"context_line":""},{"line_number":39,"context_line":"Out of an abundance of caution we will avoid publishing Operating System,"},{"line_number":40,"context_line":"Web Browser, and google search terms as well."}],"source_content_type":"text/x-rst","patch_set":1,"id":"3fa7e38b_36f8c416","line":37,"updated":"2020-02-22 05:44:00.000000000","message":"May be good to call out examples like people following links from internal company defect trackers or knowledge bases.","commit_id":"b80ad31d9a7df1e6ad19fd9b57d2c8b94cc7bb9e"},{"author":{"_account_id":5263,"name":"Jeremy Stanley","display_name":"fungi","email":"fungi@yuggoth.org","username":"fungi","status":"missing, presumed fed"},"change_message_id":"a1d87e485f0c973fa2ccdaaf9b8894509670b682","unresolved":false,"context_lines":[{"line_number":37,"context_line":"referer information as this can be used to infer who users are as well."},{"line_number":38,"context_line":""},{"line_number":39,"context_line":"Out of an abundance of caution we will avoid publishing Operating System,"},{"line_number":40,"context_line":"Web Browser, and google search terms as well."},{"line_number":41,"context_line":""},{"line_number":42,"context_line":"Proposed Change"},{"line_number":43,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"}],"source_content_type":"text/x-rst","patch_set":1,"id":"3fa7e38b_56fb0013","line":40,"updated":"2020-02-22 05:44:00.000000000","message":"These seem somewhat less problematic but we can add them later if there\u0027s interest and we deem it safe.","commit_id":"b80ad31d9a7df1e6ad19fd9b57d2c8b94cc7bb9e"},{"author":{"_account_id":5263,"name":"Jeremy Stanley","display_name":"fungi","email":"fungi@yuggoth.org","username":"fungi","status":"missing, presumed fed"},"change_message_id":"41a8ea66cedce3f01b97d7ebe7236a4164fdfe4a","unresolved":false,"context_lines":[{"line_number":55,"context_line":""},{"line_number":56,"context_line":"Finally we can publish these reports at a known location under each site::"},{"line_number":57,"context_line":""},{"line_number":58,"context_line":"  https://example.website.org/goaccess.html"},{"line_number":59,"context_line":""},{"line_number":60,"context_line":"To do this we need a configuration file that excludes the panels we do not"},{"line_number":61,"context_line":"want::"}],"source_content_type":"text/x-rst","patch_set":1,"id":"1fa4df85_d2ce7a82","line":58,"updated":"2020-02-24 02:37:07.000000000","message":"Given the logs are stored on the local filesystem, the reports could be as well... the URL can be achieved simply by aliasing it to the local file location. Of course, that makes the reports themselves semi-precious and we could end up wanting to make backups of them, at which point yeah your Zuul publication job starts to sound better and better. If we were already to the point where we were using Zuul to CD all this with the server as a static node, there would be a lot less indirection involved.","commit_id":"b80ad31d9a7df1e6ad19fd9b57d2c8b94cc7bb9e"},{"author":{"_account_id":7118,"name":"Ian Wienand","email":"iwienand@redhat.com","username":"iwienand"},"change_message_id":"2ec6a58563a0e968d609818459b2d6881d304571","unresolved":false,"context_lines":[{"line_number":53,"context_line":"data beyond that of apache\u0027s normal log rotation. This should be fine as we"},{"line_number":54,"context_line":"will have stripped all sensitive information out of the reports."},{"line_number":55,"context_line":""},{"line_number":56,"context_line":"Finally we can publish these reports at a known location under each site::"},{"line_number":57,"context_line":""},{"line_number":58,"context_line":"  https://example.website.org/goaccess.html"},{"line_number":59,"context_line":""},{"line_number":60,"context_line":"To do this we need a configuration file that excludes the panels we do not"},{"line_number":61,"context_line":"want::"}],"source_content_type":"text/x-rst","patch_set":1,"id":"1fa4df85_127df26d","line":58,"range":{"start_line":56,"start_character":0,"end_line":58,"end_character":43},"updated":"2020-02-24 01:53:52.000000000","message":"Starting from the assumption that we\u0027re trying to concentrate on all our websites being served data stored on afs volumes, i don\u0027t think this is a good idea because it means that the server will have to somehow have keys to write to every site it serves.  this might be across opendev/openstack/starlingx/airship etc.\n\nwe could certainly export this to something like /afs/.openstack.org/stats.opendev.org/\u003chostname\u003e and that way the static host would only need one key for that directory and could just use a local cron job.\n\nbut better still IMO would be if this was driven from zuul via periodic jobs.  I\u0027ve been thinking that through a bit and proposed for the existing system\n\nhttps://review.opendev.org/709382 static: Framework for periodic 404 scraping job \n\nIn writing this, I realised it would be even better if Zuul could run ansible directly on the host.\n\nI don\u0027t think we can dynamically add a host to the executor\u0027s inventory with a private key from a secret file?  If we could, it seems like the executor could just log itself into static.opendev.org and run jobs, and then publish the output.\n\nThe other option seems to be to write out a separate inventory, and run a nested Ansible from the executor like we do for system-config testing.  the \"outer\" job can then upload as usual.\n\nthis way the static host never has to write anything to AFS, which seems ideal.\n\nbut, in short, I\u0027d like this spec to flesh out something like above so we have a agreed plan for publishing","commit_id":"b80ad31d9a7df1e6ad19fd9b57d2c8b94cc7bb9e"},{"author":{"_account_id":4146,"name":"Clark Boylan","email":"cboylan@sapwetik.org","username":"cboylan"},"change_message_id":"3ecb1d4acab61ba8d2d8c826bf35f6acef32b008","unresolved":false,"context_lines":[{"line_number":55,"context_line":""},{"line_number":56,"context_line":"Finally we can publish these reports at a known location under each site::"},{"line_number":57,"context_line":""},{"line_number":58,"context_line":"  https://example.website.org/goaccess.html"},{"line_number":59,"context_line":""},{"line_number":60,"context_line":"To do this we need a configuration file that excludes the panels we do not"},{"line_number":61,"context_line":"want::"}],"source_content_type":"text/x-rst","patch_set":1,"id":"1fa4df85_58aa9f85","line":58,"in_reply_to":"1fa4df85_d2ce7a82","updated":"2020-02-24 19:36:13.000000000","message":"I had imagined we would write the files outside of afs and then alias to them. That said I think the zuul job version of this would work fine as well. We\u0027d need to inflate all the gzipped files then feed them to goaccess. We wouldn\u0027t be able to keep persistent data beyond the log rotation period but that is probably fine (and doing persistent data has its own problems as noted below).\n\nI\u0027ll update the spec to assume zuul job generated reports.\n\nNote that you can use the zuul project\u0027s ssh key and add host to then run from executor in a nodeless job. I think it should work.","commit_id":"b80ad31d9a7df1e6ad19fd9b57d2c8b94cc7bb9e"},{"author":{"_account_id":5263,"name":"Jeremy Stanley","display_name":"fungi","email":"fungi@yuggoth.org","username":"fungi","status":"missing, presumed fed"},"change_message_id":"a1d87e485f0c973fa2ccdaaf9b8894509670b682","unresolved":false,"context_lines":[{"line_number":96,"context_line":"------------"},{"line_number":97,"context_line":""},{"line_number":98,"context_line":"We can use tracker that run in the browser like goatcounter. The downside to"},{"line_number":99,"context_line":"these tools is they don\u0027t have insight into requests that fail or are 404s."},{"line_number":100,"context_line":"If we were collecting a rich set of data they would provide much more info"},{"line_number":101,"context_line":"but because we\u0027ve decided that we do not want to collect that information"},{"line_number":102,"context_line":"the server logs are sufficient."}],"source_content_type":"text/x-rst","patch_set":1,"id":"3fa7e38b_f601cc04","line":99,"updated":"2020-02-22 05:44:00.000000000","message":"Technically not true, if you add them to a custom 404 page. Also goaccess still doesn\u0027t get us information on which local referrers are sending clients to nonexistent pages, which would be useful for cleaning up old/incorrect references, but we could always put together a custom solution for that specific case if we wanted.","commit_id":"b80ad31d9a7df1e6ad19fd9b57d2c8b94cc7bb9e"},{"author":{"_account_id":5263,"name":"Jeremy Stanley","display_name":"fungi","email":"fungi@yuggoth.org","username":"fungi","status":"missing, presumed fed"},"change_message_id":"a1d87e485f0c973fa2ccdaaf9b8894509670b682","unresolved":false,"context_lines":[{"line_number":147,"context_line":"Documentation"},{"line_number":148,"context_line":"-------------"},{"line_number":149,"context_line":""},{"line_number":150,"context_line":"We will need to document where the stats can be retrieved once available."},{"line_number":151,"context_line":"We should also document the choices we made around which data is collected."},{"line_number":152,"context_line":""},{"line_number":153,"context_line":"Security"}],"source_content_type":"text/x-rst","patch_set":1,"id":"3fa7e38b_1605880f","line":150,"updated":"2020-02-22 05:44:00.000000000","message":"Conveniently, we have a \"static.opendev.org\" vhost which could serve this if desired.","commit_id":"b80ad31d9a7df1e6ad19fd9b57d2c8b94cc7bb9e"},{"author":{"_account_id":13252,"name":"Dr. Jens Harbott","display_name":"Jens Harbott (frickler)","email":"frickler@offenerstapel.de","username":"jrosenboom"},"change_message_id":"a98251b4c22b42bbc1336bdd98f4b98ec4f98d86","unresolved":false,"context_lines":[{"line_number":2,"context_line":""},{"line_number":3,"context_line":"  Copyright 2020 OpenStack Foundation"},{"line_number":4,"context_line":""},{"line_number":5,"context_line":"  This work is licensed under a Creative Commons Attribution 3.0"},{"line_number":6,"context_line":"  Unported License."},{"line_number":7,"context_line":"  http://creativecommons.org/licenses/by/3.0/legalcode"},{"line_number":8,"context_line":""},{"line_number":9,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"},{"line_number":10,"context_line":"Website Activity Stats"}],"source_content_type":"text/x-rst","patch_set":2,"id":"1fa4df85_9bfed103","line":7,"range":{"start_line":5,"start_character":2,"end_line":7,"end_character":54},"updated":"2020-02-24 18:06:31.000000000","message":"Not directly related but is there a specific reason that we are not using the latest (4.0) version of the license? Where \"latest\" seems to be from 2013, so nothing too unstable.","commit_id":"0c602b507a7113d519640b6ee5907f399013eab4"},{"author":{"_account_id":5263,"name":"Jeremy Stanley","display_name":"fungi","email":"fungi@yuggoth.org","username":"fungi","status":"missing, presumed fed"},"change_message_id":"815d8ee78bb837d7d6d35f94f68c8ee03bbe8265","unresolved":false,"context_lines":[{"line_number":4,"context_line":""},{"line_number":5,"context_line":"  This work is licensed under a Creative Commons Attribution 3.0"},{"line_number":6,"context_line":"  Unported License."},{"line_number":7,"context_line":"  http://creativecommons.org/licenses/by/3.0/legalcode"},{"line_number":8,"context_line":""},{"line_number":9,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"},{"line_number":10,"context_line":"Website Activity Stats"}],"source_content_type":"text/x-rst","patch_set":2,"id":"1fa4df85_5e3a87e7","line":7,"updated":"2020-02-24 19:50:44.000000000","message":"Pretty sure it\u0027s just what was contemporary with when we created our template, which is in turn taken from a dusty old OSF board resolution allowing OpenStack documentation to use CC-BY-3.0, but... it\u0027s been many years now and I\u0027m not inclined to go dig through minutes of every board meeting to find out. ;)\n\nI think we should be able to update this and/or our template if we care.","commit_id":"0c602b507a7113d519640b6ee5907f399013eab4"},{"author":{"_account_id":4146,"name":"Clark Boylan","email":"cboylan@sapwetik.org","username":"cboylan"},"change_message_id":"3ecb1d4acab61ba8d2d8c826bf35f6acef32b008","unresolved":false,"context_lines":[{"line_number":2,"context_line":""},{"line_number":3,"context_line":"  Copyright 2020 OpenStack Foundation"},{"line_number":4,"context_line":""},{"line_number":5,"context_line":"  This work is licensed under a Creative Commons Attribution 3.0"},{"line_number":6,"context_line":"  Unported License."},{"line_number":7,"context_line":"  http://creativecommons.org/licenses/by/3.0/legalcode"},{"line_number":8,"context_line":""},{"line_number":9,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"},{"line_number":10,"context_line":"Website Activity Stats"}],"source_content_type":"text/x-rst","patch_set":2,"id":"1fa4df85_feefb3ad","line":7,"range":{"start_line":5,"start_character":2,"end_line":7,"end_character":54},"in_reply_to":"1fa4df85_9bfed103","updated":"2020-02-24 19:36:13.000000000","message":"Not that I am aware of. I\u0027ve just been copying from the template.rst file. We can ask around and see if anyone knows.","commit_id":"0c602b507a7113d519640b6ee5907f399013eab4"},{"author":{"_account_id":13252,"name":"Dr. Jens Harbott","display_name":"Jens Harbott (frickler)","email":"frickler@offenerstapel.de","username":"jrosenboom"},"change_message_id":"a98251b4c22b42bbc1336bdd98f4b98ec4f98d86","unresolved":false,"context_lines":[{"line_number":33,"context_line":"information."},{"line_number":34,"context_line":""},{"line_number":35,"context_line":"In particular we don\u0027t want to leak IP Addresses or subnets as IPs are"},{"line_number":36,"context_line":"considered PII and without significant traffic subnets typicaly identify"},{"line_number":37,"context_line":"specific users. We also wants to avoid publishing referer information as"},{"line_number":38,"context_line":"this can be used to infer who users are as well. This can happen if users"},{"line_number":39,"context_line":"follow links from internal company wikis, bug trackers or code hosting"}],"source_content_type":"text/x-rst","patch_set":2,"id":"1fa4df85_58b7ff87","line":36,"range":{"start_line":36,"start_character":55,"end_line":36,"end_character":63},"updated":"2020-02-24 18:06:31.000000000","message":"typically","commit_id":"0c602b507a7113d519640b6ee5907f399013eab4"},{"author":{"_account_id":13252,"name":"Dr. Jens Harbott","display_name":"Jens Harbott (frickler)","email":"frickler@offenerstapel.de","username":"jrosenboom"},"change_message_id":"a98251b4c22b42bbc1336bdd98f4b98ec4f98d86","unresolved":false,"context_lines":[{"line_number":34,"context_line":""},{"line_number":35,"context_line":"In particular we don\u0027t want to leak IP Addresses or subnets as IPs are"},{"line_number":36,"context_line":"considered PII and without significant traffic subnets typicaly identify"},{"line_number":37,"context_line":"specific users. We also wants to avoid publishing referer information as"},{"line_number":38,"context_line":"this can be used to infer who users are as well. This can happen if users"},{"line_number":39,"context_line":"follow links from internal company wikis, bug trackers or code hosting"},{"line_number":40,"context_line":"systems."}],"source_content_type":"text/x-rst","patch_set":2,"id":"1fa4df85_18b18790","line":37,"range":{"start_line":37,"start_character":24,"end_line":37,"end_character":29},"updated":"2020-02-24 18:06:31.000000000","message":"want","commit_id":"0c602b507a7113d519640b6ee5907f399013eab4"},{"author":{"_account_id":13252,"name":"Dr. Jens Harbott","display_name":"Jens Harbott (frickler)","email":"frickler@offenerstapel.de","username":"jrosenboom"},"change_message_id":"a98251b4c22b42bbc1336bdd98f4b98ec4f98d86","unresolved":false,"context_lines":[{"line_number":103,"context_line":"------------"},{"line_number":104,"context_line":""},{"line_number":105,"context_line":"We can use tracker that run in the browser like goatcounter. One downside"},{"line_number":106,"context_line":"to this approach is that we would need to run custom 404 pages into order"},{"line_number":107,"context_line":"to collect data on 404s. This is more complicated than the web server logs"},{"line_number":108,"context_line":"appraoch. One upside to this approach is that we could track referrers to"},{"line_number":109,"context_line":"404s enabling us to more easily fix our own broken links."}],"source_content_type":"text/x-rst","patch_set":2,"id":"1fa4df85_784afb80","line":106,"range":{"start_line":106,"start_character":63,"end_line":106,"end_character":67},"updated":"2020-02-24 18:06:31.000000000","message":"in","commit_id":"0c602b507a7113d519640b6ee5907f399013eab4"},{"author":{"_account_id":13252,"name":"Dr. Jens Harbott","display_name":"Jens Harbott (frickler)","email":"frickler@offenerstapel.de","username":"jrosenboom"},"change_message_id":"a98251b4c22b42bbc1336bdd98f4b98ec4f98d86","unresolved":false,"context_lines":[{"line_number":105,"context_line":"We can use tracker that run in the browser like goatcounter. One downside"},{"line_number":106,"context_line":"to this approach is that we would need to run custom 404 pages into order"},{"line_number":107,"context_line":"to collect data on 404s. This is more complicated than the web server logs"},{"line_number":108,"context_line":"appraoch. One upside to this approach is that we could track referrers to"},{"line_number":109,"context_line":"404s enabling us to more easily fix our own broken links."},{"line_number":110,"context_line":""},{"line_number":111,"context_line":"If we were collecting a rich set of data they would provide much more info,"}],"source_content_type":"text/x-rst","patch_set":2,"id":"1fa4df85_385c834a","line":108,"range":{"start_line":108,"start_character":0,"end_line":108,"end_character":8},"updated":"2020-02-24 18:06:31.000000000","message":"approach","commit_id":"0c602b507a7113d519640b6ee5907f399013eab4"},{"author":{"_account_id":5263,"name":"Jeremy Stanley","display_name":"fungi","email":"fungi@yuggoth.org","username":"fungi","status":"missing, presumed fed"},"change_message_id":"c72c86f3b0f24b48339ce365687c56c70daa8ac6","unresolved":false,"context_lines":[{"line_number":10,"context_line":"Website Activity Stats"},{"line_number":11,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"},{"line_number":12,"context_line":""},{"line_number":13,"context_line":"https://storyboard.openstack.org/TODO"},{"line_number":14,"context_line":""},{"line_number":15,"context_line":"Basic website activity stats around which pages are hit most often, which"},{"line_number":16,"context_line":"pages are 404s, and total number of visitors aid in properly running a"}],"source_content_type":"text/x-rst","patch_set":3,"id":"1fa4df85_9185c05c","line":13,"updated":"2020-03-04 19:32:08.000000000","message":"Should probably update this with a real story number in a follow-up before approving.","commit_id":"f8c1dd508ee00165bc80d61f4a57e0f5de609e67"},{"author":{"_account_id":4146,"name":"Clark Boylan","email":"cboylan@sapwetik.org","username":"cboylan"},"change_message_id":"c1d778df361a8ac11208b42046a5e7148224de8e","unresolved":false,"context_lines":[{"line_number":10,"context_line":"Website Activity Stats"},{"line_number":11,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"},{"line_number":12,"context_line":""},{"line_number":13,"context_line":"https://storyboard.openstack.org/TODO"},{"line_number":14,"context_line":""},{"line_number":15,"context_line":"Basic website activity stats around which pages are hit most often, which"},{"line_number":16,"context_line":"pages are 404s, and total number of visitors aid in properly running a"}],"source_content_type":"text/x-rst","patch_set":3,"id":"1fa4df85_01540b20","line":13,"in_reply_to":"1fa4df85_9185c05c","updated":"2020-03-06 00:54:22.000000000","message":"Addressed in https://review.opendev.org/#/c/711569/1","commit_id":"f8c1dd508ee00165bc80d61f4a57e0f5de609e67"},{"author":{"_account_id":5263,"name":"Jeremy Stanley","display_name":"fungi","email":"fungi@yuggoth.org","username":"fungi","status":"missing, presumed fed"},"change_message_id":"c72c86f3b0f24b48339ce365687c56c70daa8ac6","unresolved":false,"context_lines":[{"line_number":66,"context_line":""},{"line_number":67,"context_line":"If publishing into AFS we would write them to a known location for each site::"},{"line_number":68,"context_line":""},{"line_number":69,"context_line":"  https://example.website.org/goaccess.html"},{"line_number":70,"context_line":""},{"line_number":71,"context_line":"To do this we need a configuration file that excludes the panels we do not"},{"line_number":72,"context_line":"want::"}],"source_content_type":"text/x-rst","patch_set":3,"id":"1fa4df85_b18a3c67","line":69,"updated":"2020-03-04 19:32:08.000000000","message":"I\u0027m fine considering the publication process an implementation detail. If we discover during development that we should change it for some reason, I\u0027m not particularly worried. For this spec I\u0027m mostly concerned with what we publish rather than where.","commit_id":"f8c1dd508ee00165bc80d61f4a57e0f5de609e67"},{"author":{"_account_id":1,"name":"James E. Blair","email":"jim@acmegating.com","username":"corvus"},"change_message_id":"b38fe8b5058f242cb62c86da90523bd248c5e9aa","unresolved":false,"context_lines":[{"line_number":66,"context_line":""},{"line_number":67,"context_line":"If publishing into AFS we would write them to a known location for each site::"},{"line_number":68,"context_line":""},{"line_number":69,"context_line":"  https://example.website.org/goaccess.html"},{"line_number":70,"context_line":""},{"line_number":71,"context_line":"To do this we need a configuration file that excludes the panels we do not"},{"line_number":72,"context_line":"want::"}],"source_content_type":"text/x-rst","patch_set":3,"id":"1fa4df85_f3d6f74f","line":69,"updated":"2020-02-25 19:23:14.000000000","message":"If that\u0027s an issue, we could make a new service principal for this with access to every location.\n\nRegardless, multiple jobs might be better just for site segregation anyway.","commit_id":"f8c1dd508ee00165bc80d61f4a57e0f5de609e67"},{"author":{"_account_id":7118,"name":"Ian Wienand","email":"iwienand@redhat.com","username":"iwienand"},"change_message_id":"2623ddf13d62fb8542ff76eddbd16f514a2df9eb","unresolved":false,"context_lines":[{"line_number":66,"context_line":""},{"line_number":67,"context_line":"If publishing into AFS we would write them to a known location for each site::"},{"line_number":68,"context_line":""},{"line_number":69,"context_line":"  https://example.website.org/goaccess.html"},{"line_number":70,"context_line":""},{"line_number":71,"context_line":"To do this we need a configuration file that excludes the panels we do not"},{"line_number":72,"context_line":"want::"}],"source_content_type":"text/x-rst","patch_set":3,"id":"1fa4df85_c4ab0a73","line":69,"updated":"2020-02-24 21:37:41.000000000","message":"i still think this is a bit difficult from one job as there might be different service principals covering the website directory in afs; so just having a webstats.opendev.org with a directory for each covered site is probably easier.  but that is a detail we can consider","commit_id":"f8c1dd508ee00165bc80d61f4a57e0f5de609e67"},{"author":{"_account_id":7118,"name":"Ian Wienand","email":"iwienand@redhat.com","username":"iwienand"},"change_message_id":"0415b3fb351c4efe58acbd76bf8d839c2fab7f9e","unresolved":false,"context_lines":[{"line_number":66,"context_line":""},{"line_number":67,"context_line":"If publishing into AFS we would write them to a known location for each site::"},{"line_number":68,"context_line":""},{"line_number":69,"context_line":"  https://example.website.org/goaccess.html"},{"line_number":70,"context_line":""},{"line_number":71,"context_line":"To do this we need a configuration file that excludes the panels we do not"},{"line_number":72,"context_line":"want::"}],"source_content_type":"text/x-rst","patch_set":3,"id":"1fa4df85_20ddee41","line":69,"in_reply_to":"1fa4df85_402b4a2f","updated":"2020-02-27 23:31:39.000000000","message":"++ i like that this automatically gets cleaned up.  it seems if we really needed a permalink, you could do have a fixed url that sends you to the latest job that ran via the api.\n\npersonally i wouldn\u0027t like to see it being written to local disk -- we just cleared out systems like that to make sure we\u0027re not keeping data in single-use locations like that so putting it back in seems not so great.","commit_id":"f8c1dd508ee00165bc80d61f4a57e0f5de609e67"},{"author":{"_account_id":4146,"name":"Clark Boylan","email":"cboylan@sapwetik.org","username":"cboylan"},"change_message_id":"ed6f6c554e5e54a17353631a9556296c629a713a","unresolved":false,"context_lines":[{"line_number":66,"context_line":""},{"line_number":67,"context_line":"If publishing into AFS we would write them to a known location for each site::"},{"line_number":68,"context_line":""},{"line_number":69,"context_line":"  https://example.website.org/goaccess.html"},{"line_number":70,"context_line":""},{"line_number":71,"context_line":"To do this we need a configuration file that excludes the panels we do not"},{"line_number":72,"context_line":"want::"}],"source_content_type":"text/x-rst","patch_set":3,"id":"1fa4df85_402b4a2f","line":69,"in_reply_to":"1fa4df85_f3d6f74f","updated":"2020-02-27 23:26:42.000000000","message":"No objection to multiple jobs or writing to a common location. We could also write to local disk and not afs then alias those files in from the vhosts.\n\nI think to start we should definitely just consider publishing to our zuul swift logs locations. The data will be discoverable there from the zuul builds dashboard. This is simple and allows us to get this working without too much effort on additional details.","commit_id":"f8c1dd508ee00165bc80d61f4a57e0f5de609e67"},{"author":{"_account_id":5263,"name":"Jeremy Stanley","display_name":"fungi","email":"fungi@yuggoth.org","username":"fungi","status":"missing, presumed fed"},"change_message_id":"c72c86f3b0f24b48339ce365687c56c70daa8ac6","unresolved":false,"context_lines":[{"line_number":106,"context_line":"to this approach is that we would need to run custom 404 pages in order"},{"line_number":107,"context_line":"to collect data on 404s. This is more complicated than the web server logs"},{"line_number":108,"context_line":"approach. One upside to this approach is that we could track referrers to"},{"line_number":109,"context_line":"404s enabling us to more easily fix our own broken links."},{"line_number":110,"context_line":""},{"line_number":111,"context_line":"If we were collecting a rich set of data they would provide much more info,"},{"line_number":112,"context_line":"but because we\u0027ve decided that we do not want to collect that information"}],"source_content_type":"text/x-rst","patch_set":3,"id":"1fa4df85_517bc879","line":109,"updated":"2020-03-04 19:32:08.000000000","message":"Technically we could get that through log analysis too, just maybe not using goaccess?","commit_id":"f8c1dd508ee00165bc80d61f4a57e0f5de609e67"},{"author":{"_account_id":5263,"name":"Jeremy Stanley","display_name":"fungi","email":"fungi@yuggoth.org","username":"fungi","status":"missing, presumed fed"},"change_message_id":"c72c86f3b0f24b48339ce365687c56c70daa8ac6","unresolved":false,"context_lines":[{"line_number":119,"context_line":"-----------"},{"line_number":120,"context_line":""},{"line_number":121,"context_line":"Primary assignee:"},{"line_number":122,"context_line":"  TBD"},{"line_number":123,"context_line":""},{"line_number":124,"context_line":"Gerrit Topic"},{"line_number":125,"context_line":"------------"}],"source_content_type":"text/x-rst","patch_set":3,"id":"1fa4df85_71804447","line":122,"updated":"2020-03-04 19:32:08.000000000","message":"Either update this in a follow-up before approving, or move it into the help-wanted section of the index.","commit_id":"f8c1dd508ee00165bc80d61f4a57e0f5de609e67"},{"author":{"_account_id":4146,"name":"Clark Boylan","email":"cboylan@sapwetik.org","username":"cboylan"},"change_message_id":"c1d778df361a8ac11208b42046a5e7148224de8e","unresolved":false,"context_lines":[{"line_number":119,"context_line":"-----------"},{"line_number":120,"context_line":""},{"line_number":121,"context_line":"Primary assignee:"},{"line_number":122,"context_line":"  TBD"},{"line_number":123,"context_line":""},{"line_number":124,"context_line":"Gerrit Topic"},{"line_number":125,"context_line":"------------"}],"source_content_type":"text/x-rst","patch_set":3,"id":"1fa4df85_c137d3fe","line":122,"in_reply_to":"1fa4df85_71804447","updated":"2020-03-06 00:54:22.000000000","message":"Addressed in https://review.opendev.org/#/c/711569/1","commit_id":"f8c1dd508ee00165bc80d61f4a57e0f5de609e67"},{"author":{"_account_id":5263,"name":"Jeremy Stanley","display_name":"fungi","email":"fungi@yuggoth.org","username":"fungi","status":"missing, presumed fed"},"change_message_id":"c72c86f3b0f24b48339ce365687c56c70daa8ac6","unresolved":false,"context_lines":[{"line_number":164,"context_line":"We could potentially leak sensitive client information unintentionally."},{"line_number":165,"context_line":"The example config file used above is intended to do its best to avoid that"},{"line_number":166,"context_line":"by explicitly disabling all available goaccess panels then enabling the few"},{"line_number":167,"context_line":"we know are safe."},{"line_number":168,"context_line":""},{"line_number":169,"context_line":"Testing"},{"line_number":170,"context_line":"-------"}],"source_content_type":"text/x-rst","patch_set":3,"id":"1fa4df85_1199d0c2","line":167,"updated":"2020-03-04 19:32:08.000000000","message":"This may also be the place to talk about the tradeoff between copying the logs off the systems where they\u0027re recorded vs having to install the analysis tooling on the Web servers directly.","commit_id":"f8c1dd508ee00165bc80d61f4a57e0f5de609e67"}]}