)]}'
{"/PATCHSET_LEVEL":[{"author":{"_account_id":7118,"name":"Ian Wienand","email":"iwienand@redhat.com","username":"iwienand"},"change_message_id":"ef729dc1ef8e8ac8bd35ca8de993cc91a52159ec","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":1,"id":"d231be7a_89ca5d3d","updated":"2024-10-28 22:31:11.000000000","message":"This is OK as is; I wonder if we could codify this (inline?)","commit_id":"433318d20b5cafab018b9f5b178c3528dc306d4c"}],"doc/source/sysadmin.rst":[{"author":{"_account_id":7118,"name":"Ian Wienand","email":"iwienand@redhat.com","username":"iwienand"},"change_message_id":"ef729dc1ef8e8ac8bd35ca8de993cc91a52159ec","unresolved":true,"context_lines":[{"line_number":270,"context_line":"we first disable the account::"},{"line_number":271,"context_line":""},{"line_number":272,"context_line":"  sudo usermod -s /usr/sbin/nologin borg-ethercalc02"},{"line_number":273,"context_line":"  sudo rm -rf /opt/backups/borg-ethercalc02/.ssh"},{"line_number":274,"context_line":""},{"line_number":275,"context_line":"Then we delete the borg backup directory::"},{"line_number":276,"context_line":""}],"source_content_type":"text/x-rst","patch_set":1,"id":"02e4d658_20c375a8","line":273,"updated":"2024-10-28 22:31:11.000000000","message":"Since you have to do this per-backup host which introduces room for skew ... I wonder if we could codify this?  I think it\u0027s maybe not too crazy ...\n\nIn https://opendev.org/opendev/system-config/src/branch/master/playbooks/roles/borg-backup-server/tasks/main.yaml we have the list of active users from the inventory set into `borg_users`.\n\nBecause the usernames are regular with the homedirs on disk, we could walk directories in `/opt/backups/*` and if the `dirname` doesn\u0027t exist in `borg_users` then use ansible to ensure the above (user disabled and no `.ssh`).  I feel like this is nice for consistency, and even though the user is disabled we\u0027re still having ansible periodically check on it just in case of any funny business.\n\nWhile it could also delete the backups; if it knows it\u0027s not in inventory, this could also then write out a `.retired` into `/opt/backups/\u003cuser\u003e` and we could add a quick change to https://opendev.org/opendev/system-config/src/branch/master/playbooks/roles/borg-backup-server/files/prune-borg-backups.sh to look for the `.retired` file when pruning, and if found change the prune flags to just keep one latest backup\n\n(as a saftey double-check maybe `.retried` should have a timestamp in it, and the prune can check we are, say, \u003e 1 month from the retired date before using the extreme prune flag).","commit_id":"433318d20b5cafab018b9f5b178c3528dc306d4c"},{"author":{"_account_id":7118,"name":"Ian Wienand","email":"iwienand@redhat.com","username":"iwienand"},"change_message_id":"f60797a928e51f33ed2bfadd990c1103846c54f0","unresolved":true,"context_lines":[{"line_number":270,"context_line":"we first disable the account::"},{"line_number":271,"context_line":""},{"line_number":272,"context_line":"  sudo usermod -s /usr/sbin/nologin borg-ethercalc02"},{"line_number":273,"context_line":"  sudo rm -rf /opt/backups/borg-ethercalc02/.ssh"},{"line_number":274,"context_line":""},{"line_number":275,"context_line":"Then we delete the borg backup directory::"},{"line_number":276,"context_line":""}],"source_content_type":"text/x-rst","patch_set":1,"id":"f9140e48_9c8aa3a0","line":273,"in_reply_to":"02e4d658_20c375a8","updated":"2024-10-28 22:39:49.000000000","message":"... two seconds after I clicked this it then struck me that if we add a host to the disabled list then it also doesn\u0027t appear in the list of current backup users from the loop\n\n```\nwith_inventory_hostnames: \u0027borg-backup:!disabled\u0027\n```\n\n... just because the host is temporarily disabled doesn\u0027t mean it\u0027s retired...\n\nI guess you could make a separate list without the `!disabled` and anything not in _that_ could be considered retired ... I agree complexity is rising though :/","commit_id":"433318d20b5cafab018b9f5b178c3528dc306d4c"},{"author":{"_account_id":4146,"name":"Clark Boylan","email":"cboylan@sapwetik.org","username":"cboylan"},"change_message_id":"8965fe5ab0b0ae99dca1fa52d85fd3f1a356aa1a","unresolved":true,"context_lines":[{"line_number":270,"context_line":"we first disable the account::"},{"line_number":271,"context_line":""},{"line_number":272,"context_line":"  sudo usermod -s /usr/sbin/nologin borg-ethercalc02"},{"line_number":273,"context_line":"  sudo rm -rf /opt/backups/borg-ethercalc02/.ssh"},{"line_number":274,"context_line":""},{"line_number":275,"context_line":"Then we delete the borg backup directory::"},{"line_number":276,"context_line":""}],"source_content_type":"text/x-rst","patch_set":1,"id":"e0659bb0_8b5fbf57","line":273,"in_reply_to":"730623d8_ef987cb5","updated":"2024-10-30 14:56:35.000000000","message":"I like this approach as a way to capture historical changes to the system while also preserving the behaviors I listed previously.","commit_id":"433318d20b5cafab018b9f5b178c3528dc306d4c"},{"author":{"_account_id":7118,"name":"Ian Wienand","email":"iwienand@redhat.com","username":"iwienand"},"change_message_id":"a96f53b378039064b049bd74fdd7f719796922a1","unresolved":true,"context_lines":[{"line_number":270,"context_line":"we first disable the account::"},{"line_number":271,"context_line":""},{"line_number":272,"context_line":"  sudo usermod -s /usr/sbin/nologin borg-ethercalc02"},{"line_number":273,"context_line":"  sudo rm -rf /opt/backups/borg-ethercalc02/.ssh"},{"line_number":274,"context_line":""},{"line_number":275,"context_line":"Then we delete the borg backup directory::"},{"line_number":276,"context_line":""}],"source_content_type":"text/x-rst","patch_set":1,"id":"730623d8_ef987cb5","line":273,"in_reply_to":"9cca52b6_3c009a16","updated":"2024-10-30 01:54:18.000000000","message":"I certainly don\u0027t disagree with documenting, and better to have some policy to refine from rather than nothing 😊  Not trying to block this.\n\n--\n\nI admit it\u0027s too scary having events triggered from a host not being in inventory, because there\u0027s always that possibility that it gets incorrectly triggered.\n\nI\u0027m being paranoid but I do just feel like we should have some history of people approving the data removal?  \n\nWhat if the backup hosts had access to two variables; a \"retired\" list and a \"purge\" list?  Ansible could look at the retired list and make sure ssh is disabled and flag the on-disk borg directory, and the prune script could be made to look for this flag to keep only latest (or indeed, specific archival flags could be specified and passed through).  \"purge\" would be complete removal and ansible would just ensure the borg dir doesn\u0027t exist.\n\nretired/purge could be different per-backup-host.  the lists would be manually managed through reviews, as would the prune step, but serve as a git history of backup status?","commit_id":"433318d20b5cafab018b9f5b178c3528dc306d4c"},{"author":{"_account_id":4146,"name":"Clark Boylan","email":"cboylan@sapwetik.org","username":"cboylan"},"change_message_id":"932cd49f450067d66b361f99bad4ad2ffcad8f18","unresolved":true,"context_lines":[{"line_number":270,"context_line":"we first disable the account::"},{"line_number":271,"context_line":""},{"line_number":272,"context_line":"  sudo usermod -s /usr/sbin/nologin borg-ethercalc02"},{"line_number":273,"context_line":"  sudo rm -rf /opt/backups/borg-ethercalc02/.ssh"},{"line_number":274,"context_line":""},{"line_number":275,"context_line":"Then we delete the borg backup directory::"},{"line_number":276,"context_line":""}],"source_content_type":"text/x-rst","patch_set":1,"id":"9cca52b6_3c009a16","line":273,"in_reply_to":"f9140e48_9c8aa3a0","updated":"2024-10-29 19:31:19.000000000","message":"A couple of other concerns I have with this approach:\n\n * I\u0027m not sure we want to drop backups for a service/server as soon as we remove it from production. Keeping the backups around for some time is probably a good idea.\n * We may actually want to keep backups for removed servers/services on the larger backup server for longer than we keep them on the smaller server and take advantage of the extra disk space.\n\nI agree we could probably find a way to make this smarter but doing the simple and safe thing here seems fine in the interim.","commit_id":"433318d20b5cafab018b9f5b178c3528dc306d4c"}]}