)]}'
{"/PATCHSET_LEVEL":[{"author":{"_account_id":12898,"name":"Tony Breeds","email":"tony@bakeyournoodle.com","username":"tonyb"},"change_message_id":"816fe169070168d4fccd7ccebd86dc0d72508e76","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":2,"id":"cc2b0d6a_7e302064","updated":"2025-03-11 19:47:58.000000000","message":"Feel free to ignore my -1 if I\u0027m confused","commit_id":"648d1f8acb789a5764c6012a85dd6ad2604ea356"},{"author":{"_account_id":7118,"name":"Ian Wienand","email":"iwienand@redhat.com","username":"iwienand"},"change_message_id":"288b162bebe50f97a5ba396c7e7d2893af286db9","unresolved":true,"context_lines":[],"source_content_type":"","patch_set":2,"id":"2ac75523_f9d212e2","in_reply_to":"cc2b0d6a_7e302064","updated":"2025-03-12 00:03:01.000000000","message":"definitely not confused\n\nwe have a bit of the same issue in testing -- https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_c1c/943999/2/check/system-config-run-base/c1c2ea4/bridge99.opendev.org/ansible/\n\nI think we keep 3 copies there -- it \"tees\" so zuul saves it as output, we save it to disk via stdout redirect and then also log it with ansible 😊\n\none difference is that the ansible log is timestamped per line.  we discussed this and it\u0027s not been too much of an issue as we put a start timestamp, and remote operations can be timestamped from the remote host.  \n\nthere is a \"timestamp\" stdout_callback but it\u0027s in community.general.  the effort required to deploy this consistently alongside ansible is probably not worth it.","commit_id":"648d1f8acb789a5764c6012a85dd6ad2604ea356"},{"author":{"_account_id":12898,"name":"Tony Breeds","email":"tony@bakeyournoodle.com","username":"tonyb"},"change_message_id":"b099186d2e14b4cc8f9fe5c0acce854ea45b336a","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":3,"id":"b09f2c39_6870f640","updated":"2025-03-12 14:11:55.000000000","message":"Thanks!","commit_id":"a51784a5139f1f7ee902651c3fb735428e3a4570"}],"playbooks/zuul/run-production-playbook.yaml":[{"author":{"_account_id":7118,"name":"Ian Wienand","email":"iwienand@redhat.com","username":"iwienand"},"change_message_id":"29e4c5af631f37f3e183970fd9b75340e937b585","unresolved":true,"context_lines":[{"line_number":18,"context_line":""},{"line_number":19,"context_line":"      - name: Run specified playbook on bastion host and redirect output"},{"line_number":20,"context_line":"        become: yes"},{"line_number":21,"context_line":"        shell: \u0027ansible-playbook -v -f {{ infra_prod_ansible_forks }} /home/zuul/src/opendev.org/opendev/system-config/playbooks/{{ playbook_name }}\u0027"},{"line_number":22,"context_line":"        register: _run"},{"line_number":23,"context_line":"        environment:"},{"line_number":24,"context_line":"          ANSIBLE_LOG_PATH: \u0027/var/log/ansible/{{ playbook_name }}.log"}],"source_content_type":"text/x-yaml","patch_set":1,"id":"4b381ba9_199f606f","line":21,"updated":"2025-03-11 04:51:08.000000000","message":"We don\u0027t want to do this because we rely on this not echoing out anything to avoid anything showing up in jobs.\n\nI guess this was _not_ capturing stderr in the prior code and that was OK, although I think unintentional\n\nOur concern is that something is coming out on stdout that should be saved","commit_id":"3b2d6308073d1c592ac0d159a34cf6a6b4b31da9"},{"author":{"_account_id":4146,"name":"Clark Boylan","email":"cboylan@sapwetik.org","username":"cboylan"},"change_message_id":"e8e3f94aacf52810c36e90e1159c6d85f33ba83a","unresolved":true,"context_lines":[{"line_number":18,"context_line":""},{"line_number":19,"context_line":"      - name: Run specified playbook on bastion host and redirect output"},{"line_number":20,"context_line":"        become: yes"},{"line_number":21,"context_line":"        shell: \u0027ansible-playbook -v -f {{ infra_prod_ansible_forks }} /home/zuul/src/opendev.org/opendev/system-config/playbooks/{{ playbook_name }} \u003e\u003e /var/log/ansible/{{ playbook_name }}}.log 2\u003e\u00261\u0027"},{"line_number":22,"context_line":"        register: _run"},{"line_number":23,"context_line":"        environment:"},{"line_number":24,"context_line":"          # Override the global config for zuul induced runs to not"}],"source_content_type":"text/x-yaml","patch_set":2,"id":"ba4d5b8c_a57680b7","line":21,"range":{"start_line":21,"start_character":194,"end_line":21,"end_character":198},"updated":"2025-03-11 15:03:46.000000000","message":"I guess we\u0027re saying this is safe because it would\u0027ve been captured by zuul anyway and published if it happened so putting it in the log file that we may publish is safe?\n\nWe only publish the logfile for one job: manage-projects. If we want to be extra careful this change could toggle that off for manage-projects and then we can toggle it back after reviewing the output file.","commit_id":"648d1f8acb789a5764c6012a85dd6ad2604ea356"},{"author":{"_account_id":12898,"name":"Tony Breeds","email":"tony@bakeyournoodle.com","username":"tonyb"},"change_message_id":"816fe169070168d4fccd7ccebd86dc0d72508e76","unresolved":true,"context_lines":[{"line_number":18,"context_line":""},{"line_number":19,"context_line":"      - name: Run specified playbook on bastion host and redirect output"},{"line_number":20,"context_line":"        become: yes"},{"line_number":21,"context_line":"        shell: \u0027ansible-playbook -v -f {{ infra_prod_ansible_forks }} /home/zuul/src/opendev.org/opendev/system-config/playbooks/{{ playbook_name }} \u003e\u003e /var/log/ansible/{{ playbook_name }}}.log 2\u003e\u00261\u0027"},{"line_number":22,"context_line":"        register: _run"},{"line_number":23,"context_line":"        environment:"},{"line_number":24,"context_line":"          # Override the global config for zuul induced runs to not"}],"source_content_type":"text/x-yaml","patch_set":2,"id":"e5964b6e_492cf49c","line":21,"range":{"start_line":21,"start_character":188,"end_line":21,"end_character":189},"updated":"2025-03-11 19:47:58.000000000","message":"Is this mismatched or is my syntax highlighting confused?","commit_id":"648d1f8acb789a5764c6012a85dd6ad2604ea356"},{"author":{"_account_id":4146,"name":"Clark Boylan","email":"cboylan@sapwetik.org","username":"cboylan"},"change_message_id":"58f5db11526dfbbca1f1a814293c8a452a2416bc","unresolved":false,"context_lines":[{"line_number":18,"context_line":""},{"line_number":19,"context_line":"      - name: Run specified playbook on bastion host and redirect output"},{"line_number":20,"context_line":"        become: yes"},{"line_number":21,"context_line":"        shell: \u0027ansible-playbook -v -f {{ infra_prod_ansible_forks }} /home/zuul/src/opendev.org/opendev/system-config/playbooks/{{ playbook_name }} \u003e\u003e /var/log/ansible/{{ playbook_name }}}.log 2\u003e\u00261\u0027"},{"line_number":22,"context_line":"        register: _run"},{"line_number":23,"context_line":"        environment:"},{"line_number":24,"context_line":"          # Override the global config for zuul induced runs to not"}],"source_content_type":"text/x-yaml","patch_set":2,"id":"dfd5bc84_f890c60a","line":21,"range":{"start_line":21,"start_character":194,"end_line":21,"end_character":198},"in_reply_to":"ba4d5b8c_a57680b7","updated":"2025-03-11 15:46:19.000000000","message":"Thank you for confirming. I pulled up the recent manage-projects run from yesterday and I think you can see the stderr there: https://zuul.opendev.org/t/openstack/build/789fbf5c0e6e4086886e20f947a5ac06/console#2/1/3/bridge01.opendev.org\n\nNote it is marked as stdout because zuul merges stderr and stdout in the shell tasks by default. But that is what escaped our stdout only redirect from ansible and gets written to stderr.","commit_id":"648d1f8acb789a5764c6012a85dd6ad2604ea356"},{"author":{"_account_id":5263,"name":"Jeremy Stanley","display_name":"fungi","email":"fungi@yuggoth.org","username":"fungi","status":"missing, presumed fed"},"change_message_id":"47b43d9116a459628edddceb68e997fc5433b5db","unresolved":false,"context_lines":[{"line_number":18,"context_line":""},{"line_number":19,"context_line":"      - name: Run specified playbook on bastion host and redirect output"},{"line_number":20,"context_line":"        become: yes"},{"line_number":21,"context_line":"        shell: \u0027ansible-playbook -v -f {{ infra_prod_ansible_forks }} /home/zuul/src/opendev.org/opendev/system-config/playbooks/{{ playbook_name }} \u003e\u003e /var/log/ansible/{{ playbook_name }}}.log 2\u003e\u00261\u0027"},{"line_number":22,"context_line":"        register: _run"},{"line_number":23,"context_line":"        environment:"},{"line_number":24,"context_line":"          # Override the global config for zuul induced runs to not"}],"source_content_type":"text/x-yaml","patch_set":2,"id":"0b9cd9d8_75abe3a0","line":21,"in_reply_to":"ba4d5b8c_a57680b7","updated":"2025-03-11 15:08:04.000000000","message":"Yes I think so, we haven\u0027t been redirecting stderr up to now, so we\u0027ve been potentially leaking it through zuul\u0027s ansible output for all jobs using this playbook anyway.","commit_id":"648d1f8acb789a5764c6012a85dd6ad2604ea356"},{"author":{"_account_id":7118,"name":"Ian Wienand","email":"iwienand@redhat.com","username":"iwienand"},"change_message_id":"b5a8319cd0d61971c7d878d5700934abff4c7330","unresolved":true,"context_lines":[{"line_number":18,"context_line":""},{"line_number":19,"context_line":"      - name: Run specified playbook on bastion host and redirect output"},{"line_number":20,"context_line":"        become: yes"},{"line_number":21,"context_line":"        shell: \u0027ansible-playbook -v -f {{ infra_prod_ansible_forks }} /home/zuul/src/opendev.org/opendev/system-config/playbooks/{{ playbook_name }} \u003e\u003e /var/log/ansible/{{ playbook_name }}}.log 2\u003e\u00261\u0027"},{"line_number":22,"context_line":"        register: _run"},{"line_number":23,"context_line":"        environment:"},{"line_number":24,"context_line":"          # Override the global config for zuul induced runs to not"}],"source_content_type":"text/x-yaml","patch_set":2,"id":"d03e08cb_eab6b7eb","line":21,"range":{"start_line":21,"start_character":194,"end_line":21,"end_character":198},"in_reply_to":"dfd5bc84_f890c60a","updated":"2025-03-11 22:11:39.000000000","message":"Ohhh, I had assumed it was mostly just forgotten to do the `2\u003e\u00261` and didn\u0027t consider it was a explicit thing to keep stuff out of logs when `infra_prod_playbook_collect_log` was on.  However, as you say, since this will go into the Zuul logs, if that is the intention it\u0027s not really working 😊  \n\nFor all the others that we currently encrypt and attach, I can\u0027t really imagine why we want stderr captured in the public zuul results.  it just seems a vector to accidentally put something in there.\n\ni feel like the most useful thing for debugging any issues is to have this all in the one file.","commit_id":"648d1f8acb789a5764c6012a85dd6ad2604ea356"},{"author":{"_account_id":7118,"name":"Ian Wienand","email":"iwienand@redhat.com","username":"iwienand"},"change_message_id":"288b162bebe50f97a5ba396c7e7d2893af286db9","unresolved":false,"context_lines":[{"line_number":18,"context_line":""},{"line_number":19,"context_line":"      - name: Run specified playbook on bastion host and redirect output"},{"line_number":20,"context_line":"        become: yes"},{"line_number":21,"context_line":"        shell: \u0027ansible-playbook -v -f {{ infra_prod_ansible_forks }} /home/zuul/src/opendev.org/opendev/system-config/playbooks/{{ playbook_name }} \u003e\u003e /var/log/ansible/{{ playbook_name }}}.log 2\u003e\u00261\u0027"},{"line_number":22,"context_line":"        register: _run"},{"line_number":23,"context_line":"        environment:"},{"line_number":24,"context_line":"          # Override the global config for zuul induced runs to not"}],"source_content_type":"text/x-yaml","patch_set":2,"id":"e5e2a9f1_de7db8b4","line":21,"range":{"start_line":21,"start_character":188,"end_line":21,"end_character":189},"in_reply_to":"e5964b6e_492cf49c","updated":"2025-03-12 00:03:01.000000000","message":"Done","commit_id":"648d1f8acb789a5764c6012a85dd6ad2604ea356"}]}