)]}'
{"swift/obj/server.py":[{"author":{"_account_id":15343,"name":"Tim Burke","email":"tburke@nvidia.com","username":"tburke"},"change_message_id":"e5bfbe1c2b88e908e3ec265ce93dfa0f5e237c66","unresolved":false,"context_lines":[{"line_number":344,"context_line":"                    {\u0027ip\u0027: ip, \u0027port\u0027: port, \u0027dev\u0027: contdevice})"},{"line_number":345,"context_line":"        data \u003d {\u0027op\u0027: op, \u0027account\u0027: account, \u0027container\u0027: container,"},{"line_number":346,"context_line":"                \u0027obj\u0027: obj, \u0027headers\u0027: headers_out,"},{"line_number":347,"context_line":"                \u0027update_ctime\u0027: time.time()}"},{"line_number":348,"context_line":"        if redirect_data:"},{"line_number":349,"context_line":"            self.logger.debug("},{"line_number":350,"context_line":"                \u0027Update to %(path)s redirected to %(redirect)s\u0027,"}],"source_content_type":"text/x-python","patch_set":1,"id":"ff570b3c_146a7e18","line":347,"updated":"2020-06-02 02:54:06.000000000","message":"We can\u0027t just go off of the x-timestamp value?\n\nI guess we have to worry about back-dated work like reconciler data movement... but how busy does that really tend to be?","commit_id":"572ddc01d3e96bb963da5960df743b0d9c0d4d96"},{"author":{"_account_id":597,"name":"Pete Zaitcev","email":"zaitcev@kotori.zaitcev.us","username":"zaitcev"},"change_message_id":"e5df0c4ce18edd54739d454a46f7e691854dd02d","unresolved":false,"context_lines":[{"line_number":344,"context_line":"                    {\u0027ip\u0027: ip, \u0027port\u0027: port, \u0027dev\u0027: contdevice})"},{"line_number":345,"context_line":"        data \u003d {\u0027op\u0027: op, \u0027account\u0027: account, \u0027container\u0027: container,"},{"line_number":346,"context_line":"                \u0027obj\u0027: obj, \u0027headers\u0027: headers_out,"},{"line_number":347,"context_line":"                \u0027update_ctime\u0027: time.time()}"},{"line_number":348,"context_line":"        if redirect_data:"},{"line_number":349,"context_line":"            self.logger.debug("},{"line_number":350,"context_line":"                \u0027Update to %(path)s redirected to %(redirect)s\u0027,"}],"source_content_type":"text/x-python","patch_set":1,"id":"ff570b3c_56aacc18","line":347,"in_reply_to":"ff570b3c_146a7e18","updated":"2020-06-11 01:04:04.000000000","message":"You guys know that gettimeofday() is not even a syscall on Linux, right? Apps such as desktops and browsers hit it super hard, because they time all events. So, I went and make up a few tests to see if time.time() was any good.\n\nResults were not good. It\u0027s basically equivalent to fp.read(1) off an in-memory file, such as \"/dev/urandom\". And it\u0027s the same on py27 and py37 (with py37 being generally slightly faster).\n\nI think now that Tim is right to be concerned, although this is only 1 more call per request. If we added a time.time() per every buffer, I\u0027d add a -1 right away.\n\nUsing X-timestamp may be more correct semantically, regardless of the performance.","commit_id":"572ddc01d3e96bb963da5960df743b0d9c0d4d96"},{"author":{"_account_id":15343,"name":"Tim Burke","email":"tburke@nvidia.com","username":"tburke"},"change_message_id":"3187c374757921bf4f997e5ed0e5954785abd661","unresolved":false,"context_lines":[{"line_number":344,"context_line":"                    {\u0027ip\u0027: ip, \u0027port\u0027: port, \u0027dev\u0027: contdevice})"},{"line_number":345,"context_line":"        data \u003d {\u0027op\u0027: op, \u0027account\u0027: account, \u0027container\u0027: container,"},{"line_number":346,"context_line":"                \u0027obj\u0027: obj, \u0027headers\u0027: headers_out,"},{"line_number":347,"context_line":"                \u0027update_ctime\u0027: time.time()}"},{"line_number":348,"context_line":"        if redirect_data:"},{"line_number":349,"context_line":"            self.logger.debug("},{"line_number":350,"context_line":"                \u0027Update to %(path)s redirected to %(redirect)s\u0027,"}],"source_content_type":"text/x-python","patch_set":1,"id":"ff570b3c_2aa239ae","line":347,"in_reply_to":"ff570b3c_56aacc18","updated":"2020-06-11 21:11:32.000000000","message":"OK, I see comparable results between the two on my laptop:\n\n $ python -m timeit -s \u0027import time\u0027 \u0027time.time()\u0027\n 5000000 loops, best of 5: 75 nsec per loop\n $ python -m timeit -s \u0027x\u003dopen(\"/dev/urandom\", \"rb\")\u0027 \u0027x.read(1)\u0027\n 5000000 loops, best of 5: 74.6 nsec per loop\n\nBut I feel like we\u0027re being penny-wise and pound-foolish by chasing sub-microsecond timings when we\u0027re already committed to writing something to disk. Even ignoring the pickling and tempfile renaming, we\u0027re talking milliseconds:\n\n $ python -m timeit -s \u0027import os; x\u003dopen(\"./test-file\", \"wb\")\u0027 \\\n     \u0027x.write(b\"x\"); x.flush(); os.fsync(x.fileno())\u0027\n 100 loops, best of 5: 3.8 msec per loop\n\nEven in a more resource-constrained environment like my raspberry pi, time.time() is pretty damn cheap:\n\n pi@tburke-pi:~ $ python -m timeit -s \u0027import time\u0027 \u0027time.time()\u0027\n 1000000 loops, best of 3: 0.824 usec per loop\n\nA decent bit cheaper than reading a random byte, in fact:\n\n pi@tburke-pi:~ $ python -m timeit -s \u0027x\u003dopen(\"/dev/urandom\", \"rb\")\u0027 \u0027x.read(1)\u0027\n 1000000 loops, best of 3: 1.94 usec per loop\n\nI think the main question ought to be whether the metric is useful and can be understood -- and in that context, the possibility of container-sync being enabled for a container with arbitrarily-old data *already in it* makes a pretty compelling case for using the current time.","commit_id":"572ddc01d3e96bb963da5960df743b0d9c0d4d96"}],"swift/obj/updater.py":[{"author":{"_account_id":15343,"name":"Tim Burke","email":"tburke@nvidia.com","username":"tburke"},"change_message_id":"c9a82304be91521ebb31cce43ba94766492779ea","unresolved":false,"context_lines":[{"line_number":387,"context_line":""},{"line_number":388,"context_line":"            if success:"},{"line_number":389,"context_line":"                self.stats.successes +\u003d 1"},{"line_number":390,"context_line":"                self.logger.increment(\u0027successes\u0027)"},{"line_number":391,"context_line":"                self.logger.debug(\u0027Update sent for %(obj)s %(path)s\u0027,"},{"line_number":392,"context_line":"                                  {\u0027obj\u0027: obj, \u0027path\u0027: update_path})"},{"line_number":393,"context_line":"                self.stats.unlinks +\u003d 1"}],"source_content_type":"text/x-python","patch_set":1,"id":"ff570b3c_afe3ad04","line":390,"updated":"2020-06-02 05:36:39.000000000","message":"Would it be better to emit the timing here? It seems a little strange to me that, as written, this emits multiple times per update...\n\nIt might also be good to emit for both successes and failures. Otherwise you risk having your metric stay low even as the situation is deteriorating, just because some containers are overloaded and can barely eat *any* updates. I\u0027m guessing we\u0027d want to distinguish them as lag.success and lag.failure.","commit_id":"572ddc01d3e96bb963da5960df743b0d9c0d4d96"},{"author":{"_account_id":1179,"name":"Clay Gerrard","email":"clay.gerrard@gmail.com","username":"clay-gerrard"},"change_message_id":"a81bc2111949f0a28136b9cc220623c272e6948b","unresolved":false,"context_lines":[{"line_number":478,"context_line":"                     \u0027port\u0027: node[\u0027port\u0027], \u0027device\u0027: node[\u0027device\u0027]})"},{"line_number":479,"context_line":"            else:"},{"line_number":480,"context_line":"                if update_ctime:"},{"line_number":481,"context_line":"                    self.logger.timing_since(\u0027lag\u0027, update_ctime)"},{"line_number":482,"context_line":"            return success, node[\u0027id\u0027], redirect"},{"line_number":483,"context_line":"        except (Exception, Timeout):"},{"line_number":484,"context_line":"            self.logger.exception(_(\u0027ERROR with remote server \u0027"}],"source_content_type":"text/x-python","patch_set":1,"id":"df33271e_7b50685e","line":481,"updated":"2020-03-28 13:15:05.000000000","message":"this could be really big or really small - sometimes days sometimes less than a minute.  Also this metric might be sparse - no data for a while, then 100/s.  Are we going to be able to make sense of that in a time series database?","commit_id":"572ddc01d3e96bb963da5960df743b0d9c0d4d96"},{"author":{"_account_id":13852,"name":"Romain LE DISEZ","email":"romain.le-disez@corp.ovh.com","username":"rledisez"},"change_message_id":"4ecdec89466d6bc876bca08f61496b32186d5677","unresolved":false,"context_lines":[{"line_number":478,"context_line":"                     \u0027port\u0027: node[\u0027port\u0027], \u0027device\u0027: node[\u0027device\u0027]})"},{"line_number":479,"context_line":"            else:"},{"line_number":480,"context_line":"                if update_ctime:"},{"line_number":481,"context_line":"                    self.logger.timing_since(\u0027lag\u0027, update_ctime)"},{"line_number":482,"context_line":"            return success, node[\u0027id\u0027], redirect"},{"line_number":483,"context_line":"        except (Exception, Timeout):"},{"line_number":484,"context_line":"            self.logger.exception(_(\u0027ERROR with remote server \u0027"}],"source_content_type":"text/x-python","patch_set":1,"id":"df33271e_b5112607","line":481,"in_reply_to":"df33271e_7b50685e","updated":"2020-03-31 19:30:15.000000000","message":"You\u0027re right that, by itself, it may not make much sense. I refrained myself from adding the (account,container) in the serie name because the cardinality would kill most timeseries platform.\n\nIt\u0027s far from perfect, but at least it can help the operator to notify if there is any issue (eg: watching the 90percentile or something like that)","commit_id":"572ddc01d3e96bb963da5960df743b0d9c0d4d96"},{"author":{"_account_id":15343,"name":"Tim Burke","email":"tburke@nvidia.com","username":"tburke"},"change_message_id":"e5bfbe1c2b88e908e3ec265ce93dfa0f5e237c66","unresolved":false,"context_lines":[{"line_number":478,"context_line":"                     \u0027port\u0027: node[\u0027port\u0027], \u0027device\u0027: node[\u0027device\u0027]})"},{"line_number":479,"context_line":"            else:"},{"line_number":480,"context_line":"                if update_ctime:"},{"line_number":481,"context_line":"                    self.logger.timing_since(\u0027lag\u0027, update_ctime)"},{"line_number":482,"context_line":"            return success, node[\u0027id\u0027], redirect"},{"line_number":483,"context_line":"        except (Exception, Timeout):"},{"line_number":484,"context_line":"            self.logger.exception(_(\u0027ERROR with remote server \u0027"}],"source_content_type":"text/x-python","patch_set":1,"id":"ff570b3c_8c9b5715","line":481,"in_reply_to":"df33271e_b5112607","updated":"2020-06-02 02:54:06.000000000","message":"Yeah, I\u0027m not *so* worried about it -- seems roughly on par with our proxy-server timings.","commit_id":"572ddc01d3e96bb963da5960df743b0d9c0d4d96"}]}