)]}'
{"specs/juno/on-demand-compute-update.rst":[{"author":{"_account_id":7166,"name":"Sylvain Bauza","email":"sbauza@redhat.com","username":"sbauza"},"change_message_id":"d01971d6834b27d8b301c838ff3f7f6ddd7c457d","unresolved":false,"context_lines":[{"line_number":28,"context_line":"updates a row in the DB once every 60 seconds (the default period for"},{"line_number":29,"context_line":"this update).  This is unnecessary and a scalability problem given that"},{"line_number":30,"context_line":"the status info is mostly static and doesn\u0027t change very often, mainly"},{"line_number":31,"context_line":"it changes when an instance is created or destroyed."},{"line_number":32,"context_line":""},{"line_number":33,"context_line":"Proposed change"},{"line_number":34,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"}],"source_content_type":"text/x-rst","patch_set":1,"id":"1ae5cdf2_8febf311","line":31,"updated":"2014-06-13 08:05:59.000000000","message":"The problem is also that you send the whole State, even if that has not changed.\n\nSo, to me, there are 2 tied problems :\n - updates are every 60 secs, even if an unexpected change occurred, so that requires to make use of RT claims until the next run\n - that\u0027s full updates, even if RT knows that its State haven\u0027t changed","commit_id":"295d82b06c9e62b9d2dff0702723ae1e2645cdfa"},{"author":{"_account_id":1981,"name":"Don Dugger","email":"n0ano@n0ano.com","username":"n0ano"},"change_message_id":"431270c1300a43a246b8c5254811b489a4370cef","unresolved":false,"context_lines":[{"line_number":28,"context_line":"updates a row in the DB once every 60 seconds (the default period for"},{"line_number":29,"context_line":"this update).  This is unnecessary and a scalability problem given that"},{"line_number":30,"context_line":"the status info is mostly static and doesn\u0027t change very often, mainly"},{"line_number":31,"context_line":"it changes when an instance is created or destroyed."},{"line_number":32,"context_line":""},{"line_number":33,"context_line":"Proposed change"},{"line_number":34,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"}],"source_content_type":"text/x-rst","patch_set":1,"id":"1ae5cdf2_7062fa3e","line":31,"in_reply_to":"1ae5cdf2_8febf311","updated":"2014-06-13 15:46:33.000000000","message":"Sending the entire state doesn\u0027t bother me that much.  The big performance hit is the DB update, not that we\u0027re update 5 vs. 10 fields.  I\u0027d prefer to address selective updates in a separate BP, if it becomes necessary.","commit_id":"295d82b06c9e62b9d2dff0702723ae1e2645cdfa"},{"author":{"_account_id":7166,"name":"Sylvain Bauza","email":"sbauza@redhat.com","username":"sbauza"},"change_message_id":"d01971d6834b27d8b301c838ff3f7f6ddd7c457d","unresolved":false,"context_lines":[{"line_number":36,"context_line":"Compute node only sends an update if its status changes.  On a periodic"},{"line_number":37,"context_line":"interval (using the current default period of 60 seconds) the compute"},{"line_number":38,"context_line":"node will compare its status with the status saved from the last update."},{"line_number":39,"context_line":"Only if the state has changed will the DB be updated."},{"line_number":40,"context_line":""},{"line_number":41,"context_line":"The advantage to this method is that it should significantly cut down"},{"line_number":42,"context_line":"on the number of DB updates while changing almost nothing about the way"}],"source_content_type":"text/x-rst","patch_set":1,"id":"1ae5cdf2_2fdadf77","line":39,"updated":"2014-06-13 08:05:59.000000000","message":"Don\u0027t forget to mention that the usage is also compared with Claims.","commit_id":"295d82b06c9e62b9d2dff0702723ae1e2645cdfa"},{"author":{"_account_id":1981,"name":"Don Dugger","email":"n0ano@n0ano.com","username":"n0ano"},"change_message_id":"431270c1300a43a246b8c5254811b489a4370cef","unresolved":false,"context_lines":[{"line_number":36,"context_line":"Compute node only sends an update if its status changes.  On a periodic"},{"line_number":37,"context_line":"interval (using the current default period of 60 seconds) the compute"},{"line_number":38,"context_line":"node will compare its status with the status saved from the last update."},{"line_number":39,"context_line":"Only if the state has changed will the DB be updated."},{"line_number":40,"context_line":""},{"line_number":41,"context_line":"The advantage to this method is that it should significantly cut down"},{"line_number":42,"context_line":"on the number of DB updates while changing almost nothing about the way"}],"source_content_type":"text/x-rst","patch_set":1,"id":"1ae5cdf2_3075d2f9","line":39,"in_reply_to":"1ae5cdf2_2fdadf77","updated":"2014-06-13 15:46:33.000000000","message":"Done.","commit_id":"295d82b06c9e62b9d2dff0702723ae1e2645cdfa"},{"author":{"_account_id":8690,"name":"Khanh-Toan TRAN","email":"khtoantran@gmail.com","username":"toan-tran"},"change_message_id":"47fb1e9e8447b823581443acbbeaeda8931853ee","unresolved":false,"context_lines":[{"line_number":44,"context_line":"seconds before they are updated but any change, for any reason, will"},{"line_number":45,"context_line":"ultimately be reported."},{"line_number":46,"context_line":""},{"line_number":47,"context_line":"One potential issue with this is a possible sensitivity concern, updates"},{"line_number":48,"context_line":"shouldn\u0027t be constantly sent if, for example, steady state system activity"},{"line_number":49,"context_line":"causes something like RAM usage to change slightly.  Some experiments will"},{"line_number":50,"context_line":"have to be run to decide if adding in a sensitivity control for certain"}],"source_content_type":"text/x-rst","patch_set":1,"id":"1ae5cdf2_bee275ff","line":47,"updated":"2014-06-06 14:52:49.000000000","message":"There is another consequence of this change: currently nova-conductor uses period update as notification of the compute_node \u0027s keep-alive message. So if you intend to send period message only if changing, we should need some keep-alive message. Otherwise failed nodes cannot be recognized until a VM arrive.\n\nMaybe that\u0027s not a problem since we have Retry. However, admin certainly needs to be able to check which hosts are down.","commit_id":"295d82b06c9e62b9d2dff0702723ae1e2645cdfa"},{"author":{"_account_id":8690,"name":"Khanh-Toan TRAN","email":"khtoantran@gmail.com","username":"toan-tran"},"change_message_id":"7a660fb1ec597a63c62ee6b06c226ac2e9eb150b","unresolved":false,"context_lines":[{"line_number":44,"context_line":"seconds before they are updated but any change, for any reason, will"},{"line_number":45,"context_line":"ultimately be reported."},{"line_number":46,"context_line":""},{"line_number":47,"context_line":"One potential issue with this is a possible sensitivity concern, updates"},{"line_number":48,"context_line":"shouldn\u0027t be constantly sent if, for example, steady state system activity"},{"line_number":49,"context_line":"causes something like RAM usage to change slightly.  Some experiments will"},{"line_number":50,"context_line":"have to be run to decide if adding in a sensitivity control for certain"}],"source_content_type":"text/x-rst","patch_set":1,"id":"1ae5cdf2_84480c89","line":47,"in_reply_to":"1ae5cdf2_bee275ff","updated":"2014-06-06 15:05:54.000000000","message":"My bad. This is done by other period_tasks. So not-updating-DB will not be a problem :)","commit_id":"295d82b06c9e62b9d2dff0702723ae1e2645cdfa"},{"author":{"_account_id":7166,"name":"Sylvain Bauza","email":"sbauza@redhat.com","username":"sbauza"},"change_message_id":"d01971d6834b27d8b301c838ff3f7f6ddd7c457d","unresolved":false,"context_lines":[{"line_number":49,"context_line":"causes something like RAM usage to change slightly.  Some experiments will"},{"line_number":50,"context_line":"have to be run to decide if adding in a sensitivity control for certain"},{"line_number":51,"context_line":"status metrics is needed.  This can be a follow on optimization, if it\u0027s"},{"line_number":52,"context_line":"necessary, since this design is no worse than the current mechanism."},{"line_number":53,"context_line":""},{"line_number":54,"context_line":"Alternatives"},{"line_number":55,"context_line":"------------"}],"source_content_type":"text/x-rst","patch_set":1,"id":"1ae5cdf2_4f53abe1","line":52,"updated":"2014-06-13 08:05:59.000000000","message":"We could indeed imagine the polling period for each resource to be determined by the operator, but that would be a bit tricky to manage, especially if we have heterogeneous hardware (could we consider same polling period for disks if one node is disk hot-swappable and the other not ?)","commit_id":"295d82b06c9e62b9d2dff0702723ae1e2645cdfa"},{"author":{"_account_id":7166,"name":"Sylvain Bauza","email":"sbauza@redhat.com","username":"sbauza"},"change_message_id":"1f036d94c9d5fcb9d3d51b473db804ab38790434","unresolved":false,"context_lines":[{"line_number":49,"context_line":"causes something like RAM usage to change slightly.  Some experiments will"},{"line_number":50,"context_line":"have to be run to decide if adding in a sensitivity control for certain"},{"line_number":51,"context_line":"status metrics is needed.  This can be a follow on optimization, if it\u0027s"},{"line_number":52,"context_line":"necessary, since this design is no worse than the current mechanism."},{"line_number":53,"context_line":""},{"line_number":54,"context_line":"Alternatives"},{"line_number":55,"context_line":"------------"}],"source_content_type":"text/x-rst","patch_set":1,"id":"1ae5cdf2_7dc8eaa8","line":52,"in_reply_to":"1ae5cdf2_106a9619","updated":"2014-06-16 07:48:25.000000000","message":"Fair enough.","commit_id":"295d82b06c9e62b9d2dff0702723ae1e2645cdfa"},{"author":{"_account_id":1981,"name":"Don Dugger","email":"n0ano@n0ano.com","username":"n0ano"},"change_message_id":"431270c1300a43a246b8c5254811b489a4370cef","unresolved":false,"context_lines":[{"line_number":49,"context_line":"causes something like RAM usage to change slightly.  Some experiments will"},{"line_number":50,"context_line":"have to be run to decide if adding in a sensitivity control for certain"},{"line_number":51,"context_line":"status metrics is needed.  This can be a follow on optimization, if it\u0027s"},{"line_number":52,"context_line":"necessary, since this design is no worse than the current mechanism."},{"line_number":53,"context_line":""},{"line_number":54,"context_line":"Alternatives"},{"line_number":55,"context_line":"------------"}],"source_content_type":"text/x-rst","patch_set":1,"id":"1ae5cdf2_106a9619","line":52,"in_reply_to":"1ae5cdf2_4f53abe1","updated":"2014-06-13 15:46:33.000000000","message":"I don\u0027t think there\u0027s a need to create variable polling periods.  As long as we make the base polling period fast enough for all use cases (and I think 60 seconds fits that) then only updating on changes will cut down needless updates while still updating changes in a timely fashion.","commit_id":"295d82b06c9e62b9d2dff0702723ae1e2645cdfa"},{"author":{"_account_id":7166,"name":"Sylvain Bauza","email":"sbauza@redhat.com","username":"sbauza"},"change_message_id":"d01971d6834b27d8b301c838ff3f7f6ddd7c457d","unresolved":false,"context_lines":[{"line_number":58,"context_line":"The update code would only be called at system start up, instance creation"},{"line_number":59,"context_line":"and instance deletion.  This would reduce the latency for status updates"},{"line_number":60,"context_line":"(the DB is modified as soon as the system state changes) but it suffers"},{"line_number":61,"context_line":"from some disadvantages:"},{"line_number":62,"context_line":""},{"line_number":63,"context_line":"1)  Finding all of the appropriate events to record a status change.  Are"},{"line_number":64,"context_line":"statup/creation/destruction the only places where the system state changes,"}],"source_content_type":"text/x-rst","patch_set":1,"id":"1ae5cdf2_8f0773c9","line":61,"updated":"2014-06-13 08:05:59.000000000","message":"it is alrealy done, and it\u0027s called Claims :D","commit_id":"295d82b06c9e62b9d2dff0702723ae1e2645cdfa"},{"author":{"_account_id":8690,"name":"Khanh-Toan TRAN","email":"khtoantran@gmail.com","username":"toan-tran"},"change_message_id":"47fb1e9e8447b823581443acbbeaeda8931853ee","unresolved":false,"context_lines":[{"line_number":67,"context_line":"2)  Future changes could add new events that change status and recognizing"},{"line_number":68,"context_line":"that an update is needed is an easy mistake to miss."},{"line_number":69,"context_line":""},{"line_number":70,"context_line":"3)  Status could change unknown to the nova code, image something like a"},{"line_number":71,"context_line":"hot plug add of memory."},{"line_number":72,"context_line":""},{"line_number":73,"context_line":"Data model impact"}],"source_content_type":"text/x-rst","patch_set":1,"id":"1ae5cdf2_7e3f0dd7","line":70,"updated":"2014-06-06 14:52:49.000000000","message":"image \u003d\u003e \"imagine\"","commit_id":"295d82b06c9e62b9d2dff0702723ae1e2645cdfa"},{"author":{"_account_id":1981,"name":"Don Dugger","email":"n0ano@n0ano.com","username":"n0ano"},"change_message_id":"431270c1300a43a246b8c5254811b489a4370cef","unresolved":false,"context_lines":[{"line_number":67,"context_line":"2)  Future changes could add new events that change status and recognizing"},{"line_number":68,"context_line":"that an update is needed is an easy mistake to miss."},{"line_number":69,"context_line":""},{"line_number":70,"context_line":"3)  Status could change unknown to the nova code, image something like a"},{"line_number":71,"context_line":"hot plug add of memory."},{"line_number":72,"context_line":""},{"line_number":73,"context_line":"Data model impact"}],"source_content_type":"text/x-rst","patch_set":1,"id":"1ae5cdf2_f06e0a2e","line":70,"in_reply_to":"1ae5cdf2_7e3f0dd7","updated":"2014-06-13 15:46:33.000000000","message":"Done.","commit_id":"295d82b06c9e62b9d2dff0702723ae1e2645cdfa"},{"author":{"_account_id":7166,"name":"Sylvain Bauza","email":"sbauza@redhat.com","username":"sbauza"},"change_message_id":"d01971d6834b27d8b301c838ff3f7f6ddd7c457d","unresolved":false,"context_lines":[{"line_number":73,"context_line":"Data model impact"},{"line_number":74,"context_line":"-----------------"},{"line_number":75,"context_line":""},{"line_number":76,"context_line":"None."},{"line_number":77,"context_line":""},{"line_number":78,"context_line":"REST API impact"},{"line_number":79,"context_line":"---------------"}],"source_content_type":"text/x-rst","patch_set":1,"id":"1ae5cdf2_ef10f701","line":76,"updated":"2014-06-13 08:05:59.000000000","message":"Uh, please describe more","commit_id":"295d82b06c9e62b9d2dff0702723ae1e2645cdfa"},{"author":{"_account_id":1981,"name":"Don Dugger","email":"n0ano@n0ano.com","username":"n0ano"},"change_message_id":"431270c1300a43a246b8c5254811b489a4370cef","unresolved":false,"context_lines":[{"line_number":73,"context_line":"Data model impact"},{"line_number":74,"context_line":"-----------------"},{"line_number":75,"context_line":""},{"line_number":76,"context_line":"None."},{"line_number":77,"context_line":""},{"line_number":78,"context_line":"REST API impact"},{"line_number":79,"context_line":"---------------"}],"source_content_type":"text/x-rst","patch_set":1,"id":"1ae5cdf2_d084aed4","line":76,"in_reply_to":"1ae5cdf2_ef10f701","updated":"2014-06-13 15:46:33.000000000","message":"Not sure what your concern is, this BP only changes when and update occurs, not what gets updated, but I changed this section to reflect that.","commit_id":"295d82b06c9e62b9d2dff0702723ae1e2645cdfa"},{"author":{"_account_id":7166,"name":"Sylvain Bauza","email":"sbauza@redhat.com","username":"sbauza"},"change_message_id":"d01971d6834b27d8b301c838ff3f7f6ddd7c457d","unresolved":false,"context_lines":[{"line_number":129,"context_line":"----------"},{"line_number":130,"context_line":""},{"line_number":131,"context_line":"Change should be fairly simple, change the current update code to only call"},{"line_number":132,"context_line":"the DB update if a \u0027state_modified\u0027 routine returns true.  The"},{"line_number":133,"context_line":"\u0027state_modified\u0027 routine returns false if the current state matches the"},{"line_number":134,"context_line":"last recorded state.  Otherwise, it saves the current state as the last"},{"line_number":135,"context_line":"recorded state and returns true."}],"source_content_type":"text/x-rst","patch_set":1,"id":"1ae5cdf2_cfdb3b4d","line":132,"updated":"2014-06-13 08:05:59.000000000","message":"If this routine looking at one resource by one, or for the whole State ? If latter, I don\u0027t see a dramatic decrease in calls, IMHO.","commit_id":"295d82b06c9e62b9d2dff0702723ae1e2645cdfa"},{"author":{"_account_id":7166,"name":"Sylvain Bauza","email":"sbauza@redhat.com","username":"sbauza"},"change_message_id":"1f036d94c9d5fcb9d3d51b473db804ab38790434","unresolved":false,"context_lines":[{"line_number":129,"context_line":"----------"},{"line_number":130,"context_line":""},{"line_number":131,"context_line":"Change should be fairly simple, change the current update code to only call"},{"line_number":132,"context_line":"the DB update if a \u0027state_modified\u0027 routine returns true.  The"},{"line_number":133,"context_line":"\u0027state_modified\u0027 routine returns false if the current state matches the"},{"line_number":134,"context_line":"last recorded state.  Otherwise, it saves the current state as the last"},{"line_number":135,"context_line":"recorded state and returns true."}],"source_content_type":"text/x-rst","patch_set":1,"id":"1ae5cdf2_1d67e6b8","line":132,"in_reply_to":"1ae5cdf2_b081e2c4","updated":"2014-06-16 07:48:25.000000000","message":"Thanks for the feedback, that could possibly be changed later on.\nMaybe it\u0027s worth mentioning it in the next patchset ?","commit_id":"295d82b06c9e62b9d2dff0702723ae1e2645cdfa"},{"author":{"_account_id":1981,"name":"Don Dugger","email":"n0ano@n0ano.com","username":"n0ano"},"change_message_id":"431270c1300a43a246b8c5254811b489a4370cef","unresolved":false,"context_lines":[{"line_number":129,"context_line":"----------"},{"line_number":130,"context_line":""},{"line_number":131,"context_line":"Change should be fairly simple, change the current update code to only call"},{"line_number":132,"context_line":"the DB update if a \u0027state_modified\u0027 routine returns true.  The"},{"line_number":133,"context_line":"\u0027state_modified\u0027 routine returns false if the current state matches the"},{"line_number":134,"context_line":"last recorded state.  Otherwise, it saves the current state as the last"},{"line_number":135,"context_line":"recorded state and returns true."}],"source_content_type":"text/x-rst","patch_set":1,"id":"1ae5cdf2_b081e2c4","line":132,"in_reply_to":"1ae5cdf2_cfdb3b4d","updated":"2014-06-13 15:46:33.000000000","message":"I postulated a `state_modifed\u0027 routine so that it would be easy to do something like checking specific fields but my initial tests indicate that a simple compare of the entire state is sufficient.  On a compute node with an active instance this test resulted in no updates to the DB after the instance was created.","commit_id":"295d82b06c9e62b9d2dff0702723ae1e2645cdfa"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"ba30f69fa2127b87213b8c0189ee4e0350a7ac30","unresolved":false,"context_lines":[{"line_number":14,"context_line":""},{"line_number":15,"context_line":"Currently, all compute nodes update status info in the DB on a periodic"},{"line_number":16,"context_line":"basis (the period is currently 60 seconds). Given that the status of"},{"line_number":17,"context_line":"the node only changes at specific points (manyline image"},{"line_number":18,"context_line":"creation/destruction) this leads to significant DB overhead on a large"},{"line_number":19,"context_line":"system. This BP changes the update mechanism to only update the DB when"},{"line_number":20,"context_line":"a node state changes, specifically at node startup, instance creation"}],"source_content_type":"text/x-rst","patch_set":2,"id":"1ae5cdf2_c5b2f705","line":17,"updated":"2014-06-16 14:53:44.000000000","message":"I\u0027m not sure what \u0027manyline image creation/destruction\u0027 means?","commit_id":"095919c3a9777c70be0b2cbb12010d5624b1de2a"},{"author":{"_account_id":1981,"name":"Don Dugger","email":"n0ano@n0ano.com","username":"n0ano"},"change_message_id":"260e9ffd70dce56139d620145bc68915ac8fb4df","unresolved":false,"context_lines":[{"line_number":14,"context_line":""},{"line_number":15,"context_line":"Currently, all compute nodes update status info in the DB on a periodic"},{"line_number":16,"context_line":"basis (the period is currently 60 seconds). Given that the status of"},{"line_number":17,"context_line":"the node only changes at specific points (manyline image"},{"line_number":18,"context_line":"creation/destruction) this leads to significant DB overhead on a large"},{"line_number":19,"context_line":"system. This BP changes the update mechanism to only update the DB when"},{"line_number":20,"context_line":"a node state changes, specifically at node startup, instance creation"}],"source_content_type":"text/x-rst","patch_set":2,"id":"1ae5cdf2_2072a9e3","line":17,"in_reply_to":"1ae5cdf2_c5b2f705","updated":"2014-06-16 16:03:25.000000000","message":"s/manyline/mainly","commit_id":"095919c3a9777c70be0b2cbb12010d5624b1de2a"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"ba30f69fa2127b87213b8c0189ee4e0350a7ac30","unresolved":false,"context_lines":[{"line_number":28,"context_line":"updates a row in the DB once every 60 seconds (the default period for"},{"line_number":29,"context_line":"this update).  This is unnecessary and a scalability problem given that"},{"line_number":30,"context_line":"the status info is mostly static and doesn\u0027t change very often, mainly"},{"line_number":31,"context_line":"it changes when an instance is created or destroyed."},{"line_number":32,"context_line":""},{"line_number":33,"context_line":"Proposed change"},{"line_number":34,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"}],"source_content_type":"text/x-rst","patch_set":2,"id":"1ae5cdf2_e55d337b","line":31,"updated":"2014-06-16 14:53:44.000000000","message":"I don\u0027t think create/destroy is the whole picture, I\u0027ve been looking at nova.compute.manager._instance_update lately due to some of the objects conversion in the compute manager, see where this is used:\n\nhttp://git.openstack.org/cgit/openstack/nova/tree/nova/compute/manager.py#n614\n\nIt\u0027s mostly called in the building an instance flow, but it\u0027s also called from the @reverts_task_state decorator, which is not only in the create/destroy flows (it\u0027s on most of the API calls).  For example, set_admin_password.  That shouldn\u0027t do anything to the state of the compute node, it\u0027s setting the root password in the instance.\n\nSo this statement really comes down to what happens on this call:\n\nrt.update_usage(context, instance_ref)","commit_id":"095919c3a9777c70be0b2cbb12010d5624b1de2a"},{"author":{"_account_id":1981,"name":"Don Dugger","email":"n0ano@n0ano.com","username":"n0ano"},"change_message_id":"260e9ffd70dce56139d620145bc68915ac8fb4df","unresolved":false,"context_lines":[{"line_number":28,"context_line":"updates a row in the DB once every 60 seconds (the default period for"},{"line_number":29,"context_line":"this update).  This is unnecessary and a scalability problem given that"},{"line_number":30,"context_line":"the status info is mostly static and doesn\u0027t change very often, mainly"},{"line_number":31,"context_line":"it changes when an instance is created or destroyed."},{"line_number":32,"context_line":""},{"line_number":33,"context_line":"Proposed change"},{"line_number":34,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"}],"source_content_type":"text/x-rst","patch_set":2,"id":"1ae5cdf2_22e1ccfe","line":31,"in_reply_to":"1ae5cdf2_e55d337b","updated":"2014-06-16 16:03:25.000000000","message":"I think my statement is still correct, the status mainly changes on create/delete.  Note I qualified with `mainly` just because there could (and are) other places where state changes but, again, these are infrequent events.","commit_id":"095919c3a9777c70be0b2cbb12010d5624b1de2a"},{"author":{"_account_id":8690,"name":"Khanh-Toan TRAN","email":"khtoantran@gmail.com","username":"toan-tran"},"change_message_id":"82a76cb1b33ee785b077b4b0cf85ecab6bd3837e","unresolved":false,"context_lines":[{"line_number":50,"context_line":"have to be run to decide if adding in a sensitivity control for certain"},{"line_number":51,"context_line":"status metrics is needed.  This can be a follow on optimization, if it\u0027s"},{"line_number":52,"context_line":"necessary, since this design is no worse than the current mechanism."},{"line_number":53,"context_line":""},{"line_number":54,"context_line":"Alternatives"},{"line_number":55,"context_line":"------------"},{"line_number":56,"context_line":""}],"source_content_type":"text/x-rst","patch_set":2,"id":"1ae5cdf2_69858c58","line":53,"updated":"2014-06-16 09:40:06.000000000","message":"Does this take into account metric data (e.g. cpu_monitor) also ? This data changes very frequently (at every period!).","commit_id":"095919c3a9777c70be0b2cbb12010d5624b1de2a"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"ba30f69fa2127b87213b8c0189ee4e0350a7ac30","unresolved":false,"context_lines":[{"line_number":51,"context_line":"status metrics is needed.  This can be a follow on optimization, if it\u0027s"},{"line_number":52,"context_line":"necessary, since this design is no worse than the current mechanism."},{"line_number":53,"context_line":""},{"line_number":54,"context_line":"Alternatives"},{"line_number":55,"context_line":"------------"},{"line_number":56,"context_line":""},{"line_number":57,"context_line":"Another idea would be to only update the DB at certain well defined events."}],"source_content_type":"text/x-rst","patch_set":2,"id":"1ae5cdf2_20f469ce","line":54,"updated":"2014-06-16 14:53:44.000000000","message":"During the no-db scheduler session at the Juno summit there was a lot of talk about rather than polling and the compute nodes sending information to the scheduler on intervals, we use notifications - is there any alternative work in progress right now in this space?  Thinking something that Devananda or Boris are working on?","commit_id":"095919c3a9777c70be0b2cbb12010d5624b1de2a"},{"author":{"_account_id":1981,"name":"Don Dugger","email":"n0ano@n0ano.com","username":"n0ano"},"change_message_id":"260e9ffd70dce56139d620145bc68915ac8fb4df","unresolved":false,"context_lines":[{"line_number":51,"context_line":"status metrics is needed.  This can be a follow on optimization, if it\u0027s"},{"line_number":52,"context_line":"necessary, since this design is no worse than the current mechanism."},{"line_number":53,"context_line":""},{"line_number":54,"context_line":"Alternatives"},{"line_number":55,"context_line":"------------"},{"line_number":56,"context_line":""},{"line_number":57,"context_line":"Another idea would be to only update the DB at certain well defined events."}],"source_content_type":"text/x-rst","patch_set":2,"id":"1ae5cdf2_3b7f38ad","line":54,"in_reply_to":"1ae5cdf2_20f469ce","updated":"2014-06-16 16:03:25.000000000","message":"I don\u0027t know of any work in this area (we\u0027ve even stopped work on no-db as it was determined that the no-db work was a premature optimization this isn\u0027t needed at this time).  If we do something with notifications we can revisit this code but I think this is a good thing to do for now.","commit_id":"095919c3a9777c70be0b2cbb12010d5624b1de2a"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"ba30f69fa2127b87213b8c0189ee4e0350a7ac30","unresolved":false,"context_lines":[{"line_number":67,"context_line":"2)  Future changes could add new events that change status and recognizing"},{"line_number":68,"context_line":"that an update is needed is an easy mistake to miss."},{"line_number":69,"context_line":""},{"line_number":70,"context_line":"3)  Status could change unknown to the nova code, imagine something like a"},{"line_number":71,"context_line":"hot plug add of memory."},{"line_number":72,"context_line":""},{"line_number":73,"context_line":"Data model impact"}],"source_content_type":"text/x-rst","patch_set":2,"id":"1ae5cdf2_45ab4775","line":70,"updated":"2014-06-16 14:53:44.000000000","message":"If we aren\u0027t handling out of band changes today, why would we care now?","commit_id":"095919c3a9777c70be0b2cbb12010d5624b1de2a"},{"author":{"_account_id":1981,"name":"Don Dugger","email":"n0ano@n0ano.com","username":"n0ano"},"change_message_id":"260e9ffd70dce56139d620145bc68915ac8fb4df","unresolved":false,"context_lines":[{"line_number":67,"context_line":"2)  Future changes could add new events that change status and recognizing"},{"line_number":68,"context_line":"that an update is needed is an easy mistake to miss."},{"line_number":69,"context_line":""},{"line_number":70,"context_line":"3)  Status could change unknown to the nova code, imagine something like a"},{"line_number":71,"context_line":"hot plug add of memory."},{"line_number":72,"context_line":""},{"line_number":73,"context_line":"Data model impact"}],"source_content_type":"text/x-rst","patch_set":2,"id":"1ae5cdf2_3f3803d4","line":70,"in_reply_to":"1ae5cdf2_45ab4775","updated":"2014-06-16 16:03:25.000000000","message":"We do handle this specific out of band change today with the current code.  Just pointing out that a notification based technique would not handle out of band changes anymore.","commit_id":"095919c3a9777c70be0b2cbb12010d5624b1de2a"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"ba30f69fa2127b87213b8c0189ee4e0350a7ac30","unresolved":false,"context_lines":[{"line_number":123,"context_line":"-----------"},{"line_number":124,"context_line":""},{"line_number":125,"context_line":"Primary assignee:"},{"line_number":126,"context_line":"  n0ano"},{"line_number":127,"context_line":""},{"line_number":128,"context_line":"Other contributors:"},{"line_number":129,"context_line":""}],"source_content_type":"text/x-rst","patch_set":2,"id":"1ae5cdf2_65e04397","line":126,"updated":"2014-06-16 14:53:44.000000000","message":"Please put your full name and email in addition to IRC nick.","commit_id":"095919c3a9777c70be0b2cbb12010d5624b1de2a"},{"author":{"_account_id":1981,"name":"Don Dugger","email":"n0ano@n0ano.com","username":"n0ano"},"change_message_id":"260e9ffd70dce56139d620145bc68915ac8fb4df","unresolved":false,"context_lines":[{"line_number":123,"context_line":"-----------"},{"line_number":124,"context_line":""},{"line_number":125,"context_line":"Primary assignee:"},{"line_number":126,"context_line":"  n0ano"},{"line_number":127,"context_line":""},{"line_number":128,"context_line":"Other contributors:"},{"line_number":129,"context_line":""}],"source_content_type":"text/x-rst","patch_set":2,"id":"1ae5cdf2_1f5b472f","line":126,"in_reply_to":"1ae5cdf2_65e04397","updated":"2014-06-16 16:03:25.000000000","message":"s/n0ano/Don Dugger \u003cdonald.d.dugger@intel.com\u003e","commit_id":"095919c3a9777c70be0b2cbb12010d5624b1de2a"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"ba30f69fa2127b87213b8c0189ee4e0350a7ac30","unresolved":false,"context_lines":[{"line_number":131,"context_line":"----------"},{"line_number":132,"context_line":""},{"line_number":133,"context_line":"Change should be fairly simple, change the current update code to only call"},{"line_number":134,"context_line":"the DB update if a \u0027state_modified\u0027 routine returns true.  The"},{"line_number":135,"context_line":"\u0027state_modified\u0027 routine returns false if the current state matches the"},{"line_number":136,"context_line":"last recorded state.  Otherwise, it saves the current state as the last"},{"line_number":137,"context_line":"recorded state and returns true."}],"source_content_type":"text/x-rst","patch_set":2,"id":"1ae5cdf2_05877fb7","line":134,"updated":"2014-06-16 14:53:44.000000000","message":"Can you be more specific about what \u0027state_modified\u0027 is going to entail?","commit_id":"095919c3a9777c70be0b2cbb12010d5624b1de2a"},{"author":{"_account_id":1981,"name":"Don Dugger","email":"n0ano@n0ano.com","username":"n0ano"},"change_message_id":"260e9ffd70dce56139d620145bc68915ac8fb4df","unresolved":false,"context_lines":[{"line_number":131,"context_line":"----------"},{"line_number":132,"context_line":""},{"line_number":133,"context_line":"Change should be fairly simple, change the current update code to only call"},{"line_number":134,"context_line":"the DB update if a \u0027state_modified\u0027 routine returns true.  The"},{"line_number":135,"context_line":"\u0027state_modified\u0027 routine returns false if the current state matches the"},{"line_number":136,"context_line":"last recorded state.  Otherwise, it saves the current state as the last"},{"line_number":137,"context_line":"recorded state and returns true."}],"source_content_type":"text/x-rst","patch_set":2,"id":"1ae5cdf2_ffa3fbfd","line":134,"in_reply_to":"1ae5cdf2_05877fb7","updated":"2014-06-16 16:03:25.000000000","message":"Will do.","commit_id":"095919c3a9777c70be0b2cbb12010d5624b1de2a"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"ba30f69fa2127b87213b8c0189ee4e0350a7ac30","unresolved":false,"context_lines":[{"line_number":133,"context_line":"Change should be fairly simple, change the current update code to only call"},{"line_number":134,"context_line":"the DB update if a \u0027state_modified\u0027 routine returns true.  The"},{"line_number":135,"context_line":"\u0027state_modified\u0027 routine returns false if the current state matches the"},{"line_number":136,"context_line":"last recorded state.  Otherwise, it saves the current state as the last"},{"line_number":137,"context_line":"recorded state and returns true."},{"line_number":138,"context_line":""},{"line_number":139,"context_line":""}],"source_content_type":"text/x-rst","patch_set":2,"id":"1ae5cdf2_c01e655f","line":136,"updated":"2014-06-16 14:53:44.000000000","message":"I\u0027m assuming the last recorded state is something stored in memory so you don\u0027t need to hit the DB to look that up (which would defeat the purpose of the spec).\n\nIs there anything to consider for state changes that are lost between the 60 second window if the controller or compute services go down?  What happens when they restart?  On compute node startup I\u0027m assuming it just reports current state as usual, in the case of the controller service going down I assume that things just pick up where they left off when the next report interval shows up - but if nothing changed in the compute node since that time it won\u0027t send any information.\n\nI realize if the controller services goes down you have bigger problems...","commit_id":"095919c3a9777c70be0b2cbb12010d5624b1de2a"},{"author":{"_account_id":1981,"name":"Don Dugger","email":"n0ano@n0ano.com","username":"n0ano"},"change_message_id":"260e9ffd70dce56139d620145bc68915ac8fb4df","unresolved":false,"context_lines":[{"line_number":133,"context_line":"Change should be fairly simple, change the current update code to only call"},{"line_number":134,"context_line":"the DB update if a \u0027state_modified\u0027 routine returns true.  The"},{"line_number":135,"context_line":"\u0027state_modified\u0027 routine returns false if the current state matches the"},{"line_number":136,"context_line":"last recorded state.  Otherwise, it saves the current state as the last"},{"line_number":137,"context_line":"recorded state and returns true."},{"line_number":138,"context_line":""},{"line_number":139,"context_line":""}],"source_content_type":"text/x-rst","patch_set":2,"id":"1ae5cdf2_02dcd045","line":136,"in_reply_to":"1ae5cdf2_c01e655f","updated":"2014-06-16 16:03:25.000000000","message":"As you suspect, the old state is stored in memory so no, you don\u0027t hit the DB to retrieve it.  Also, when the compute node starts up the in-memory state is cleared so, at the first 60 second update period, the new state is guaranteed to be different so the DB will be updated.","commit_id":"095919c3a9777c70be0b2cbb12010d5624b1de2a"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"ba30f69fa2127b87213b8c0189ee4e0350a7ac30","unresolved":false,"context_lines":[{"line_number":146,"context_line":"Testing"},{"line_number":147,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d"},{"line_number":148,"context_line":""},{"line_number":149,"context_line":"No changes should be needed, any tests that currently check the periodic"},{"line_number":150,"context_line":"update mechanism should still pass or fail appropriately with this"},{"line_number":151,"context_line":"mechanism."},{"line_number":152,"context_line":""}],"source_content_type":"text/x-rst","patch_set":2,"id":"1ae5cdf2_a077999e","line":149,"updated":"2014-06-16 14:53:44.000000000","message":"You\u0027re going to need unit tests for the new \u0027state_modified\u0027 routine described above.\n\nThere are always testing impacts...","commit_id":"095919c3a9777c70be0b2cbb12010d5624b1de2a"},{"author":{"_account_id":1981,"name":"Don Dugger","email":"n0ano@n0ano.com","username":"n0ano"},"change_message_id":"260e9ffd70dce56139d620145bc68915ac8fb4df","unresolved":false,"context_lines":[{"line_number":146,"context_line":"Testing"},{"line_number":147,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d"},{"line_number":148,"context_line":""},{"line_number":149,"context_line":"No changes should be needed, any tests that currently check the periodic"},{"line_number":150,"context_line":"update mechanism should still pass or fail appropriately with this"},{"line_number":151,"context_line":"mechanism."},{"line_number":152,"context_line":""}],"source_content_type":"text/x-rst","patch_set":2,"id":"1ae5cdf2_9f5037d6","line":149,"in_reply_to":"1ae5cdf2_a077999e","updated":"2014-06-16 16:03:25.000000000","message":"Point taken.","commit_id":"095919c3a9777c70be0b2cbb12010d5624b1de2a"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"ba30f69fa2127b87213b8c0189ee4e0350a7ac30","unresolved":false,"context_lines":[{"line_number":154,"context_line":"Documentation Impact"},{"line_number":155,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"},{"line_number":156,"context_line":""},{"line_number":157,"context_line":"Section 5 of the Training Guide is slightly incorrect and should be fixed."},{"line_number":158,"context_line":"It currently says \"All compute nodes (also known as hosts in terms of"},{"line_number":159,"context_line":"OpenStack) periodically publish their status, resources available and"},{"line_number":160,"context_line":"hardware capabilities to nova-scheduler through the queue.\"  This should"}],"source_content_type":"text/x-rst","patch_set":2,"id":"1ae5cdf2_60e2f130","line":157,"updated":"2014-06-16 14:53:44.000000000","message":"Please link to the specific section, there are a lot of guides now and I\u0027m having a hard time finding this between the training guides, admin guide and ops guide.","commit_id":"095919c3a9777c70be0b2cbb12010d5624b1de2a"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"ba30f69fa2127b87213b8c0189ee4e0350a7ac30","unresolved":false,"context_lines":[{"line_number":160,"context_line":"hardware capabilities to nova-scheduler through the queue.\"  This should"},{"line_number":161,"context_line":"be modified to reflect that the status is updated in the DB which is then"},{"line_number":162,"context_line":"queried by the scheduler.  (Note this is a generic fix that is really"},{"line_number":163,"context_line":"unrealted to this blueprint.)"},{"line_number":164,"context_line":""},{"line_number":165,"context_line":""},{"line_number":166,"context_line":"References"}],"source_content_type":"text/x-rst","patch_set":2,"id":"1ae5cdf2_8099bdd2","line":163,"updated":"2014-06-16 14:53:44.000000000","message":"s/unrealted/unrelated/.","commit_id":"095919c3a9777c70be0b2cbb12010d5624b1de2a"}]}