)]}'
{"specs/convergence-observer.rst":[{"author":{"_account_id":4715,"name":"Angus Salkeld","email":"asalkeld@redhat.com","username":"asalkeld"},"change_message_id":"f84cc7d02f8ce10dcf64891b6e1e3d88bc04ea6c","unresolved":false,"context_lines":[{"line_number":19,"context_line":""},{"line_number":20,"context_line":"External systems hosting the physical resources of a stack will change"},{"line_number":21,"context_line":"independent of operations in Heat. There is a need to have a way to record"},{"line_number":22,"context_line":"and respond to these changes."},{"line_number":23,"context_line":""},{"line_number":24,"context_line":"Proposed change"},{"line_number":25,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"}],"source_content_type":"text/x-rst","patch_set":1,"id":"1ae5cdf2_0a3c9a92","line":22,"updated":"2014-06-03 01:04:56.000000000","message":"will we support manual overrides as a part of this? So is there a need to allow (maybe temporarily) a user to halt/reboot/etc... a resource without heat fighting back and forcing it back to *the correct state*\nSo first can we detect that a user has done something on purpose? and then have a policy to ignore it. - not sure how desirable this is.","commit_id":"06929b161dac9f94e6e7ebe791e982507f94e18b"},{"author":{"_account_id":4328,"name":"Steven Hardy","email":"steven.hardy@suse.com","username":"shardy"},"change_message_id":"71bb7aed2707ca3e748aa73f032812640fa2f4cd","unresolved":false,"context_lines":[{"line_number":19,"context_line":""},{"line_number":20,"context_line":"External systems hosting the physical resources of a stack will change"},{"line_number":21,"context_line":"independent of operations in Heat. There is a need to have a way to record"},{"line_number":22,"context_line":"and respond to these changes."},{"line_number":23,"context_line":""},{"line_number":24,"context_line":"Proposed change"},{"line_number":25,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"}],"source_content_type":"text/x-rst","patch_set":1,"id":"1ae5cdf2_6b4fd6bd","line":22,"in_reply_to":"1ae5cdf2_0a3c9a92","updated":"2014-06-03 09:46:03.000000000","message":"IMO the convergence should only be active when the stack is in an IN_PROGRESS state, so if you do a manual out-of-band action while the stack is in a COMPLETE state, heat won\u0027t do anything.  \n\nBut then you might do a heat stack-update --converge or something later, which would re-align the world with the expected state?\n\nClint, is that how you expect this to work?","commit_id":"06929b161dac9f94e6e7ebe791e982507f94e18b"},{"author":{"_account_id":4328,"name":"Steven Hardy","email":"steven.hardy@suse.com","username":"shardy"},"change_message_id":"fb4db234df410fc4fe7aeeedecb62a4776893c41","unresolved":false,"context_lines":[{"line_number":19,"context_line":""},{"line_number":20,"context_line":"External systems hosting the physical resources of a stack will change"},{"line_number":21,"context_line":"independent of operations in Heat. There is a need to have a way to record"},{"line_number":22,"context_line":"and respond to these changes."},{"line_number":23,"context_line":""},{"line_number":24,"context_line":"Proposed change"},{"line_number":25,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"}],"source_content_type":"text/x-rst","patch_set":1,"id":"1ae5cdf2_3be86a7b","line":22,"in_reply_to":"1ae5cdf2_2c575a03","updated":"2014-06-05 08:46:34.000000000","message":"The state of a stack is the combination of the action and the status, so UPDATE, IN_PROGRESS is the exact same thing as \"UPDATING\" IMO.\n\nWe could change the terminology, but I don\u0027t see the point given that all it\u0027s likely to do is confuse existing users.","commit_id":"06929b161dac9f94e6e7ebe791e982507f94e18b"},{"author":{"_account_id":6488,"name":"Clint Byrum","email":"clint@fewbar.com","username":"clint-fewbar"},"change_message_id":"235ac635440166c7353833d5b8a2b59575f3de0a","unresolved":false,"context_lines":[{"line_number":19,"context_line":""},{"line_number":20,"context_line":"External systems hosting the physical resources of a stack will change"},{"line_number":21,"context_line":"independent of operations in Heat. There is a need to have a way to record"},{"line_number":22,"context_line":"and respond to these changes."},{"line_number":23,"context_line":""},{"line_number":24,"context_line":"Proposed change"},{"line_number":25,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"}],"source_content_type":"text/x-rst","patch_set":1,"id":"1ae5cdf2_35523667","line":22,"in_reply_to":"1ae5cdf2_3be86a7b","updated":"2014-06-05 10:15:23.000000000","message":"Agree with you there Steven. There is quite a lot of helpful information in the combination of those two things for users. Also we don\u0027t want to break userspace, so keeping the external concept of ACTION+STATUS is important.\n\nMainly what we\u0027re changing is what you can do within those states.","commit_id":"06929b161dac9f94e6e7ebe791e982507f94e18b"},{"author":{"_account_id":6800,"name":"Christopher Armstrong","email":"chris.armstrong@rackspace.com","username":"radix"},"change_message_id":"84a3971c0c788dd7a9ff0d87356cae514c0516e2","unresolved":false,"context_lines":[{"line_number":19,"context_line":""},{"line_number":20,"context_line":"External systems hosting the physical resources of a stack will change"},{"line_number":21,"context_line":"independent of operations in Heat. There is a need to have a way to record"},{"line_number":22,"context_line":"and respond to these changes."},{"line_number":23,"context_line":""},{"line_number":24,"context_line":"Proposed change"},{"line_number":25,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"}],"source_content_type":"text/x-rst","patch_set":1,"id":"1ae5cdf2_af0c463c","line":22,"in_reply_to":"1ae5cdf2_6b4fd6bd","updated":"2014-06-04 20:56:58.000000000","message":"I think ultimately Heat needs to be able to have constant convergence. Doing convergence only during stack operations may be a useful intermediate step, though.\n\nIt can be a matter of policy/configuration about when convergence is triggered, perhaps, but I think there are pretty important use cases that involve converging when any observed divergence occurs.","commit_id":"06929b161dac9f94e6e7ebe791e982507f94e18b"},{"author":{"_account_id":8246,"name":"Qiming Teng","email":"tengqm@outlook.com","username":"tengqm"},"change_message_id":"154321f086bb26f20f6d1a8360c5cda31a9739b3","unresolved":false,"context_lines":[{"line_number":19,"context_line":""},{"line_number":20,"context_line":"External systems hosting the physical resources of a stack will change"},{"line_number":21,"context_line":"independent of operations in Heat. There is a need to have a way to record"},{"line_number":22,"context_line":"and respond to these changes."},{"line_number":23,"context_line":""},{"line_number":24,"context_line":"Proposed change"},{"line_number":25,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"}],"source_content_type":"text/x-rst","patch_set":1,"id":"1ae5cdf2_2c575a03","line":22,"in_reply_to":"1ae5cdf2_6b4fd6bd","updated":"2014-06-05 04:59:09.000000000","message":"Personally, I don\u0027t like the design that has \u0027IN_PROGRESS\u0027, \u0027COMPLETE\u0027 etc. as stack states.  \u0027IN_PROGRESS\u0027 can be a state of a create/update/delete operation, not a state of a stack.  A stack state can be something like \u0027NEW\u0027, \u0027READY\u0027, \u0027UPDATING\u0027, \u0027DELETING\u0027, ..., \u0027ERROR\u0027, etc.","commit_id":"06929b161dac9f94e6e7ebe791e982507f94e18b"},{"author":{"_account_id":4715,"name":"Angus Salkeld","email":"asalkeld@redhat.com","username":"asalkeld"},"change_message_id":"7287a113364d12dcfa1533cf9a7d48e8579aa151","unresolved":false,"context_lines":[{"line_number":19,"context_line":""},{"line_number":20,"context_line":"External systems hosting the physical resources of a stack will change"},{"line_number":21,"context_line":"independent of operations in Heat. There is a need to have a way to record"},{"line_number":22,"context_line":"and respond to these changes."},{"line_number":23,"context_line":""},{"line_number":24,"context_line":"Proposed change"},{"line_number":25,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"}],"source_content_type":"text/x-rst","patch_set":1,"id":"1ae5cdf2_9f719c78","line":22,"in_reply_to":"1ae5cdf2_6b4fd6bd","updated":"2014-06-04 02:53:56.000000000","message":"well steve the point was to keep running the checker, so it would \"repair\" - I was just imagining having a fight with the \"repairer\" unexpectedly.","commit_id":"06929b161dac9f94e6e7ebe791e982507f94e18b"},{"author":{"_account_id":6488,"name":"Clint Byrum","email":"clint@fewbar.com","username":"clint-fewbar"},"change_message_id":"cc66dc8070bc9ecb7c8f51f53c4f5b2ef16a6c48","unresolved":false,"context_lines":[{"line_number":19,"context_line":""},{"line_number":20,"context_line":"External systems hosting the physical resources of a stack will change"},{"line_number":21,"context_line":"independent of operations in Heat. There is a need to have a way to record"},{"line_number":22,"context_line":"and respond to these changes."},{"line_number":23,"context_line":""},{"line_number":24,"context_line":"Proposed change"},{"line_number":25,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"}],"source_content_type":"text/x-rst","patch_set":1,"id":"1ae5cdf2_1ee2e52e","line":22,"in_reply_to":"1ae5cdf2_9f719c78","updated":"2014-06-04 09:11:02.000000000","message":"Heat owns those resources. So if you kill it, Heat _should_ resurrect it. If you go and stop it, Heat should start it. I think initially Steve is right, that the observer will only be doing observing during IN_PROGRESS states. But there is a real use case there for the observer to just constantly observe, and request convergence on anything that changes. How often I think will I think be up to operators as the engines will spend their spare cycles doing periodic converging, and we may even want to just avoid going there entirely until we have a clear path to using the notifications bus.\n\nRegarding the initial question, there was a mailing list thread about how to stop a server with Heat, and I suggested we add a way to specify the desired server state (with ACTIVE being the default) which would give you the desired use case. With updates being decoupled from the update action, this should be a rather straight forward way for operators to maintain their servers. And it also becomes a good place to reconfigure other pieces of the app to route around the stopped service if possible.","commit_id":"06929b161dac9f94e6e7ebe791e982507f94e18b"},{"author":{"_account_id":8246,"name":"Qiming Teng","email":"tengqm@outlook.com","username":"tengqm"},"change_message_id":"154321f086bb26f20f6d1a8360c5cda31a9739b3","unresolved":false,"context_lines":[{"line_number":19,"context_line":""},{"line_number":20,"context_line":"External systems hosting the physical resources of a stack will change"},{"line_number":21,"context_line":"independent of operations in Heat. There is a need to have a way to record"},{"line_number":22,"context_line":"and respond to these changes."},{"line_number":23,"context_line":""},{"line_number":24,"context_line":"Proposed change"},{"line_number":25,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"}],"source_content_type":"text/x-rst","patch_set":1,"id":"1ae5cdf2_cce1962b","line":22,"in_reply_to":"1ae5cdf2_af0c463c","updated":"2014-06-05 04:59:09.000000000","message":"I agree with Chris.\n\nAnother thought is that maybe we need to consider a \u0027critical section\u0027 alike concept as well.  It is not just about locks on stacks or resources.  It is about a (short, hopefully) execution path where no convergence operation is allowed.  Are we modelling such a path as a \u0027job\u0027, which will be guaranteed to either 1) finish as expected or 2) nothing change at all.","commit_id":"06929b161dac9f94e6e7ebe791e982507f94e18b"},{"author":{"_account_id":6488,"name":"Clint Byrum","email":"clint@fewbar.com","username":"clint-fewbar"},"change_message_id":"235ac635440166c7353833d5b8a2b59575f3de0a","unresolved":false,"context_lines":[{"line_number":19,"context_line":""},{"line_number":20,"context_line":"External systems hosting the physical resources of a stack will change"},{"line_number":21,"context_line":"independent of operations in Heat. There is a need to have a way to record"},{"line_number":22,"context_line":"and respond to these changes."},{"line_number":23,"context_line":""},{"line_number":24,"context_line":"Proposed change"},{"line_number":25,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"}],"source_content_type":"text/x-rst","patch_set":1,"id":"1ae5cdf2_1561ba04","line":22,"in_reply_to":"1ae5cdf2_cce1962b","updated":"2014-06-05 10:15:23.000000000","message":"This sounds like workflow to me. Our lifecycle signaling bits (other bp) will help give a workflow system a chance to hook in and implement critical sections. We\u0027ve been looking at using something like ZooKeeper or etcd to elect leaders for RabbitMQ and Galera in TripleO for instance. That is entirely a workflow problem. Heat, however, does not want to ever allow you to express the workflow as it is an orchestration tool and workflows come in quite a few different shapes and sizes.","commit_id":"06929b161dac9f94e6e7ebe791e982507f94e18b"},{"author":{"_account_id":4715,"name":"Angus Salkeld","email":"asalkeld@redhat.com","username":"asalkeld"},"change_message_id":"f84cc7d02f8ce10dcf64891b6e1e3d88bc04ea6c","unresolved":false,"context_lines":[{"line_number":22,"context_line":"and respond to these changes."},{"line_number":23,"context_line":""},{"line_number":24,"context_line":"Proposed change"},{"line_number":25,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"},{"line_number":26,"context_line":""},{"line_number":27,"context_line":"* Observer is responsible for managing the model of reality - both"},{"line_number":28,"context_line":"  polling and handling notifications it provides an abstraction so that"}],"source_content_type":"text/x-rst","patch_set":1,"id":"1ae5cdf2_0a873a62","line":25,"updated":"2014-06-03 01:04:56.000000000","message":"I\u0027d suggest this is a super simple entity that just calls \"check()\" in the plugin and leave it up to the plugin what mechanism it wants to use notification/polling. Else there is going to be a mess of checking code that is separated from the plugin. More importantly is how the plugin update the state, do we have a \"nova style conductor\" that abstracts the db access? do we emit new notifications \"state-dirvergence-message\"? what is that mechanism?\nOur engine api is very fluid atm and this needs to be made more structured to allow for updates (the api that the resource uses to talk to the engine)","commit_id":"06929b161dac9f94e6e7ebe791e982507f94e18b"},{"author":{"_account_id":6488,"name":"Clint Byrum","email":"clint@fewbar.com","username":"clint-fewbar"},"change_message_id":"cc66dc8070bc9ecb7c8f51f53c4f5b2ef16a6c48","unresolved":false,"context_lines":[{"line_number":22,"context_line":"and respond to these changes."},{"line_number":23,"context_line":""},{"line_number":24,"context_line":"Proposed change"},{"line_number":25,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"},{"line_number":26,"context_line":""},{"line_number":27,"context_line":"* Observer is responsible for managing the model of reality - both"},{"line_number":28,"context_line":"  polling and handling notifications it provides an abstraction so that"}],"source_content_type":"text/x-rst","patch_set":1,"id":"1ae5cdf2_9e127507","line":25,"in_reply_to":"1ae5cdf2_0a873a62","updated":"2014-06-04 09:11:02.000000000","message":"I copied these directly from the etherpad, but I think they don\u0027t quite communicate the vision. Your suggestion is entirely what I see as well. The only difference will be that check() needs to give us back data about what it has observed. Initially we can just interpolate a positive result as \"ACTIVE\" and others as \"ERROR\", but I think we\u0027ll want to have the results of those checks turn into data that the convergence engine can use to know what needs to be re-asserted.\n\nI will take another stab at communicating that more clearly in my next patch upload.","commit_id":"06929b161dac9f94e6e7ebe791e982507f94e18b"},{"author":{"_account_id":8246,"name":"Qiming Teng","email":"tengqm@outlook.com","username":"tengqm"},"change_message_id":"154321f086bb26f20f6d1a8360c5cda31a9739b3","unresolved":false,"context_lines":[{"line_number":30,"context_line":""},{"line_number":31,"context_line":"    * polls nova/neutron/etc"},{"line_number":32,"context_line":""},{"line_number":33,"context_line":"    * listens for ceilometer etc"},{"line_number":34,"context_line":""},{"line_number":35,"context_line":"    * conceptually polls heat stack descriptions to update internal resources"},{"line_number":36,"context_line":""}],"source_content_type":"text/x-rst","patch_set":1,"id":"1ae5cdf2_ac544aca","line":33,"updated":"2014-06-05 04:59:09.000000000","message":"well, we can listen to nova, neutron as well, not just poll them.","commit_id":"06929b161dac9f94e6e7ebe791e982507f94e18b"},{"author":{"_account_id":9420,"name":"Matthew Gilliard","email":"matthew.gilliard@hp.com","username":"gilliard"},"change_message_id":"4e7467dabc30243e774d70bf2dde85f78a83598a","unresolved":false,"context_lines":[{"line_number":36,"context_line":""},{"line_number":37,"context_line":"* Data model will need to store \"observed state\""},{"line_number":38,"context_line":""},{"line_number":39,"context_line":"* REST API will need to display \"observed state\""},{"line_number":40,"context_line":""},{"line_number":41,"context_line":"* Periodic tasks will be run on each observer to periodically poll"},{"line_number":42,"context_line":"  backend services until such time as all backend services are covered by"}],"source_content_type":"text/x-rst","patch_set":1,"id":"1ae5cdf2_efc2ce83","line":39,"updated":"2014-06-04 21:05:16.000000000","message":"What\u0027s the purpose of this?  So that people can monitor how their stack is being updated?","commit_id":"06929b161dac9f94e6e7ebe791e982507f94e18b"},{"author":{"_account_id":6488,"name":"Clint Byrum","email":"clint@fewbar.com","username":"clint-fewbar"},"change_message_id":"235ac635440166c7353833d5b8a2b59575f3de0a","unresolved":false,"context_lines":[{"line_number":36,"context_line":""},{"line_number":37,"context_line":"* Data model will need to store \"observed state\""},{"line_number":38,"context_line":""},{"line_number":39,"context_line":"* REST API will need to display \"observed state\""},{"line_number":40,"context_line":""},{"line_number":41,"context_line":"* Periodic tasks will be run on each observer to periodically poll"},{"line_number":42,"context_line":"  backend services until such time as all backend services are covered by"}],"source_content_type":"text/x-rst","patch_set":1,"id":"1ae5cdf2_55fb4246","line":39,"in_reply_to":"1ae5cdf2_31996b6e","updated":"2014-06-05 10:15:23.000000000","message":"The intention is to expose very little. But this particular bit is one thing I think we need to expose. To hide it will I think result in frustration as users dig through event lists trying to figure out why their servers are being brought back from the dead or rebuilt with new images.","commit_id":"06929b161dac9f94e6e7ebe791e982507f94e18b"},{"author":{"_account_id":9420,"name":"Matthew Gilliard","email":"matthew.gilliard@hp.com","username":"gilliard"},"change_message_id":"a81721fadbd5b5758ce7dfab21172cbd884675fd","unresolved":false,"context_lines":[{"line_number":36,"context_line":""},{"line_number":37,"context_line":"* Data model will need to store \"observed state\""},{"line_number":38,"context_line":""},{"line_number":39,"context_line":"* REST API will need to display \"observed state\""},{"line_number":40,"context_line":""},{"line_number":41,"context_line":"* Periodic tasks will be run on each observer to periodically poll"},{"line_number":42,"context_line":"  backend services until such time as all backend services are covered by"}],"source_content_type":"text/x-rst","patch_set":1,"id":"1ae5cdf2_31996b6e","line":39,"in_reply_to":"1ae5cdf2_efc2ce83","updated":"2014-06-05 09:34:47.000000000","message":"In fact, I guess that leads to a more general question of how much of this we expose to users.  Is that covered somewhere?","commit_id":"06929b161dac9f94e6e7ebe791e982507f94e18b"},{"author":{"_account_id":4328,"name":"Steven Hardy","email":"steven.hardy@suse.com","username":"shardy"},"change_message_id":"71bb7aed2707ca3e748aa73f032812640fa2f4cd","unresolved":false,"context_lines":[{"line_number":67,"context_line":""},{"line_number":68,"context_line":"* Modify data model to record resource state"},{"line_number":69,"context_line":"* Modify public API to display observed state"},{"line_number":70,"context_line":"* Create new observer RPC API calls"},{"line_number":71,"context_line":"* Create new observer entry point"},{"line_number":72,"context_line":"* Move \"check_active\" calls to use observer API"},{"line_number":73,"context_line":""}],"source_content_type":"text/x-rst","patch_set":1,"id":"1ae5cdf2_eb6e8600","line":70,"updated":"2014-06-03 09:46:03.000000000","message":"Isn\u0027t this just an additional field added to all stacks/resources?\n\nI\u0027m not sure I see the value of taking the internal implementation detail of polling/handling notifications and abstracting it behind a public API - seems like we should just expose the state via existing API\u0027s, and have the observer tasks write state direct to the DB, so we can avoid the overhead of yet-another layer of API calls?","commit_id":"06929b161dac9f94e6e7ebe791e982507f94e18b"},{"author":{"_account_id":6488,"name":"Clint Byrum","email":"clint@fewbar.com","username":"clint-fewbar"},"change_message_id":"cc66dc8070bc9ecb7c8f51f53c4f5b2ef16a6c48","unresolved":false,"context_lines":[{"line_number":67,"context_line":""},{"line_number":68,"context_line":"* Modify data model to record resource state"},{"line_number":69,"context_line":"* Modify public API to display observed state"},{"line_number":70,"context_line":"* Create new observer RPC API calls"},{"line_number":71,"context_line":"* Create new observer entry point"},{"line_number":72,"context_line":"* Move \"check_active\" calls to use observer API"},{"line_number":73,"context_line":""}],"source_content_type":"text/x-rst","patch_set":1,"id":"1ae5cdf2_becf317e","line":70,"in_reply_to":"1ae5cdf2_eb6e8600","updated":"2014-06-04 09:11:02.000000000","message":"The new calls are for dispatching observation jobs. It will definitely just go write to the DB with the observed state. But we need to tell the observer about new things to go and observe.","commit_id":"06929b161dac9f94e6e7ebe791e982507f94e18b"}]}