)]}'
{"specs/backlog/approved/common-shared-state-scheduler.rst":[{"author":{"_account_id":15648,"name":"Yingxin","email":"yingxin.cheng@intel.com","username":"Yingxin"},"change_message_id":"bdff80b74d7d5922badcdf41f4ababcf90e2cd7a","unresolved":false,"context_lines":[{"line_number":26,"context_line":"Proposed change"},{"line_number":27,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"},{"line_number":28,"context_line":""},{"line_number":29,"context_line":"With a common shared state scheduler, a remote in-memory data store"},{"line_number":30,"context_line":"(such as Redis) is used to hold the in-memory state of all hosts."},{"line_number":31,"context_line":"The host manager can then attach to the remote data store to read the state"},{"line_number":32,"context_line":"of all of the hosts."}],"source_content_type":"text/x-rst","patch_set":1,"id":"3aaa91ec_0cf5b782","line":29,"range":{"start_line":29,"start_character":38,"end_line":29,"end_character":67},"updated":"2016-06-21 16:18:21.000000000","message":"Well, this is the main difference between this design and mine. The global data store is not required in my design, it\u0027s just a nice-to-have thing. So there\u0027s no third-party software required, and I want it to be compatible with resource provider design and with its placement db.\n\nEach Scheduler has a local copy of the entire global resource view, so they can make decisions from their local cache in the 100% lock-free and optimistic way. And the main claim logics in resource trackers with the retry logics are preserved (note: Jay Pipes has a different opinion on this) to guarantee the accuracy of final decisions in compute-node side.\n\nAnd I won\u0027t call my current design \"shared-state scheduler\" any more, it\u0027s an eye-catching concept and may confuse most of the reviewers. The \"eventually consistent host states\" is the only best name describing my central idea.","commit_id":"e129505754e0af4cc207541831c198f420fe6aa1"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"7482194ffaf870eb026d2ccf9ca4d65b110a3cab","unresolved":false,"context_lines":[{"line_number":34,"context_line":"A new scheduler will need to be created (SharedMemoryScheduler) that utilizes"},{"line_number":35,"context_line":"a new host manager that connects to the remote data store."},{"line_number":36,"context_line":""},{"line_number":37,"context_line":"When a compute node receives a new instance, when updating its host state,"},{"line_number":38,"context_line":"along with updating its state in the database, it can also update its state in"},{"line_number":39,"context_line":"the remote data store."},{"line_number":40,"context_line":""}],"source_content_type":"text/x-rst","patch_set":1,"id":"7aa08908_87b318e9","line":37,"updated":"2016-06-15 19:15:56.000000000","message":"Thinking out loud, alternatively couldn\u0027t you have a separate service that is listening on notifications from nova and then updating your other data store that the scheduler reads from? Kind of a crappy solution though if your scheduler manager is in tree but the thing populating that store is out of tree.","commit_id":"e129505754e0af4cc207541831c198f420fe6aa1"},{"author":{"_account_id":1063,"name":"Ed Leafe","email":"ed@leafe.com","username":"ed-leafe"},"change_message_id":"2bb2d920d1167fa565b19674fa52b15c44dd9a49","unresolved":false,"context_lines":[{"line_number":35,"context_line":"a new host manager that connects to the remote data store."},{"line_number":36,"context_line":""},{"line_number":37,"context_line":"When a compute node receives a new instance, when updating its host state,"},{"line_number":38,"context_line":"along with updating its state in the database, it can also update its state in"},{"line_number":39,"context_line":"the remote data store."},{"line_number":40,"context_line":""},{"line_number":41,"context_line":"With this implementation, when a new scheduler is added to the pool, it will"},{"line_number":42,"context_line":"already be aware of the state of all hosts and can begin scheduling instantly."}],"source_content_type":"text/x-rst","patch_set":1,"id":"3aaa91ec_a84932b9","line":39,"range":{"start_line":38,"start_character":0,"end_line":39,"end_character":21},"updated":"2016-06-24 14:35:08.000000000","message":"This has long been my objection to the design: two separate sources of truth. But I guess unless we move to a single database, it\u0027s unavoidable.","commit_id":"e129505754e0af4cc207541831c198f420fe6aa1"},{"author":{"_account_id":16839,"name":"Cale Rath","email":"ctrath@us.ibm.com","username":"ctrath"},"change_message_id":"791630bde6db7c80b20b475e8f12a016ed3d1b27","unresolved":false,"context_lines":[{"line_number":35,"context_line":"a new host manager that connects to the remote data store."},{"line_number":36,"context_line":""},{"line_number":37,"context_line":"When a compute node receives a new instance, when updating its host state,"},{"line_number":38,"context_line":"along with updating its state in the database, it can also update its state in"},{"line_number":39,"context_line":"the remote data store."},{"line_number":40,"context_line":""},{"line_number":41,"context_line":"With this implementation, when a new scheduler is added to the pool, it will"},{"line_number":42,"context_line":"already be aware of the state of all hosts and can begin scheduling instantly."}],"source_content_type":"text/x-rst","patch_set":1,"id":"3aaa91ec_546944f8","line":39,"range":{"start_line":38,"start_character":0,"end_line":39,"end_character":21},"in_reply_to":"3aaa91ec_a84932b9","updated":"2016-06-24 15:20:50.000000000","message":"I thought of this as well.  If there were a shared memory model, would a database used for scheduling even be necessary in this case?  If the redis server goes down, couldn\u0027t the data just be reloaded for the computes?","commit_id":"e129505754e0af4cc207541831c198f420fe6aa1"},{"author":{"_account_id":1063,"name":"Ed Leafe","email":"ed@leafe.com","username":"ed-leafe"},"change_message_id":"2bb2d920d1167fa565b19674fa52b15c44dd9a49","unresolved":false,"context_lines":[{"line_number":37,"context_line":"When a compute node receives a new instance, when updating its host state,"},{"line_number":38,"context_line":"along with updating its state in the database, it can also update its state in"},{"line_number":39,"context_line":"the remote data store."},{"line_number":40,"context_line":""},{"line_number":41,"context_line":"With this implementation, when a new scheduler is added to the pool, it will"},{"line_number":42,"context_line":"already be aware of the state of all hosts and can begin scheduling instantly."},{"line_number":43,"context_line":""}],"source_content_type":"text/x-rst","patch_set":1,"id":"3aaa91ec_28154248","line":40,"updated":"2016-06-24 14:35:08.000000000","message":"Another thing to consider is the nature of the compute node updates: are they incremental (e.g., \"I just allocated 8GB RAM, 1VCPU, and 30GB disk\"), or will it be a replacement (\"I have 64GB RAM, 256 GB disk, and 4 VCPUs right now\"). I would strongly hope that you\u0027re leaning toward the latter.","commit_id":"e129505754e0af4cc207541831c198f420fe6aa1"},{"author":{"_account_id":1063,"name":"Ed Leafe","email":"ed@leafe.com","username":"ed-leafe"},"change_message_id":"2bb2d920d1167fa565b19674fa52b15c44dd9a49","unresolved":false,"context_lines":[{"line_number":38,"context_line":"along with updating its state in the database, it can also update its state in"},{"line_number":39,"context_line":"the remote data store."},{"line_number":40,"context_line":""},{"line_number":41,"context_line":"With this implementation, when a new scheduler is added to the pool, it will"},{"line_number":42,"context_line":"already be aware of the state of all hosts and can begin scheduling instantly."},{"line_number":43,"context_line":""},{"line_number":44,"context_line":"Alternatives"},{"line_number":45,"context_line":"------------"}],"source_content_type":"text/x-rst","patch_set":1,"id":"3aaa91ec_48b40e81","line":42,"range":{"start_line":41,"start_character":0,"end_line":42,"end_character":78},"updated":"2016-06-24 14:35:08.000000000","message":"What is the process of starting up a compute node, and how it will initially populate this in-memory store?","commit_id":"e129505754e0af4cc207541831c198f420fe6aa1"},{"author":{"_account_id":16839,"name":"Cale Rath","email":"ctrath@us.ibm.com","username":"ctrath"},"change_message_id":"791630bde6db7c80b20b475e8f12a016ed3d1b27","unresolved":false,"context_lines":[{"line_number":51,"context_line":"server, it could broadcast its state over RPC to all schedulers. This method"},{"line_number":52,"context_line":"still avoids the database, and all schedulers have the current state of the"},{"line_number":53,"context_line":"hosts. The problem with this method is that with every instance build,"},{"line_number":54,"context_line":"RPC will have increased traffic to send out the state to all schedulers."},{"line_number":55,"context_line":""},{"line_number":56,"context_line":"Data model impact"},{"line_number":57,"context_line":"-----------------"}],"source_content_type":"text/x-rst","patch_set":1,"id":"3aaa91ec_14017c06","line":54,"updated":"2016-06-24 15:20:50.000000000","message":"Instead of broadcasting to all schedulers, could there be a primary scheduler that handles the initial load of state data?  That said, maybe this wouldn\u0027t limit network congestion anyway.","commit_id":"e129505754e0af4cc207541831c198f420fe6aa1"},{"author":{"_account_id":1063,"name":"Ed Leafe","email":"ed@leafe.com","username":"ed-leafe"},"change_message_id":"2bb2d920d1167fa565b19674fa52b15c44dd9a49","unresolved":false,"context_lines":[{"line_number":52,"context_line":"still avoids the database, and all schedulers have the current state of the"},{"line_number":53,"context_line":"hosts. The problem with this method is that with every instance build,"},{"line_number":54,"context_line":"RPC will have increased traffic to send out the state to all schedulers."},{"line_number":55,"context_line":""},{"line_number":56,"context_line":"Data model impact"},{"line_number":57,"context_line":"-----------------"},{"line_number":58,"context_line":""}],"source_content_type":"text/x-rst","patch_set":1,"id":"3aaa91ec_43a34f00","line":55,"updated":"2016-06-24 14:35:08.000000000","message":"The other alternative is to have a single distributed database, so that there is no need for an in-memory cache: the compute nodes and the scheduler are reading from and writing to the same place. Just including this for completeness. :)","commit_id":"e129505754e0af4cc207541831c198f420fe6aa1"},{"author":{"_account_id":1063,"name":"Ed Leafe","email":"ed@leafe.com","username":"ed-leafe"},"change_message_id":"2bb2d920d1167fa565b19674fa52b15c44dd9a49","unresolved":false,"context_lines":[{"line_number":91,"context_line":""},{"line_number":92,"context_line":"If an existing cloud is using the Filter/CachingScheduler, they will need"},{"line_number":93,"context_line":"to be able to migrate the host states to the shared memory. This can be done"},{"line_number":94,"context_line":"by forcing all hosts to update their state in the database, which, after this"},{"line_number":95,"context_line":"change, will also update their state in the remote data store. New"},{"line_number":96,"context_line":"SharedMemorySchedulers can then be introduced into the pool of schedulers"},{"line_number":97,"context_line":"to live side-by-side with the existing schedulers, and can eventually be"}],"source_content_type":"text/x-rst","patch_set":1,"id":"3aaa91ec_a36793b5","line":94,"range":{"start_line":94,"start_character":3,"end_line":94,"end_character":58},"updated":"2016-06-24 14:35:08.000000000","message":"This shouldn\u0027t be necessary. The existing calls where the database is updated should add a call to a new method that updates the shared memory. To force an update of just the shared memory, you would only need to call that new method, and avoid having to interact with the DB.","commit_id":"e129505754e0af4cc207541831c198f420fe6aa1"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"7482194ffaf870eb026d2ccf9ca4d65b110a3cab","unresolved":false,"context_lines":[{"line_number":123,"context_line":"Testing"},{"line_number":124,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d"},{"line_number":125,"context_line":""},{"line_number":126,"context_line":"All existing tests will be used to ensure the new scheduler will function"},{"line_number":127,"context_line":"identically to the existing schedulers."},{"line_number":128,"context_line":""},{"line_number":129,"context_line":"Documentation Impact"}],"source_content_type":"text/x-rst","patch_set":1,"id":"7aa08908_6729ccf2","line":126,"updated":"2016-06-15 19:15:56.000000000","message":"We would need a new job that has a backing store (redis or whatever) to test this.","commit_id":"e129505754e0af4cc207541831c198f420fe6aa1"},{"author":{"_account_id":6873,"name":"Matt Riedemann","email":"mriedem.os@gmail.com","username":"mriedem"},"change_message_id":"7482194ffaf870eb026d2ccf9ca4d65b110a3cab","unresolved":false,"context_lines":[{"line_number":129,"context_line":"Documentation Impact"},{"line_number":130,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"},{"line_number":131,"context_line":""},{"line_number":132,"context_line":"None"},{"line_number":133,"context_line":""},{"line_number":134,"context_line":"References"},{"line_number":135,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"}],"source_content_type":"text/x-rst","patch_set":1,"id":"7aa08908_a733b461","line":132,"updated":"2016-06-15 19:15:56.000000000","message":"Well, yeah there would be.","commit_id":"e129505754e0af4cc207541831c198f420fe6aa1"}]}