)]}'
{"specs/super-scheduler.rst":[{"author":{"_account_id":9608,"name":"Pranesh Pandurangan","email":"praneshpg@gmail.com","username":"pran1990"},"change_message_id":"d4c1052b48deef34d4a07d29de7dccd40551f6e6","unresolved":false,"context_lines":[{"line_number":8,"context_line":"scheduling its resources that that project *maintains*. This is great for"},{"line_number":9,"context_line":"getting a project going quickly and for ensuring that the project functions"},{"line_number":10,"context_line":"correctly with those resources under load, under upgrades and such. It though"},{"line_number":11,"context_line":"does not create a `local optimum`_ that scheduling can achieve, in that there"},{"line_number":12,"context_line":"is no way to schedule in a more *global* manner. To move beyond localized"},{"line_number":13,"context_line":"scheduling this spec will start to discuss how we can move away from this"},{"line_number":14,"context_line":"existing model and work toward scheduling that can achieve a"}],"source_content_type":"text/x-rst","patch_set":1,"id":"1a4dcd0f_3b02b70c","line":11,"updated":"2015-08-07 17:45:07.000000000","message":"This isn\u0027t very clear. Do you mean that it creates a local optimum and fails to achieve global optimum, or it misses both? To me, per project scheduling is local optimum, no?","commit_id":"730391b9226d50803ea309ce51663e9eae78da4f"},{"author":{"_account_id":1297,"name":"Joshua Harlow","email":"jxharlow@godaddy.com","username":"harlowja"},"change_message_id":"239748d7edae1891ab3ffd270fa46c95353d4673","unresolved":false,"context_lines":[{"line_number":8,"context_line":"scheduling its resources that that project *maintains*. This is great for"},{"line_number":9,"context_line":"getting a project going quickly and for ensuring that the project functions"},{"line_number":10,"context_line":"correctly with those resources under load, under upgrades and such. It though"},{"line_number":11,"context_line":"does not create a `local optimum`_ that scheduling can achieve, in that there"},{"line_number":12,"context_line":"is no way to schedule in a more *global* manner. To move beyond localized"},{"line_number":13,"context_line":"scheduling this spec will start to discuss how we can move away from this"},{"line_number":14,"context_line":"existing model and work toward scheduling that can achieve a"}],"source_content_type":"text/x-rst","patch_set":1,"id":"1a4dcd0f_fe075d7f","line":11,"in_reply_to":"1a4dcd0f_3b02b70c","updated":"2015-08-07 17:55:41.000000000","message":"Yes, per project scheduling will always hit local optimums (due to it being a per-project/local scheduling process). Will tweak.","commit_id":"730391b9226d50803ea309ce51663e9eae78da4f"},{"author":{"_account_id":1297,"name":"Joshua Harlow","email":"jxharlow@godaddy.com","username":"harlowja"},"change_message_id":"0051b28b69a436b1a0c6c808acaed8ad22361875","unresolved":false,"context_lines":[{"line_number":38,"context_line":"    -   label: frontend"},{"line_number":39,"context_line":"        count: 5"},{"line_number":40,"context_line":"        image: ubuntu_vanilla"},{"line_number":41,"context_line":"        flavor: high memory low disk            "},{"line_number":42,"context_line":"        stateless: true"},{"line_number":43,"context_line":"        backend: VM        "},{"line_number":44,"context_line":"    -   label: database"}],"source_content_type":"text/x-rst","patch_set":9,"id":"9a8ffd7b_be95a932","line":41,"updated":"2015-12-02 19:44:38.000000000","message":"nit: chop off this whitespace","commit_id":"18e76651a08d1d22f20c63ab9a6b21e0b2594ae4"},{"author":{"_account_id":1297,"name":"Joshua Harlow","email":"jxharlow@godaddy.com","username":"harlowja"},"change_message_id":"0051b28b69a436b1a0c6c808acaed8ad22361875","unresolved":false,"context_lines":[{"line_number":75,"context_line":"            - memchache"},{"line_number":76,"context_line":"    constraints:"},{"line_number":77,"context_line":"    -   ref: split"},{"line_number":78,"context_line":"        params:"},{"line_number":79,"context_line":"        -   database"},{"line_number":80,"context_line":"        -   frontend"},{"line_number":81,"context_line":"    -   ref: spread"}],"source_content_type":"text/x-rst","patch_set":9,"id":"9a8ffd7b_beacc988","line":78,"updated":"2015-12-02 19:44:38.000000000","message":"Whats ref here?","commit_id":"18e76651a08d1d22f20c63ab9a6b21e0b2594ae4"},{"author":{"_account_id":1297,"name":"Joshua Harlow","email":"jxharlow@godaddy.com","username":"harlowja"},"change_message_id":"d0d6f7b0bc1bcb759980c4bdf0e8d20016c1608d","unresolved":false,"context_lines":[{"line_number":75,"context_line":"            - memchache"},{"line_number":76,"context_line":"    constraints:"},{"line_number":77,"context_line":"    -   ref: split"},{"line_number":78,"context_line":"        params:"},{"line_number":79,"context_line":"        -   database"},{"line_number":80,"context_line":"        -   frontend"},{"line_number":81,"context_line":"    -   ref: spread"}],"source_content_type":"text/x-rst","patch_set":9,"id":"9a8ffd7b_9323f3bc","line":78,"in_reply_to":"9a8ffd7b_21319e5a","updated":"2015-12-03 00:27:08.000000000","message":"gotcha, perhaps describe that in \u0027Core element description\u0027?","commit_id":"18e76651a08d1d22f20c63ab9a6b21e0b2594ae4"},{"author":{"_account_id":19273,"name":"Fabien Hermenier","email":"fabien.hermenier@unice.fr","username":"fhermeni"},"change_message_id":"edcd66a2ce56d8a6ee711590fe89898b6bb7098e","unresolved":false,"context_lines":[{"line_number":75,"context_line":"            - memchache"},{"line_number":76,"context_line":"    constraints:"},{"line_number":77,"context_line":"    -   ref: split"},{"line_number":78,"context_line":"        params:"},{"line_number":79,"context_line":"        -   database"},{"line_number":80,"context_line":"        -   frontend"},{"line_number":81,"context_line":"    -   ref: spread"}],"source_content_type":"text/x-rst","patch_set":9,"id":"9a8ffd7b_21319e5a","line":78,"in_reply_to":"9a8ffd7b_beacc988","updated":"2015-12-02 22:10:57.000000000","message":"the constraint identifier. Named ref because I just wanted to state it references an existing implementation in the scheduler library. \"id\" is ok as well. \"label\" is not strong enough to me.","commit_id":"18e76651a08d1d22f20c63ab9a6b21e0b2594ae4"},{"author":{"_account_id":1297,"name":"Joshua Harlow","email":"jxharlow@godaddy.com","username":"harlowja"},"change_message_id":"0051b28b69a436b1a0c6c808acaed8ad22361875","unresolved":false,"context_lines":[{"line_number":109,"context_line":"purpose is to have a specification that will support, aside with the official"},{"line_number":110,"context_line":"openstack constraint, i) an entry point to ease the integration of extensions"},{"line_number":111,"context_line":"that reply to very specific needs but cannot integrate the release, ii) custom"},{"line_number":112,"context_line":"schedulers that provide their own constraint library."},{"line_number":113,"context_line":""},{"line_number":114,"context_line":"Core element description"},{"line_number":115,"context_line":"------------------------"}],"source_content_type":"text/x-rst","patch_set":9,"id":"9a8ffd7b_5e483db3","line":112,"updated":"2015-12-02 19:44:38.000000000","message":"Sounds great IMHO,\n\nTo accommodate what exists in openstack I wonder if we should also have a way to mark that a projects local-scheduler can be delegated to for a given constraint (vs having to have the holistic one solve the constraint)?\n\nWhat do u think (having this ability would probably make it easier to integrate this into projects with existing schedulers); this could say allow for features to be built at 2 levels, one that uses project local constraints and one that uses more holistic constraints (and people can mix them as they choose).","commit_id":"18e76651a08d1d22f20c63ab9a6b21e0b2594ae4"},{"author":{"_account_id":19273,"name":"Fabien Hermenier","email":"fabien.hermenier@unice.fr","username":"fhermeni"},"change_message_id":"edcd66a2ce56d8a6ee711590fe89898b6bb7098e","unresolved":false,"context_lines":[{"line_number":109,"context_line":"purpose is to have a specification that will support, aside with the official"},{"line_number":110,"context_line":"openstack constraint, i) an entry point to ease the integration of extensions"},{"line_number":111,"context_line":"that reply to very specific needs but cannot integrate the release, ii) custom"},{"line_number":112,"context_line":"schedulers that provide their own constraint library."},{"line_number":113,"context_line":""},{"line_number":114,"context_line":"Core element description"},{"line_number":115,"context_line":"------------------------"}],"source_content_type":"text/x-rst","patch_set":9,"id":"9a8ffd7b_e1b156e1","line":112,"in_reply_to":"9a8ffd7b_5e483db3","updated":"2015-12-02 22:10:57.000000000","message":"You are correct. It is completely reasonable to allow a smooth transition. What I propose as a possible consistent solution is to provide some kind of a \"legacy holistic solver\" that will just implements the future \"holistic\" API but in practice will dispatch the local aspects to their associated scheduler.","commit_id":"18e76651a08d1d22f20c63ab9a6b21e0b2594ae4"},{"author":{"_account_id":19273,"name":"Fabien Hermenier","email":"fabien.hermenier@unice.fr","username":"fhermeni"},"change_message_id":"b7fad5cac616b5d8e44e989bf7d2cf8bab67ddc0","unresolved":false,"context_lines":[{"line_number":109,"context_line":"purpose is to have a specification that will support, aside with the official"},{"line_number":110,"context_line":"openstack constraint, i) an entry point to ease the integration of extensions"},{"line_number":111,"context_line":"that reply to very specific needs but cannot integrate the release, ii) custom"},{"line_number":112,"context_line":"schedulers that provide their own constraint library."},{"line_number":113,"context_line":""},{"line_number":114,"context_line":"Core element description"},{"line_number":115,"context_line":"------------------------"}],"source_content_type":"text/x-rst","patch_set":9,"id":"9a8ffd7b_6af9a44d","line":112,"in_reply_to":"9a8ffd7b_ae056e10","updated":"2015-12-03 08:04:25.000000000","message":"Early in the next week, i will push a new version with this section and remove the \"proposed change\" one","commit_id":"18e76651a08d1d22f20c63ab9a6b21e0b2594ae4"},{"author":{"_account_id":1297,"name":"Joshua Harlow","email":"jxharlow@godaddy.com","username":"harlowja"},"change_message_id":"d0d6f7b0bc1bcb759980c4bdf0e8d20016c1608d","unresolved":false,"context_lines":[{"line_number":109,"context_line":"purpose is to have a specification that will support, aside with the official"},{"line_number":110,"context_line":"openstack constraint, i) an entry point to ease the integration of extensions"},{"line_number":111,"context_line":"that reply to very specific needs but cannot integrate the release, ii) custom"},{"line_number":112,"context_line":"schedulers that provide their own constraint library."},{"line_number":113,"context_line":""},{"line_number":114,"context_line":"Core element description"},{"line_number":115,"context_line":"------------------------"}],"source_content_type":"text/x-rst","patch_set":9,"id":"9a8ffd7b_ae056e10","line":112,"in_reply_to":"9a8ffd7b_e1b156e1","updated":"2015-12-03 00:27:08.000000000","message":"Cool, seems like we should have a section for how to do this \u0027interop\u0027 for the foreseeable future (if not forever?)","commit_id":"18e76651a08d1d22f20c63ab9a6b21e0b2594ae4"},{"author":{"_account_id":1297,"name":"Joshua Harlow","email":"jxharlow@godaddy.com","username":"harlowja"},"change_message_id":"0051b28b69a436b1a0c6c808acaed8ad22361875","unresolved":false,"context_lines":[{"line_number":124,"context_line":"deployment"},{"line_number":125,"context_line":""},{"line_number":126,"context_line":""},{"line_number":127,"context_line":"**stateless:** "},{"line_number":128,"context_line":""},{"line_number":129,"context_line":"For the networks:"},{"line_number":130,"context_line":""}],"source_content_type":"text/x-rst","patch_set":9,"id":"9a8ffd7b_7edc21e4","line":127,"updated":"2015-12-02 19:44:38.000000000","message":"nit whitespace","commit_id":"18e76651a08d1d22f20c63ab9a6b21e0b2594ae4"},{"author":{"_account_id":1297,"name":"Joshua Harlow","email":"jxharlow@godaddy.com","username":"harlowja"},"change_message_id":"0051b28b69a436b1a0c6c808acaed8ad22361875","unresolved":false,"context_lines":[{"line_number":151,"context_line":""},{"line_number":152,"context_line":"Proposed change"},{"line_number":153,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"},{"line_number":154,"context_line":""},{"line_number":155,"context_line":"The general idea here would be to move away from projects having *smart*"},{"line_number":156,"context_line":"API(s) that go above and beyond the following simple (not fully REST"},{"line_number":157,"context_line":"compliant, and below is just a draft) API:"}],"source_content_type":"text/x-rst","patch_set":9,"id":"9a8ffd7b_7e03c142","line":154,"updated":"2015-12-02 19:44:38.000000000","message":"Perhaps we should leave this out for now, what do u think (maybe to early to discuss proposed changes like this).","commit_id":"18e76651a08d1d22f20c63ab9a6b21e0b2594ae4"},{"author":{"_account_id":1297,"name":"Joshua Harlow","email":"jxharlow@godaddy.com","username":"harlowja"},"change_message_id":"d0d6f7b0bc1bcb759980c4bdf0e8d20016c1608d","unresolved":false,"context_lines":[{"line_number":151,"context_line":""},{"line_number":152,"context_line":"Proposed change"},{"line_number":153,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"},{"line_number":154,"context_line":""},{"line_number":155,"context_line":"The general idea here would be to move away from projects having *smart*"},{"line_number":156,"context_line":"API(s) that go above and beyond the following simple (not fully REST"},{"line_number":157,"context_line":"compliant, and below is just a draft) API:"}],"source_content_type":"text/x-rst","patch_set":9,"id":"9a8ffd7b_6e146643","line":154,"in_reply_to":"9a8ffd7b_615c8619","updated":"2015-12-03 00:27:08.000000000","message":"Ok, I\u0027m fine with removing it (if u are going to upload a new version feel free to remove it, or I\u0027ll chop it).","commit_id":"18e76651a08d1d22f20c63ab9a6b21e0b2594ae4"},{"author":{"_account_id":19273,"name":"Fabien Hermenier","email":"fabien.hermenier@unice.fr","username":"fhermeni"},"change_message_id":"edcd66a2ce56d8a6ee711590fe89898b6bb7098e","unresolved":false,"context_lines":[{"line_number":151,"context_line":""},{"line_number":152,"context_line":"Proposed change"},{"line_number":153,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"},{"line_number":154,"context_line":""},{"line_number":155,"context_line":"The general idea here would be to move away from projects having *smart*"},{"line_number":156,"context_line":"API(s) that go above and beyond the following simple (not fully REST"},{"line_number":157,"context_line":"compliant, and below is just a draft) API:"}],"source_content_type":"text/x-rst","patch_set":9,"id":"9a8ffd7b_615c8619","line":154,"in_reply_to":"9a8ffd7b_7e03c142","updated":"2015-12-02 22:10:57.000000000","message":"Yes I agree. Maybe the routes will have to be linked in some way to the language key elements. If so, it is indeed too early","commit_id":"18e76651a08d1d22f20c63ab9a6b21e0b2594ae4"},{"author":{"_account_id":1297,"name":"Joshua Harlow","email":"jxharlow@godaddy.com","username":"harlowja"},"change_message_id":"5c2d82eccfd7cc5918efaaaf07961c48d8327d2a","unresolved":false,"context_lines":[{"line_number":127,"context_line":"   interestring in case of multiple nova compute sharding the ironic-nodes."},{"line_number":128,"context_line":""},{"line_number":129,"context_line":"A better approach to solve the resouce in-consistent view would be to have a"},{"line_number":130,"context_line":"global view of the resources. This global view of resources can serve as a"},{"line_number":131,"context_line":"source of truth for the super-scheduler. Doing so and having a wholesome view"},{"line_number":132,"context_line":"of the cloud the scheduler can select optimal compute depending on the"},{"line_number":133,"context_line":"parameters passed as part of the request. This also gets rid of task to keep"}],"source_content_type":"text/x-rst","patch_set":11,"id":"3a7e1126_a7095f2e","line":130,"updated":"2015-12-17 20:53:13.000000000","message":"It\u0027d be interesting, although unsure really how it would scale; it\u0027d of course be a major shift in lots of things and that\u0027s the real tricky part (nova afaik itself used to be this single view of resources, many years ago before it started to split into different projects).","commit_id":"0547d00ae0f41339483af7a8e9ad0549d509d6e1"},{"author":{"_account_id":11564,"name":"Chris Dent","email":"cdent@anticdent.org","username":"chdent"},"change_message_id":"a5338b2e717c91e137ba1d158f65629afa17b111","unresolved":false,"context_lines":[{"line_number":127,"context_line":"   interestring in case of multiple nova compute sharding the ironic-nodes."},{"line_number":128,"context_line":""},{"line_number":129,"context_line":"A better approach to solve the resouce in-consistent view would be to have a"},{"line_number":130,"context_line":"global view of the resources. This global view of resources can serve as a"},{"line_number":131,"context_line":"source of truth for the super-scheduler. Doing so and having a wholesome view"},{"line_number":132,"context_line":"of the cloud the scheduler can select optimal compute depending on the"},{"line_number":133,"context_line":"parameters passed as part of the request. This also gets rid of task to keep"}],"source_content_type":"text/x-rst","patch_set":11,"id":"da6ed579_fe4cbb43","line":130,"updated":"2016-01-13 16:33:22.000000000","message":"There\u0027s some discussion (throughout several currently being reviewed nova-specs) of the concept of a resource classes and providers that are managed separate from the compute nodes. A compute node is just one of many classes of resource. The idea is that the state of these resources will (eventually) be managed globally with clear requirements and methods for keeping the state up to date. I think this is a good medium to long term solution as there\u0027s a relatively clear migration path from where we are now.\n\nFor a really long term solution, though, I don\u0027t think it can scale to the kinds of volumes that we can speculate about in the not to distant future. In that future I think scheduling needs to become more event and message driven and individual resources need to be able to accept events from a global event pool. Instead of reasource tracking we just make sure there\u0027s a superabundance of resources and request specs (constraints) can be declared to be hard (must have) or soft (please try but we\u0027ll cope) and then the event can be tuned to express how many times it is willing to be retried before it will allow failures in soft constraints to equal success. This is a considerable architecture change to how OpenStack does the message bus, from what I understand, but I think is an inevitable requirement.","commit_id":"0547d00ae0f41339483af7a8e9ad0549d509d6e1"},{"author":{"_account_id":1297,"name":"Joshua Harlow","email":"jxharlow@godaddy.com","username":"harlowja"},"change_message_id":"5c2d82eccfd7cc5918efaaaf07961c48d8327d2a","unresolved":false,"context_lines":[{"line_number":131,"context_line":"source of truth for the super-scheduler. Doing so and having a wholesome view"},{"line_number":132,"context_line":"of the cloud the scheduler can select optimal compute depending on the"},{"line_number":133,"context_line":"parameters passed as part of the request. This also gets rid of task to keep"},{"line_number":134,"context_line":"the data source in sync always."},{"line_number":135,"context_line":""},{"line_number":136,"context_line":"This is just one of the discrepancy and there are many of such."},{"line_number":137,"context_line":""}],"source_content_type":"text/x-rst","patch_set":11,"id":"3a7e1126_8736a3e5","line":134,"updated":"2015-12-17 20:53:13.000000000","message":"Agreed, I predict in 10+ years the openstack community and its software will be a very interesting case-study in how or how not to do things in opensource. ha","commit_id":"0547d00ae0f41339483af7a8e9ad0549d509d6e1"},{"author":{"_account_id":19273,"name":"Fabien Hermenier","email":"fabien.hermenier@unice.fr","username":"fhermeni"},"change_message_id":"daf97caa9630e8b7c55ecf3165c3fea3bfd1ce34","unresolved":false,"context_lines":[{"line_number":133,"context_line":"parameters passed as part of the request. This also gets rid of task to keep"},{"line_number":134,"context_line":"the data source in sync always."},{"line_number":135,"context_line":""},{"line_number":136,"context_line":"This is just one of the discrepancy and there are many of such."},{"line_number":137,"context_line":""},{"line_number":138,"context_line":""},{"line_number":139,"context_line":"Core element description"}],"source_content_type":"text/x-rst","patch_set":11,"id":"3a7e1126_9ce5c6a9","line":136,"range":{"start_line":136,"start_character":0,"end_line":136,"end_character":63},"updated":"2015-12-17 10:46:34.000000000","message":"Your point is correct. Every scheduling algorithms I know suppose a consistent view of the resources.\n\nHaving this view shall solve this traditional consistency issue.\nTo still support parallel scheduling, a solution might be to lock nodes that are under the control of an ongoing scheduling phrase to prevent any resource contention.\n\nThe difficulty to provide such a consistent view will mostly depend on the infrastructure scale and the kind of data we want to exhibit. For example, static data (like VM memory allocation) are easy to sync but dynamic data (like the VM %cpu usage that can change over the time) will be very hard to maintain.","commit_id":"0547d00ae0f41339483af7a8e9ad0549d509d6e1"},{"author":{"_account_id":19214,"name":"Vincent Mahe","email":"vincent.mahe@b-com.com","username":"vmahe35"},"change_message_id":"50c555d3652abb62b6e10e896b5ae553fbe00763","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":15,"id":"3aaa91ec_def48688","updated":"2016-06-23 09:59:55.000000000","message":"Note that AT\u0026T presented an Holistic approach at the last Openstack summit (Austin) :\nhttps://www.openstack.org/videos/video/valet-holistic-data-center-optimization-for-openstack\n\nthey defined new HEAT objects to define constraints on a group of resources.\n\nMaybe TOSCA Simple Profile could be an interesting format for describing new types of objects (group of resources, ...) and constraints on it.","commit_id":"12a748dea6fac4ca7c61b7938d2a1f3a1df4dc25"},{"author":{"_account_id":5441,"name":"Andrew Laski","email":"andrew@lascii.com","username":"alaski"},"change_message_id":"a67ea80662686e1c690465522527a0a60a0583ea","unresolved":false,"context_lines":[{"line_number":44,"context_line":"   the resource state is updated by periodic run of the Resource Tracker which"},{"line_number":45,"context_line":"   runs as part of the nova-compute process. The complications become much more"},{"line_number":46,"context_line":"   interesting in case of multiple nova compute sharding the ironic-nodes (for"},{"line_number":47,"context_line":"   high availability)."},{"line_number":48,"context_line":""},{"line_number":49,"context_line":"A better approach to solve this could be to have a holistic view of"},{"line_number":50,"context_line":"resources. This holistic view of resources can serve as a source of"}],"source_content_type":"text/x-rst","patch_set":15,"id":"1af94dfe_f263a93f","line":47,"updated":"2016-03-16 14:21:15.000000000","message":"This case is not specific to Nova\u003c-\u003eIronic, it also occurs when scheduling to virt based compute nodes. The issue is not Nova doesn\u0027t know about Ironic resources, it\u0027s that the scheduler has no \"memory\" when it does something. Two scheduling requests at around the same time could be sent to the same host simply because the scheduler does not track that it just scheduled to that host.\n\nSo this case is not something that needs a more global scheduler to solve, it could be solved completely within the Nova scheduler. And there is work towards doing that.","commit_id":"12a748dea6fac4ca7c61b7938d2a1f3a1df4dc25"}]}