)]}'
{"/COMMIT_MSG":[{"author":{"_account_id":15171,"name":"Eric Larson","email":"eric.larson@rackspace.com","username":"elarson"},"change_message_id":"ec8bc49bf6430a13b8a0770fd7fda687ba2b1a6c","unresolved":false,"context_lines":[{"line_number":6,"context_line":""},{"line_number":7,"context_line":"A Worker Model for Designate"},{"line_number":8,"context_line":""},{"line_number":9,"context_line":"- A proposal for some pretty large changes to change"},{"line_number":10,"context_line":"Designate\u0027s current workflow/architecture to more of"},{"line_number":11,"context_line":"a producer-consumer model."},{"line_number":12,"context_line":""}],"source_content_type":"text/x-gerrit-commit-message","patch_set":6,"id":"fa0719c6_6e9a2216","line":9,"updated":"2016-03-21 22:14:22.000000000","message":"NIT!!!! We don\u0027t really need the dash here since we don\u0027t have a list ;)","commit_id":"6defefdf3c0cf83d0c36875986a47ab2c78c50e0"}],"specs/mitaka/worker-model.rst":[{"author":{"_account_id":16962,"name":"Federico Ceratto","email":"federico.ceratto@gmail.com","username":"ceratto"},"change_message_id":"b4f3bfeae759abf67be281d96f20475229849361","unresolved":false,"context_lines":[{"line_number":73,"context_line":"These are the services that would remain present:"},{"line_number":74,"context_line":""},{"line_number":75,"context_line":"-  ``designate-api`` - To receive JSON and parse it for Designate"},{"line_number":76,"context_line":"-  ``designate-central`` - To do validation/storage of zone/record data"},{"line_number":77,"context_line":"-  ``designate-mdns`` - To **only** AXFRs from Designate\u0027s database"},{"line_number":78,"context_line":"-  ``designate-worker`` - Any and all tasks that Designate needs to"},{"line_number":79,"context_line":"   produce state on nameservers"}],"source_content_type":"text/x-rst","patch_set":2,"id":"fa69d971_f8780f80","line":76,"updated":"2016-01-11 16:21:07.000000000","message":"Central produces tasks for the workers as well.","commit_id":"e850c8c7a9bd0bcf76b06713c853335b0f0416de"},{"author":{"_account_id":16962,"name":"Federico Ceratto","email":"federico.ceratto@gmail.com","username":"ceratto"},"change_message_id":"b4f3bfeae759abf67be281d96f20475229849361","unresolved":false,"context_lines":[{"line_number":77,"context_line":"-  ``designate-mdns`` - To **only** AXFRs from Designate\u0027s database"},{"line_number":78,"context_line":"-  ``designate-worker`` - Any and all tasks that Designate needs to"},{"line_number":79,"context_line":"   produce state on nameservers"},{"line_number":80,"context_line":"-  ``designate-producer`` - To run periodic/timed jobs, and produce work"},{"line_number":81,"context_line":"   for``designate-worker`` that is out the normal path of API operations."},{"line_number":82,"context_line":"   For example: Periodic recovery."},{"line_number":83,"context_line":""}],"source_content_type":"text/x-rst","patch_set":2,"id":"fa69d971_d8dbcb7a","line":80,"updated":"2016-01-11 16:21:07.000000000","message":"See comment above (it sounds like producer is the only component emitting tasks for worker)","commit_id":"e850c8c7a9bd0bcf76b06713c853335b0f0416de"},{"author":{"_account_id":11662,"name":"Paul Glass","email":"paul.glass@rackspace.com","username":"pglass"},"change_message_id":"a30ef178d0929ccf15c6a504fefe7b1d9849f84b","unresolved":false,"context_lines":[{"line_number":96,"context_line":"   ``tooz`` using the cache."},{"line_number":97,"context_line":""},{"line_number":98,"context_line":"Designate Worker"},{"line_number":99,"context_line":"----------------"},{"line_number":100,"context_line":""},{"line_number":101,"context_line":"The scope of ``designate-worker``\u0027s duties are essentially any and all tasks"},{"line_number":102,"context_line":"that Designate needs to take action to perform. For example:"}],"source_content_type":"text/x-rst","patch_set":2,"id":"3a7e1126_1576b63e","line":99,"updated":"2016-01-06 17:36:30.000000000","message":"I think it\u0027s more commonly called \"producer-consumer\"? \"Worker\" is fine too, but now is the time to change names if we want to.","commit_id":"e850c8c7a9bd0bcf76b06713c853335b0f0416de"},{"author":{"_account_id":16962,"name":"Federico Ceratto","email":"federico.ceratto@gmail.com","username":"ceratto"},"change_message_id":"b4f3bfeae759abf67be281d96f20475229849361","unresolved":false,"context_lines":[{"line_number":96,"context_line":"   ``tooz`` using the cache."},{"line_number":97,"context_line":""},{"line_number":98,"context_line":"Designate Worker"},{"line_number":99,"context_line":"----------------"},{"line_number":100,"context_line":""},{"line_number":101,"context_line":"The scope of ``designate-worker``\u0027s duties are essentially any and all tasks"},{"line_number":102,"context_line":"that Designate needs to take action to perform. For example:"}],"source_content_type":"text/x-rst","patch_set":2,"id":"fa69d971_33000801","line":99,"in_reply_to":"3a7e1126_1576b63e","updated":"2016-01-11 16:21:07.000000000","message":"\"producer-consumer\" describes the messaging pattern (and we already have such pattern in use)\nIt might be misleading to name components that do many things like the Worker after the pattern.\n\nGiven that Central is also a \"producer\", maybe designate-producer should be named designate-scheduler?","commit_id":"e850c8c7a9bd0bcf76b06713c853335b0f0416de"},{"author":{"_account_id":11662,"name":"Paul Glass","email":"paul.glass@rackspace.com","username":"pglass"},"change_message_id":"a30ef178d0929ccf15c6a504fefe7b1d9849f84b","unresolved":false,"context_lines":[{"line_number":168,"context_line":"You could get even smarter about the markers that you drop in a cache for these"},{"line_number":169,"context_line":"tasks. For example, on a zone update, you could drop a key in the cache of the"},{"line_number":170,"context_line":"nature ``zoneupdate-foo.com.``, and other if other zoneupdate tasks for the same"},{"line_number":171,"context_line":"zone see that key, they could know to throw away their job and move on."},{"line_number":172,"context_line":""},{"line_number":173,"context_line":"designate-mdns changes"},{"line_number":174,"context_line":"~~~~~~~~~~~~~~~~~~~~~~"}],"source_content_type":"text/x-rst","patch_set":2,"id":"fa69d971_d95d0d48","line":171,"updated":"2016-01-06 17:36:30.000000000","message":"We talked about having a jobs table at one point. The advantage there is that it\u0027s more monitor-able as you can easily check *which* jobs are in progress and which failed.\n\nI don\u0027t know if the overhead of the jobs table is worth it. You\u0027d need to have some method of cleaning out that table (like another periodic task), and additional database traffic can affect performance elsewhere.","commit_id":"e850c8c7a9bd0bcf76b06713c853335b0f0416de"},{"author":{"_account_id":8099,"name":"Graham Hayes","email":"gr@ham.ie","username":"graham"},"change_message_id":"d8657e38463b841280693fb0b9df9236664dbbfd","unresolved":false,"context_lines":[{"line_number":168,"context_line":"You could get even smarter about the markers that you drop in a cache for these"},{"line_number":169,"context_line":"tasks. For example, on a zone update, you could drop a key in the cache of the"},{"line_number":170,"context_line":"nature ``zoneupdate-foo.com.``, and other if other zoneupdate tasks for the same"},{"line_number":171,"context_line":"zone see that key, they could know to throw away their job and move on."},{"line_number":172,"context_line":""},{"line_number":173,"context_line":"designate-mdns changes"},{"line_number":174,"context_line":"~~~~~~~~~~~~~~~~~~~~~~"}],"source_content_type":"text/x-rst","patch_set":2,"id":"fa69d971_61667c52","line":171,"updated":"2016-01-06 19:22:44.000000000","message":"Yeah, I agree with Paul that a jobs table would be problematic.\n\nThe problem with the example is that the zoneupdate-foo.com. job could have updated 2 of the 3 targets,\nand still be in-progress. if the new job gets dropped on the floor we might only update one of the 3 \ntargets to the actual new information.\n\nWe will need to consider each task quite carefully before starting to use this optimisation.","commit_id":"e850c8c7a9bd0bcf76b06713c853335b0f0416de"},{"author":{"_account_id":8099,"name":"Graham Hayes","email":"gr@ham.ie","username":"graham"},"change_message_id":"d8657e38463b841280693fb0b9df9236664dbbfd","unresolved":false,"context_lines":[{"line_number":184,"context_line":"nameservers Designate manages."},{"line_number":185,"context_line":""},{"line_number":186,"context_line":"As a side-note, this would make this service much easier to optimize, or even"},{"line_number":187,"context_line":"rewrite in a faster programming language."},{"line_number":188,"context_line":""},{"line_number":189,"context_line":"Designate Producer"},{"line_number":190,"context_line":"------------------"}],"source_content_type":"text/x-rst","patch_set":2,"id":"fa69d971_0159c08d","line":187,"updated":"2016-01-06 19:22:44.000000000","message":"++","commit_id":"e850c8c7a9bd0bcf76b06713c853335b0f0416de"},{"author":{"_account_id":8099,"name":"Graham Hayes","email":"gr@ham.ie","username":"graham"},"change_message_id":"d8657e38463b841280693fb0b9df9236664dbbfd","unresolved":false,"context_lines":[{"line_number":302,"context_line":"have the worker replace them on the queue after a sleep, and let another worker"},{"line_number":303,"context_line":"try the job. If ``designate-producer`` sees that a job (probably hashed) is"},{"line_number":304,"context_line":"still in progress, it will not create another instance of the same task."},{"line_number":305,"context_line":""},{"line_number":306,"context_line":"Queue Priority"},{"line_number":307,"context_line":"--------------"},{"line_number":308,"context_line":""}],"source_content_type":"text/x-rst","patch_set":2,"id":"fa69d971_215c047c","line":305,"updated":"2016-01-06 19:22:44.000000000","message":"I am not sold on this section. I think we need to really dig into this.","commit_id":"e850c8c7a9bd0bcf76b06713c853335b0f0416de"},{"author":{"_account_id":16962,"name":"Federico Ceratto","email":"federico.ceratto@gmail.com","username":"ceratto"},"change_message_id":"cc58842b977a929ee73412dc2d9d9792334dd47c","unresolved":false,"context_lines":[{"line_number":318,"context_line":"workers can focus on high priority work first, ensuring the most important"},{"line_number":319,"context_line":"parts of Designate\u0027s job get done first."},{"line_number":320,"context_line":""},{"line_number":321,"context_line":"Some workers may be configured to only work on the low-priority queue."},{"line_number":322,"context_line":""},{"line_number":323,"context_line":"Or perhaps we have queues/exchanges for each type of task, this would be an"},{"line_number":324,"context_line":"optimal way to monitor the health of different types of tasks, and isolate the"}],"source_content_type":"text/x-rst","patch_set":2,"id":"fa69d971_7d665cee","line":321,"updated":"2016-01-11 14:37:02.000000000","message":"How is this beneficial?","commit_id":"e850c8c7a9bd0bcf76b06713c853335b0f0416de"},{"author":{"_account_id":8099,"name":"Graham Hayes","email":"gr@ham.ie","username":"graham"},"change_message_id":"d8657e38463b841280693fb0b9df9236664dbbfd","unresolved":false,"context_lines":[{"line_number":323,"context_line":"Or perhaps we have queues/exchanges for each type of task, this would be an"},{"line_number":324,"context_line":"optimal way to monitor the health of different types of tasks, and isolate the"},{"line_number":325,"context_line":"sometimes long-running tasks that periodic timers will produce from the"},{"line_number":326,"context_line":"relatively quicker, and more important CRUD operations."},{"line_number":327,"context_line":""},{"line_number":328,"context_line":""},{"line_number":329,"context_line":"Work Items"}],"source_content_type":"text/x-rst","patch_set":2,"id":"fa69d971_c1572856","line":326,"updated":"2016-01-06 19:22:44.000000000","message":"Per task queues sound good.\n\nWe need to see if this is doable in oslo.messaging, or if we need to implement it.","commit_id":"e850c8c7a9bd0bcf76b06713c853335b0f0416de"},{"author":{"_account_id":11662,"name":"Paul Glass","email":"paul.glass@rackspace.com","username":"pglass"},"change_message_id":"a30ef178d0929ccf15c6a504fefe7b1d9849f84b","unresolved":false,"context_lines":[{"line_number":323,"context_line":"Or perhaps we have queues/exchanges for each type of task, this would be an"},{"line_number":324,"context_line":"optimal way to monitor the health of different types of tasks, and isolate the"},{"line_number":325,"context_line":"sometimes long-running tasks that periodic timers will produce from the"},{"line_number":326,"context_line":"relatively quicker, and more important CRUD operations."},{"line_number":327,"context_line":""},{"line_number":328,"context_line":""},{"line_number":329,"context_line":"Work Items"}],"source_content_type":"text/x-rst","patch_set":2,"id":"fa69d971_99d6b5ef","line":326,"updated":"2016-01-06 17:36:30.000000000","message":"This is important. We don\u0027t want new changes to be blocked because workers were busy with periodic sync/recovery tasks.\n\nHaving a different queue per task type seems better overall from a monitoring and flexibility perspective. Limiting ourselves to two queues limits us to only two priority levels for tasks. There might be other task types we add later that are between the priorities of periodic tasks and regular zone creates.\n\nOperationally, it would be really nice to *avoid* needing different configurations for worker nodes. We should think a bit more about it. One way we could effectively support queues of varying priority (without needing per-node configuration) is to have workers randomly grab jobs from different queues, according to a set of weights. Workers would grab tasks from higher-weighted queues more frequently, and from lower-weighted queues less frequently.","commit_id":"e850c8c7a9bd0bcf76b06713c853335b0f0416de"},{"author":{"_account_id":8099,"name":"Graham Hayes","email":"gr@ham.ie","username":"graham"},"change_message_id":"d997e288785c0fdb3f5610a57f1f3c88d93b1170","unresolved":false,"context_lines":[{"line_number":343,"context_line":"  production using cache or some other method in ``designate-producer``"},{"line_number":344,"context_line":"- Delete ``pool-manager`` and ``zone-manager``"},{"line_number":345,"context_line":"- Profit!!!"},{"line_number":346,"context_line":""},{"line_number":347,"context_line":"Milestones"},{"line_number":348,"context_line":"----------"},{"line_number":349,"context_line":""}],"source_content_type":"text/x-rst","patch_set":2,"id":"fa69d971_47293d9a","line":346,"updated":"2016-01-11 14:52:41.000000000","message":"I think that is a good idea - it can go in the main blurb.\n\nFor the Log messages, that should be OK - we attach the request_id to every log message, so corellation across machines is easy to do.\nWe already need to do that for api, central, pool-manager, zone-manager and mdns.","commit_id":"e850c8c7a9bd0bcf76b06713c853335b0f0416de"},{"author":{"_account_id":16962,"name":"Federico Ceratto","email":"federico.ceratto@gmail.com","username":"ceratto"},"change_message_id":"cc58842b977a929ee73412dc2d9d9792334dd47c","unresolved":false,"context_lines":[{"line_number":343,"context_line":"  production using cache or some other method in ``designate-producer``"},{"line_number":344,"context_line":"- Delete ``pool-manager`` and ``zone-manager``"},{"line_number":345,"context_line":"- Profit!!!"},{"line_number":346,"context_line":""},{"line_number":347,"context_line":"Milestones"},{"line_number":348,"context_line":"----------"},{"line_number":349,"context_line":""}],"source_content_type":"text/x-rst","patch_set":2,"id":"fa69d971_dd0508c2","line":346,"updated":"2016-01-11 14:37:02.000000000","message":"Would it be useful to provide a summary of benefits and drawbacks for the end users?\n\n- Increased scalability: the service can be rolled out with a minimum of two workers only. Workers can be added over time, as needed.\n- Increased reliability: the workers can do both pool-manager and zone-manager work and can survive losing N-1 workers. \n(however we are introducing producers, what will be the producers failure mode?)\n\n- Slightly more difficult log analysis. Workers handle a miscellanea of tasks as fire-and-forget: ops will have to gather logs from every worker and correlate them to understand what task was being performed, when, and on which worker.","commit_id":"e850c8c7a9bd0bcf76b06713c853335b0f0416de"},{"author":{"_account_id":8099,"name":"Graham Hayes","email":"gr@ham.ie","username":"graham"},"change_message_id":"d997e288785c0fdb3f5610a57f1f3c88d93b1170","unresolved":false,"context_lines":[{"line_number":348,"context_line":"----------"},{"line_number":349,"context_line":""},{"line_number":350,"context_line":"Target Milestone for completion:"},{"line_number":351,"context_line":"  Mitaka-3 (¯\\_(ツ)_/¯)"},{"line_number":352,"context_line":""},{"line_number":353,"context_line":"Author(s)"},{"line_number":354,"context_line":"---------"}],"source_content_type":"text/x-rst","patch_set":2,"id":"fa69d971_e7187186","line":351,"updated":"2016-01-11 14:52:41.000000000","message":"This may have to .... slip. depends on the impact.","commit_id":"e850c8c7a9bd0bcf76b06713c853335b0f0416de"},{"author":{"_account_id":16962,"name":"Federico Ceratto","email":"federico.ceratto@gmail.com","username":"ceratto"},"change_message_id":"ab687fb5dab4c5d5ca69f651c203e5c1d76236a4","unresolved":false,"context_lines":[{"line_number":70,"context_line":""},{"line_number":71,"context_line":"To the end user, the results of this change would be relatively simple:"},{"line_number":72,"context_line":""},{"line_number":73,"context_line":"- Scalability limited only by DNS servers, datastores, and queues. If at any"},{"line_number":74,"context_line":"point Designate starts to slow down in some aspect, unless there\u0027s an issue"},{"line_number":75,"context_line":"with those services listed above, the problem can be solved by throwing more"},{"line_number":76,"context_line":"Designate processes at the problem."}],"source_content_type":"text/x-rst","patch_set":4,"id":"9a68dd71_465247df","line":73,"updated":"2016-01-20 15:35:26.000000000","message":"s/Designate processes/workers/ ?","commit_id":"40beb75e0decc191b5bf63eeb216c713d804f057"},{"author":{"_account_id":8174,"name":"Tim Simmons","email":"tim.r.simmons@gmail.com","username":"TimSimmons"},"change_message_id":"38f0f1fa651e313af28213546ee0ee73269c15a3","unresolved":false,"context_lines":[{"line_number":70,"context_line":""},{"line_number":71,"context_line":"To the end user, the results of this change would be relatively simple:"},{"line_number":72,"context_line":""},{"line_number":73,"context_line":"- Scalability limited only by DNS servers, datastores, and queues. If at any"},{"line_number":74,"context_line":"point Designate starts to slow down in some aspect, unless there\u0027s an issue"},{"line_number":75,"context_line":"with those services listed above, the problem can be solved by throwing more"},{"line_number":76,"context_line":"Designate processes at the problem."}],"source_content_type":"text/x-rst","patch_set":4,"id":"9a68dd71_790045ab","line":73,"in_reply_to":"9a68dd71_465247df","updated":"2016-01-20 16:04:34.000000000","message":"Scaling workers would certainly be a way to accomplish more of certain types of work. But more workers won\u0027t improve API performance, that\u0027s API/Central. More workers won\u0027t improve outbound AXFRs, MiniDNS will. But all of the components in this case (assuming the api/mdns mutli-process listening on the same port works) horizontally scale perfectly, without one accepting any single responsibility like the leader election in pool/zone manager, or with any unequal division of work.","commit_id":"40beb75e0decc191b5bf63eeb216c713d804f057"},{"author":{"_account_id":16962,"name":"Federico Ceratto","email":"federico.ceratto@gmail.com","username":"ceratto"},"change_message_id":"ab687fb5dab4c5d5ca69f651c203e5c1d76236a4","unresolved":false,"context_lines":[{"line_number":84,"context_line":"or disappear altogether."},{"line_number":85,"context_line":"- Operationally, this opens the door for simpler scenarios for small"},{"line_number":86,"context_line":"deployments where a customer need only scale a couple components (even on"},{"line_number":87,"context_line":"the same machine) to get more performance. You could even go so far as to"},{"line_number":88,"context_line":"deploy app nodes that only shared datastore (db, cache) and had their own"},{"line_number":89,"context_line":"designate components and queues."},{"line_number":90,"context_line":""}],"source_content_type":"text/x-rst","patch_set":4,"id":"9a68dd71_264b935b","line":87,"updated":"2016-01-20 15:35:26.000000000","message":"I would two workers (on two different hosts) as a smallest case scenario. (It\u0027s still a pretty small footprint)","commit_id":"40beb75e0decc191b5bf63eeb216c713d804f057"},{"author":{"_account_id":8099,"name":"Graham Hayes","email":"gr@ham.ie","username":"graham"},"change_message_id":"14d546267a8c9cc9ba4c759bcadc975173ba9a68","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":6,"id":"fa0719c6_7db66c8b","updated":"2016-03-21 15:38:14.000000000","message":"mitaka -\u003e newton","commit_id":"6defefdf3c0cf83d0c36875986a47ab2c78c50e0"},{"author":{"_account_id":15171,"name":"Eric Larson","email":"eric.larson@rackspace.com","username":"elarson"},"change_message_id":"ec8bc49bf6430a13b8a0770fd7fda687ba2b1a6c","unresolved":false,"context_lines":[{"line_number":38,"context_line":"  operations for that set of zones gets heavy, a single zone manager process"},{"line_number":39,"context_line":"  could become overwhelmed."},{"line_number":40,"context_line":"- We rely on ``tooz`` to manage the extremely delicate task of ensuring"},{"line_number":41,"context_line":"  balance and coverage of all zones by zone manager processes."},{"line_number":42,"context_line":"- Certain work (export) that\u0027s in the critical path of operations has already"},{"line_number":43,"context_line":"  crept into the component that wasn\u0027t really meant for that. As a substitute"},{"line_number":44,"context_line":"  for proper workers, the zone-manager is looking like the current answer."}],"source_content_type":"text/x-rst","patch_set":6,"id":"fa0719c6_01cff77f","line":41,"updated":"2016-03-21 22:14:22.000000000","message":"Tooz doesn\u0027t seem like that bad an idea to me, so maybe it would be good to be slightly more specific here. For example, tooz has primarily proven to be most reliable with zookeeper, even though it is the most difficult driver to operate, with other drivers (redis, mysql, etc.) not working at all. \n\n#thanksoslobama","commit_id":"6defefdf3c0cf83d0c36875986a47ab2c78c50e0"},{"author":{"_account_id":15171,"name":"Eric Larson","email":"eric.larson@rackspace.com","username":"elarson"},"change_message_id":"ec8bc49bf6430a13b8a0770fd7fda687ba2b1a6c","unresolved":false,"context_lines":[{"line_number":72,"context_line":""},{"line_number":73,"context_line":"- Scalability limited only by DNS servers, datastores, and queues. If at any"},{"line_number":74,"context_line":"  point Designate starts to slow down in some aspect, unless there\u0027s an issue"},{"line_number":75,"context_line":"  with those services listed above, the problem can be solved by throwing more"},{"line_number":76,"context_line":"  Designate processes at the problem."},{"line_number":77,"context_line":"- Fault tolerance. One or more Designate processes dying would be inivisible"},{"line_number":78,"context_line":"  in almost every case. An operator wouldn\u0027t have to fear a certain process"}],"source_content_type":"text/x-rst","patch_set":6,"id":"fa0719c6_017d371c","line":75,"updated":"2016-03-21 22:14:22.000000000","message":"s/throwing/adding \n\nSounds more \"enterprisey\" that way ;)","commit_id":"6defefdf3c0cf83d0c36875986a47ab2c78c50e0"},{"author":{"_account_id":15171,"name":"Eric Larson","email":"eric.larson@rackspace.com","username":"elarson"},"change_message_id":"ec8bc49bf6430a13b8a0770fd7fda687ba2b1a6c","unresolved":false,"context_lines":[{"line_number":182,"context_line":"An API customer creates many recordsets very quickly. The work being dispatched"},{"line_number":183,"context_line":"to ``designate-worker`` processes would go a lot of different places, and one of"},{"line_number":184,"context_line":"the first updates to actually reach a nameserver might contain all the changes"},{"line_number":185,"context_line":"necessary to bring the zone up-to-date. The other tasks being worked should"},{"line_number":186,"context_line":"check before they send their NOTIFY that the state is still behind, and check"},{"line_number":187,"context_line":"again after they\u0027ve sent their NOTIFY, but before they\u0027ve began polling, so"},{"line_number":188,"context_line":"that they can cut down on unnecessary work for themselves, and the nameservers."}],"source_content_type":"text/x-rst","patch_set":6,"id":"fa0719c6_a103ab72","line":185,"range":{"start_line":185,"start_character":68,"end_line":185,"end_character":69},"updated":"2016-03-21 22:14:22.000000000","message":"worked on?","commit_id":"6defefdf3c0cf83d0c36875986a47ab2c78c50e0"},{"author":{"_account_id":15171,"name":"Eric Larson","email":"eric.larson@rackspace.com","username":"elarson"},"change_message_id":"ec8bc49bf6430a13b8a0770fd7fda687ba2b1a6c","unresolved":false,"context_lines":[{"line_number":253,"context_line":"processes will put on the queue."},{"line_number":254,"context_line":""},{"line_number":255,"context_line":"Dividing work"},{"line_number":256,"context_line":"~~~~~~~~~~~~~"},{"line_number":257,"context_line":""},{"line_number":258,"context_line":"To explain more clearly, the biggest problem we have in this service is making"},{"line_number":259,"context_line":"it fault-tolerant, but not duplicating work for ``designate-worker`` processes"}],"source_content_type":"text/x-rst","patch_set":6,"id":"fa0719c6_61f35374","line":256,"updated":"2016-03-21 22:14:22.000000000","message":"Can you make this a subheading as this really extends the paragraph in the previous section.","commit_id":"6defefdf3c0cf83d0c36875986a47ab2c78c50e0"}],"specs/newton/worker-model.rst":[{"author":{"_account_id":15699,"name":"sonu","email":"sonu.bhumca11@gmail.com","username":"sonu.kumar"},"change_message_id":"d52cc4128f0603d10c4b1e3845c64c02c7e9caa3","unresolved":false,"context_lines":[{"line_number":32,"context_line":"- Pool Managers are tied to one pool"},{"line_number":33,"context_line":""},{"line_number":34,"context_line":"``designate-zone-manager`` does a good job at executing periodic timers for the"},{"line_number":35,"context_line":"zones it managers. However:"},{"line_number":36,"context_line":""},{"line_number":37,"context_line":"- One zone-manager process is responsible for a certain set of zones, if the"},{"line_number":38,"context_line":"  operations for that set of zones gets heavy, a single zone manager process"}],"source_content_type":"text/x-rst","patch_set":7,"id":"ba0121b8_03b4fbce","line":35,"range":{"start_line":35,"start_character":9,"end_line":35,"end_character":17},"updated":"2016-03-30 09:09:16.000000000","message":"managers should be changed to manages.","commit_id":"65410e9d2f14928555c804656c3fd8539b8aaa2a"},{"author":{"_account_id":15171,"name":"Eric Larson","email":"eric.larson@rackspace.com","username":"elarson"},"change_message_id":"c0c4ca8abfbd9ce7ae130d655b572381b4a96812","unresolved":false,"context_lines":[{"line_number":32,"context_line":"- Pool Managers are tied to one pool"},{"line_number":33,"context_line":""},{"line_number":34,"context_line":"``designate-zone-manager`` does a good job at executing periodic timers for the"},{"line_number":35,"context_line":"zones it managers. However:"},{"line_number":36,"context_line":""},{"line_number":37,"context_line":"- One zone-manager process is responsible for a certain set of zones, if the"},{"line_number":38,"context_line":"  operations for that set of zones gets heavy, a single zone manager process"}],"source_content_type":"text/x-rst","patch_set":7,"id":"dab17558_4012afb5","line":35,"range":{"start_line":35,"start_character":9,"end_line":35,"end_character":17},"in_reply_to":"ba0121b8_03b4fbce","updated":"2016-05-06 18:54:15.000000000","message":"Done","commit_id":"65410e9d2f14928555c804656c3fd8539b8aaa2a"},{"author":{"_account_id":14926,"name":"Jeff Feng","email":"jianhua@us.ibm.com","username":"jhfeng"},"change_message_id":"d910268261eb3e13b43d54d8b6b611e34ea4d0fc","unresolved":false,"context_lines":[{"line_number":96,"context_line":""},{"line_number":97,"context_line":"-  ``designate-api`` - To receive JSON and parse it for Designate"},{"line_number":98,"context_line":"-  ``designate-central`` - To do validation/storage of zone/record data and"},{"line_number":99,"context_line":"   send send CRUD tasks to ``designate-worker``."},{"line_number":100,"context_line":"-  ``designate-mdns`` - To **only** perfrom AXFRs from Designate\u0027s database"},{"line_number":101,"context_line":"-  ``designate-worker`` - Any and all tasks that Designate needs to"},{"line_number":102,"context_line":"   produce state on nameservers"}],"source_content_type":"text/x-rst","patch_set":7,"id":"dab17558_d6a7914d","line":99,"range":{"start_line":99,"start_character":8,"end_line":99,"end_character":12},"updated":"2016-05-06 16:50:38.000000000","message":"remove one \u0027send\u0027","commit_id":"65410e9d2f14928555c804656c3fd8539b8aaa2a"},{"author":{"_account_id":15171,"name":"Eric Larson","email":"eric.larson@rackspace.com","username":"elarson"},"change_message_id":"c0c4ca8abfbd9ce7ae130d655b572381b4a96812","unresolved":false,"context_lines":[{"line_number":96,"context_line":""},{"line_number":97,"context_line":"-  ``designate-api`` - To receive JSON and parse it for Designate"},{"line_number":98,"context_line":"-  ``designate-central`` - To do validation/storage of zone/record data and"},{"line_number":99,"context_line":"   send send CRUD tasks to ``designate-worker``."},{"line_number":100,"context_line":"-  ``designate-mdns`` - To **only** perfrom AXFRs from Designate\u0027s database"},{"line_number":101,"context_line":"-  ``designate-worker`` - Any and all tasks that Designate needs to"},{"line_number":102,"context_line":"   produce state on nameservers"}],"source_content_type":"text/x-rst","patch_set":7,"id":"dab17558_2017aba4","line":99,"range":{"start_line":99,"start_character":8,"end_line":99,"end_character":12},"in_reply_to":"dab17558_d6a7914d","updated":"2016-05-06 18:54:15.000000000","message":"Done","commit_id":"65410e9d2f14928555c804656c3fd8539b8aaa2a"},{"author":{"_account_id":15699,"name":"sonu","email":"sonu.bhumca11@gmail.com","username":"sonu.kumar"},"change_message_id":"d52cc4128f0603d10c4b1e3845c64c02c7e9caa3","unresolved":false,"context_lines":[{"line_number":180,"context_line":""},{"line_number":181,"context_line":"As an example:"},{"line_number":182,"context_line":"An API customer creates many recordsets very quickly. The work being dispatched"},{"line_number":183,"context_line":"to ``designate-worker`` processes would go a lot of different places, and one of"},{"line_number":184,"context_line":"the first updates to actually reach a nameserver might contain all the changes"},{"line_number":185,"context_line":"necessary to bring the zone up-to-date. The other tasks being worked should"},{"line_number":186,"context_line":"check before they send their NOTIFY that the state is still behind, and check"}],"source_content_type":"text/x-rst","patch_set":7,"id":"ba0121b8_1c0e193b","line":183,"range":{"start_line":183,"start_character":42,"end_line":183,"end_character":43},"updated":"2016-03-30 09:09:16.000000000","message":"here the statement should be \" processes would go to a lot of different places,\" .","commit_id":"65410e9d2f14928555c804656c3fd8539b8aaa2a"},{"author":{"_account_id":15171,"name":"Eric Larson","email":"eric.larson@rackspace.com","username":"elarson"},"change_message_id":"c0c4ca8abfbd9ce7ae130d655b572381b4a96812","unresolved":false,"context_lines":[{"line_number":180,"context_line":""},{"line_number":181,"context_line":"As an example:"},{"line_number":182,"context_line":"An API customer creates many recordsets very quickly. The work being dispatched"},{"line_number":183,"context_line":"to ``designate-worker`` processes would go a lot of different places, and one of"},{"line_number":184,"context_line":"the first updates to actually reach a nameserver might contain all the changes"},{"line_number":185,"context_line":"necessary to bring the zone up-to-date. The other tasks being worked should"},{"line_number":186,"context_line":"check before they send their NOTIFY that the state is still behind, and check"}],"source_content_type":"text/x-rst","patch_set":7,"id":"dab17558_c0269f15","line":183,"range":{"start_line":183,"start_character":42,"end_line":183,"end_character":43},"in_reply_to":"ba0121b8_1c0e193b","updated":"2016-05-06 18:54:15.000000000","message":"Done","commit_id":"65410e9d2f14928555c804656c3fd8539b8aaa2a"}],"specs/template.rst":[{"author":{"_account_id":8099,"name":"Graham Hayes","email":"gr@ham.ie","username":"graham"},"change_message_id":"14d546267a8c9cc9ba4c759bcadc975173ba9a68","unresolved":false,"context_lines":[{"line_number":172,"context_line":""},{"line_number":173,"context_line":"Upgrade Implications"},{"line_number":174,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"},{"line_number":175,"context_line":""},{"line_number":176,"context_line":"Does the spec introduce a change for those running the current, or an older"},{"line_number":177,"context_line":"version of Designate? If so, describe the change(s)."}],"source_content_type":"text/x-rst","patch_set":6,"id":"fa0719c6_5dbba872","line":175,"updated":"2016-03-21 15:38:14.000000000","message":"This is the section that we need to work on now.\n\nI would like to see a upgrade path (even roughly) outlined here.\n\nthe plan will probably not survive contact with the real world, but we need to keep it in mind, \nand by documenting it, when we notice something that won\u0027t fit in the plan, we can adapt the plan.","commit_id":"6defefdf3c0cf83d0c36875986a47ab2c78c50e0"},{"author":{"_account_id":15171,"name":"Eric Larson","email":"eric.larson@rackspace.com","username":"elarson"},"change_message_id":"ec8bc49bf6430a13b8a0770fd7fda687ba2b1a6c","unresolved":false,"context_lines":[{"line_number":172,"context_line":""},{"line_number":173,"context_line":"Upgrade Implications"},{"line_number":174,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"},{"line_number":175,"context_line":""},{"line_number":176,"context_line":"Does the spec introduce a change for those running the current, or an older"},{"line_number":177,"context_line":"version of Designate? If so, describe the change(s)."}],"source_content_type":"text/x-rst","patch_set":6,"id":"fa0719c6_04142542","line":175,"in_reply_to":"fa0719c6_5dbba872","updated":"2016-03-21 22:14:22.000000000","message":"+1 The thing I\u0027d think would happen would be that the pool manager and zone manager processes essentially just become workers and producer respectively. Might be hairy, but offhand, that seems like one method that doesn\u0027t destroy an operator\u0027s config management world in one fail swoop.","commit_id":"6defefdf3c0cf83d0c36875986a47ab2c78c50e0"}]}