)]}'
{"doc/source/overview_container_sharding.rst":[{"author":{"_account_id":15343,"name":"Tim Burke","email":"tburke@nvidia.com","username":"tburke"},"change_message_id":"0f43d7ac0a91925898ca93593de29bd745f11a7d","unresolved":false,"context_lines":[{"line_number":118,"context_line":""},{"line_number":119,"context_line":"The basic and simplified idea is rather simple. Firstly, to enable container"},{"line_number":120,"context_line":"sharding, pass in a ``X-Container-Sharding: On`` X-Header via either PUT or POST"},{"line_number":121,"context_line":"to the container URL::"},{"line_number":122,"context_line":""},{"line_number":123,"context_line":"  curl -i -H \u0027X-Auth-Token: \u003ctoken\u003e\u0027 -H  \\"},{"line_number":124,"context_line":"  \u0027X-Container-Sharding: On\u0027 \u003curl\u003e/\u003caccount\u003e/\u003ccontainer\u003e -X PUT"}],"source_content_type":"text/x-rst","patch_set":3,"id":"ffe62b97_8301ca07","line":121,"updated":"2017-03-30 00:47:27.000000000","message":"...as a reseller admin, yeah?","commit_id":"2c1aab97c1664ede33aef4d0ef4b38c9ae492e49"},{"author":{"_account_id":15343,"name":"Tim Burke","email":"tburke@nvidia.com","username":"tburke"},"change_message_id":"0f43d7ac0a91925898ca93593de29bd745f11a7d","unresolved":false,"context_lines":[{"line_number":174,"context_line":"Here the sharder has found that object \u0027f\u0027 would be a good first pivot point."},{"line_number":175,"context_line":"So we mark it. The sharder can then shard at this point:"},{"line_number":176,"context_line":""},{"line_number":177,"context_line":".. image:: images/sharding_snip3.png"},{"line_number":178,"context_line":""},{"line_number":179,"context_line":"At this point we\u0027ll pause for a moment, because now you can see that a shard is"},{"line_number":180,"context_line":"given a unique name and placed in a hidden account that maps to the account"}],"source_content_type":"text/x-rst","patch_set":3,"id":"ffe62b97_632da693","line":177,"updated":"2017-03-30 00:47:27.000000000","message":"Still having concerns about length limits...","commit_id":"2c1aab97c1664ede33aef4d0ef4b38c9ae492e49"},{"author":{"_account_id":15343,"name":"Tim Burke","email":"tburke@nvidia.com","username":"tburke"},"change_message_id":"0f43d7ac0a91925898ca93593de29bd745f11a7d","unresolved":false,"context_lines":[{"line_number":216,"context_line":"In the slicing example above, we would find 1 pivot point and then shard it."},{"line_number":217,"context_line":"This works well when sharding is turned on from the start, as you\u0027ll only ever"},{"line_number":218,"context_line":"need to make 1 slice. But on larger containers where more than 1 slice is"},{"line_number":219,"context_line":"required it just wont scale. The rule is we want to get to the sharded state as"},{"line_number":220,"context_line":"quickly as possible, so we need to speed things up, we need to shard in"},{"line_number":221,"context_line":"batches."},{"line_number":222,"context_line":""}],"source_content_type":"text/x-rst","patch_set":3,"id":"ffe62b97_a343ae3e","line":219,"updated":"2017-03-30 00:47:27.000000000","message":"s/wont/won\u0027t/","commit_id":"2c1aab97c1664ede33aef4d0ef4b38c9ae492e49"},{"author":{"_account_id":15343,"name":"Tim Burke","email":"tburke@nvidia.com","username":"tburke"},"change_message_id":"6a65687fd0b2f358139aa9e32b0a814afec21907","unresolved":false,"context_lines":[{"line_number":878,"context_line":"**Phase 1:**"},{"line_number":879,"context_line":""},{"line_number":880,"context_line":"#. Find out if the container really has few enough objects: that is a majority"},{"line_number":881,"context_line":"   quorum of counts from all the primary nodes below the threshold (see below)."},{"line_number":882,"context_line":"#. If there are few enough objects then check the neighbours to see if it’s"},{"line_number":883,"context_line":"   possible to shrink/merge together, again this requires getting a quorum."},{"line_number":884,"context_line":"#. If all comes back successful set some metadata on the 2 containers to mark"}],"source_content_type":"text/x-rst","patch_set":3,"id":"ffe62b97_cca8938d","line":881,"updated":"2017-03-28 01:12:50.000000000","message":"Is it (quorum of count) \u003c threshold or quorum of (count \u003c threshold)? One of those seems much more difficult to obtain...\n\nHere, \"primary nodes\" means... primary for the shard container? Or the primaries for the root container?\n\nThe root container should have a record of the object counts for each shard, right? And they\u0027d have records of the neighbors... but not (necessarily?) whether the neighbors are shrinking/merging :-/","commit_id":"2c1aab97c1664ede33aef4d0ef4b38c9ae492e49"},{"author":{"_account_id":15343,"name":"Tim Burke","email":"tburke@nvidia.com","username":"tburke"},"change_message_id":"6a65687fd0b2f358139aa9e32b0a814afec21907","unresolved":false,"context_lines":[{"line_number":880,"context_line":"#. Find out if the container really has few enough objects: that is a majority"},{"line_number":881,"context_line":"   quorum of counts from all the primary nodes below the threshold (see below)."},{"line_number":882,"context_line":"#. If there are few enough objects then check the neighbours to see if it’s"},{"line_number":883,"context_line":"   possible to shrink/merge together, again this requires getting a quorum."},{"line_number":884,"context_line":"#. If all comes back successful set some metadata on the 2 containers to mark"},{"line_number":885,"context_line":"   intention and so to stop other things shrinking or pivoting into the"},{"line_number":886,"context_line":"   neighbour. The metadata set is:"}],"source_content_type":"text/x-rst","patch_set":3,"id":"ffe62b97_ccdad349","line":883,"updated":"2017-03-28 01:12:50.000000000","message":"Does this also imply getting a quorum on *who our neighbors are*? Like, the upper/lower bounds on them?\n\nAlso, does the \"see if it\u0027s possible\" include checking whether the neighbor is currently busy shrinking/merging?","commit_id":"2c1aab97c1664ede33aef4d0ef4b38c9ae492e49"},{"author":{"_account_id":15343,"name":"Tim Burke","email":"tburke@nvidia.com","username":"tburke"},"change_message_id":"6a65687fd0b2f358139aa9e32b0a814afec21907","unresolved":false,"context_lines":[{"line_number":883,"context_line":"   possible to shrink/merge together, again this requires getting a quorum."},{"line_number":884,"context_line":"#. If all comes back successful set some metadata on the 2 containers to mark"},{"line_number":885,"context_line":"   intention and so to stop other things shrinking or pivoting into the"},{"line_number":886,"context_line":"   neighbour. The metadata set is:"},{"line_number":887,"context_line":""},{"line_number":888,"context_line":"::"},{"line_number":889,"context_line":""}],"source_content_type":"text/x-rst","patch_set":3,"id":"ffe62b97_8ccdeb67","line":886,"updated":"2017-03-28 01:12:50.000000000","message":"So both containers get both pieces of meta data? Or is it\n\n* \u003cneighbor\u003e gets X-Container-Sysmeta-Shard-Shrink: \u003cthis container\u003e and\n* \u003cthis container\u003e gets X-Container-Sysmeta-Shard-Merge: \u003cneighbour\u003e\n\n?","commit_id":"2c1aab97c1664ede33aef4d0ef4b38c9ae492e49"}]}