[sheepdog] questions for sheepdog fresh

[saxion]

Hello everyone, I'm learning distributed storage system sheepdog.
But I have some questions about sheepdog after I googled what I want to know.
As far as I know, consistent hashing is used in sheepdog for data distribution.
For example, a sheepdog cluster with 3 repica policy, an object OID will be mapped to host1, hosti and hostj.

My question is how we get those three host? For every object, an oid will be mapped to(using consistent hashing) only one host(host1 for example), how to select the other two host(hosti and hostj)? Or is there any details I got wrong idea?

And, when host1 is down, how about the data rebalance/recovery, how to choose the right host to make sure there are 3 copied for every object?

Thanks.

saxion

[Hitoshi Mitake]

Post by saxion
Hello everyone, I'm learning distributed storage system sheepdog.
But I have some questions about sheepdog after I googled what I want to know.
As far as I know, consistent hashing is used in sheepdog for data distribution.
For example, a sheepdog cluster with 3 repica policy, an object OID will be mapped to host1, hosti and hostj.
My question is how we get those three host? For every object, an oid will be mapped to(using consistent hashing) only one host(host1 for example), how to select the other two host(hosti and hostj)? Or is there any details I got wrong idea?

Simply selecting three continuing nodes from the vnodes. You can find
the logic here:
https://github.com/sheepdog/sheepdog/blob/master/include/sheep.h#L109

Post by saxion
And, when host1 is down, how about the data rebalance/recovery, how to choose the right host to make sure there are 3 copied for every object?

The vnodes are updated during membership change (e.g. join and leave).
The membership is kept consistent for all living nodes by the atomic
broadcast functionality of corosync or zookeeper. So every living node
can have identical vnodes.

Post by saxion
Thanks.
saxion
--
sheepdog mailing list
https://lists.wpkg.org/mailman/listinfo/sheepdog

[saxion]

Hi Mitake,
Thanks very much for your explanation.

It's clear for me for the first problem.

For the second one, It's still a little fuzzy for me.
For example, if node(i), node(i+1) and node(i+2) store the object Oidx.
If node(i) is down, then node(i+1) and node(i+2) still store the object Oidx, and the new node is node(i+3) which will be selected to store Oidx?
And in a similar way, if node(i+1) or node(i+2) is down, node(i+3) will also the new node, because node(i+3) is the following node of those three nodes.

Am I right?



------------------ Original ------------------
From: "Hitoshi Mitake";<***@gmail.com>;
Date: Fri, Apr 14, 2017 01:39 PM
To: "saxion"<***@qq.com>;
Cc: "sheepdog"<***@lists.wpkg.org>;
Subject: Re: [sheepdog] questions for sheepdog fresh

Post by saxion
Hello everyone, I'm learning distributed storage system sheepdog.
But I have some questions about sheepdog after I googled what I want to know.
As far as I know, consistent hashing is used in sheepdog for data distribution.
For example, a sheepdog cluster with 3 repica policy, an object OID will be mapped to host1, hosti and hostj.
My question is how we get those three host? For every object, an oid will be mapped to(using consistent hashing) only one host(host1 for example), how to select the other two host(hosti and hostj)? Or is there any details I got wrong idea?

Simply selecting three continuing nodes from the vnodes. You can find
the logic here:
https://github.com/sheepdog/sheepdog/blob/master/include/sheep.h#L109

Post by saxion
And, when host1 is down, how about the data rebalance/recovery, how to choose the right host to make sure there are 3 copied for every object?

The vnodes are updated during membership change (e.g. join and leave).
The membership is kept consistent for all living nodes by the atomic
broadcast functionality of corosync or zookeeper. So every living node
can have identical vnodes.

Post by saxion
Thanks.
saxion
--
sheepdog mailing list
https://lists.wpkg.org/mailman/listinfo/sheepdog

[Hitoshi Mitake]

Sorry for my late reply

Post by saxion
If node(i) is down, then node(i+1) and node(i+2) still store the object Oidx

It is not always true, the behaviour is probabilistic because sheepdog
uses consistent hashing for determining object placement.
In many cases node(i+1) and node(i+2) will remain but there is no
guarantee. Placement depends on the hash function and vnode weight.

Thanks,
Hitoshi

Post by saxion
Hi Mitake,
Thanks very much for your explanation.
It's clear for me for the first problem.
For the second one, It's still a little fuzzy for me.
For example, if node(i), node(i+1) and node(i+2) store the object Oidx.
If node(i) is down, then node(i+1) and node(i+2) still store the object Oidx, and the new node is node(i+3) which will be selected to store Oidx?
And in a similar way, if node(i+1) or node(i+2) is down, node(i+3) will also the new node, because node(i+3) is the following node of those three nodes.
Am I right?
------------------ Original ------------------
Date: Fri, Apr 14, 2017 01:39 PM
Subject: Re: [sheepdog] questions for sheepdog fresh

Post by saxion
Hello everyone, I'm learning distributed storage system sheepdog.
But I have some questions about sheepdog after I googled what I want to know.
As far as I know, consistent hashing is used in sheepdog for data distribution.
For example, a sheepdog cluster with 3 repica policy, an object OID will be mapped to host1, hosti and hostj.
My question is how we get those three host? For every object, an oid will be mapped to(using consistent hashing) only one host(host1 for example), how to select the other two host(hosti and hostj)? Or is there any details I got wrong idea?

Simply selecting three continuing nodes from the vnodes. You can find
https://github.com/sheepdog/sheepdog/blob/master/include/sheep.h#L109

Post by saxion
And, when host1 is down, how about the data rebalance/recovery, how to choose the right host to make sure there are 3 copied for every object?

The vnodes are updated during membership change (e.g. join and leave).
The membership is kept consistent for all living nodes by the atomic
broadcast functionality of corosync or zookeeper. So every living node
can have identical vnodes.

Post by saxion
Thanks.
saxion
--
sheepdog mailing list
https://lists.wpkg.org/mailman/listinfo/sheepdog