Is there a way to rebalance single Datanode in Hadoop.

Question

If I have 2 DataNode DN1 and DN2 with two directories storing the data i.e. ‘/DN1’ & ‘/DN2’. So, can I move the blocks from one DN to another DN and update the meta.

Answer 1

Currently Hadoop does not automatically do this. You have to do this manually:

1. Shutdown involved DataNode
2. Use mv command to move the block replica and meta pairs from one directory to another. Ensure that the sub directory structure remains same. If this is not maintained, the DN will no longer be able to locate the replicas after the move.
3. Restart the DataNode.