At fist, thank you for your suggestion.
The answer I am excepting is quite different.
I have already did what you have mentioned.
The anticipation is once slave node gets its own seed from master node that is based on mpi-rank and constant random seed. After that, how would it get random seed for subsequent models in the same slave node?
If we would know it, it will help to re-run the same experiment. It is my thinking.