The fundamental problems in dynamic load balancing and job scheduling in parallel and distributed comput- ers involve moving load between processors. In this paper, we consider a new model for load movement in synchronous parallel and distributed machines. In each step of our model, each processor can transfer load to at most one neighbor; also, any amount of load can be moved along a communication link be- tween two processors in one step. This is a reason- able model for load movement in significant classes of dynamic load balancing problems. We derive efficient algorithms for a number of task reallocation problems under our model of load move- ment. These include dynamic load balancing on pro- cessor networks, adaptive mesh re-partitioning such as those in finite element methods, and progressive job migration under dynamic generation and consump- tion of load. To obtain the above-mentioned results, we intro- duce and solve the abstract problem of Incremental Weight Migration (IWM) on arbitrary graphs. Our main result is a simple, randomized, algorithm for this problem which provably results in asymptotically op- timal convergence towards the state where weights on the nodes of the graph are all equal. This algorithmutilizes an appropriate random set of edges for a matching. Our algorithm for the IWM probl used in deriving efficient algorithms for all the lems mentioned above. Our results are very general. The algorith derive are local, and hence, scalable. They w arbitrary load distributions and for networks trary topology which can possibly undergo li ures. Of independent interest is our proof te which we use to lower bound the convergenc algorithms in terms of the eigenstructure of derlying graph. Finally, we present preliminary experim sults analyzing issues in load balancing relat algorithms.