A problem arising in scientific computation is the solution of Ax=b, where A is a large, sparse matrix, on of the most robust algorithms for solving the above equation is the conjugate gradient method especially when combined with a preconditioner. This paper will discus a new software package, MP-PCGPAK2, that implements a prallel version of the conjugate gradient method for MIMD, message passing architectures. The parallel implementation is quite general and can be applied to algorithms for non-symetric or idenfinite systems such as GMRES, Bi-CGSTAB and QMR we when present results on a 1023 processor nCUBE 2 and a 128 processor iPSC/860 for positive definitie, symmetric systems ranging from on million to over 11 milion variables.