At present, in rough set theory there are two kinds of heuristic attribute reduction algorithms, one is based on discernibility matrix, the other is based on mutual information. But if these algorithms are applied to the non-core information system, there will be much problems, such as too much calculation, excessive reduction, or insufficient reduction. So we propose an improved heuristic attribute reduction algorithm on the basis of rough set theory, in which the attribute importance is dependent on two factors, one is increment of mutual information, the other is information entropy. And we set the attribute with both the largest attribute importance and mutual information among all attributes as the core attribute, by which we solve the problem that causes the computational complexity increasing because of selecting the initial attribute randomly. By the proposed algorithm we can not only improve the efficiency of attribute reduction, but decrease the number of attribute reduction. The validity of the proposed algorithm is verified by two ways of the theoretic analysis and the simulation experiments.