Many classical methods such as Genetic Algorithm (GA), Genetic Programming (GP), Evolutionary Strategies (ES), etc. have made significant contribution to the study of evolutionary computation. And recently, a new approach named Genetic Network Programming (GNP) has been proposed for especially solving complex problems in dynamic environments. It is based on the algorithms of classical evolutionary computation techniques and uses the data structure of directed graphs which is the unique feature of GNP. Focusing on GNP's distinguished expression ability of the graph structure, our previous research proposed an approach named GNP with Reconstructed Individuals (GNP-RI) which analyzes the structures of the elite GNP individuals and extracts important information to reconstruct new individuals. Based on the GNP-RI, additionally, this paper proposes a more enhanced architecture using SARSA learning to constantly update the utility of the branches, and their mutation rates are adjusted based on the corresponding Q values. The Q values also contribute to the fine tuning of the probabilities of the nodes to which the mutated branches will potentially point. This way, the inappropriate branches could be easily located, then changed to more desirable ones, by which a gradual reduction of genetic weakness could be achieved extensively during the evolutionary process of GNP. In the enhanced architecture named Genetic Network Programming with reconstructed individuals and SARSA learning-based adaptive mutation(GNP-RISLAM), we incorporate the individual reconstruction, which reconstructs and enhances the worst individuals by using the elite information and the adaptive mutation guided by the SARSA learning. In this paper, the proposed architecture has been applied to the tile-world which is an excellent bench mark for evaluating the evolutionary computation architecture.