Particle swarm optimization (PSO) algorithm is a swarm intelligent searching algorithm based on population that simulates the social behavior of birds, bees, or fish groups. The discrete binary particle swarm optimization (BPSO) algorithm maps the continuous search space to a binary space through a new transfer function, and the update process is designed to switch the position of the particles between 0 and 1 in the binary search space. Aiming at the existed BPSO algorithms which are easy to fall into the local optimum, a new Z-shaped probability transfer function is proposed to map the continuous search space to a binary space. By adopting nine typical benchmark functions, the proposed Z-probability transfer function and the V-shaped and S-shaped transfer functions are used to carry out the performance simulation experiments. The results show that the proposed Z-shaped probability transfer function improves the convergence speed and optimization accuracy of the BPSO algorithm.