Field investigations have proven that layer‐crack structures make an essential contribution to rock burst occurrence in brittle coal or rock mass. This paper first studies the mechanical behavior of layer‐crack red sandstone specimens with different geometric configurations of vertical fissures (ie, fissure width, fissure length, and fissure number) through uniaxial compression and Brazilian tests. Then, the digital speckle correlation method (DSCM) and acoustic emission (AE) technique are used to record the deformation and failure processes. Finally, the failure mechanisms of the layer‐crack structure are presented and discussed. Under compressive conditions, the failure process can be divided into compatible support, incompatible support, and postpeak unstable failure stages. An increase in fissure width, as well as fissure length or number, causes an increase in the inner damage accumulation and a decrease in the stiffness coefficient by decreasing the strain of the compatible support stage. Thus, the bearing capacity of the layer‐crack structure is weakened. Under tensile conditions, the tensile ability decreases linearly as the fissure length increases, but the vertical pressure acting on the fissure increases linearly as the fissure width increases. Thus, the tensile strength of the layer‐crack structure decreases as the fissure length (or width) increases.