Complexity and large size of contemporary control, communication, and computer systems impose strong challenges to methods of their mathematical modeling, design, and testing. Classical approaches to their formal specification by exploiting theory of systems on groups of real numbers R and complex numbers C, often do not fulfill requirements of practice. For that reason, theory of systems on groups different from R and C has been developed. Differential operators and spectral (Fourier) analysis on groups play the same important role in such systems as in the case of systems modeled by signals defined on R and C. This paper first briefly reviews some aspects of research in system theory on groups and then presents an extension of the notion of Gibbs differentiation to matrix- valued functions on finite non-Abelian groups.