Silica and asbestos cause pneumoconioses known as silicosis and asbestosis, respectively, that are each characterized by progressive pulmonary fibrosis. On the other hand, silicosis patients often suffer from a type of immunological dysregulation that gives rise to autoimmunity. These epidemiological findings suggest that silica may affect the immune system in humans. In addition, as asbestos itself is a mineral silicate, it may possess generalized immunotoxicological effects similar to those associated with silica particles. Because asbestos-exposed patients are well-known to often develop malignant diseases such as lung cancer and mesothelioma, one silica-like dysregulatory outcome that needs to be considered (apart from autoimmunity) is an alteration in host tumor immunity. In this review, the immunotoxicological effects of both silica and asbestos are presented and discussed in terms of immune system dysregulation as manifested by the onset of autoimmunity or alterations in host tumor immunity.