Small mammals play a role in plague transmission as hosts in all plague endemic areas. Information on distribution and diversity of small mammals is therefore important for plague surveillance and control in such areas. The objective of this study was to investigate small mammals’ diversity and their distribution in plague endemic area in the West Usambara Mountains in north-eastern Tanzania. Landsat images and field surveys were used to select trapping locations in different landscapes. Three landscapes with different habitats were selected for trapping of small mammals. Three types of trap were used in order to maximise the number of species captured. In total, 188 animals and thirteen species were captured in 4,905 trap nights. Praomys delectorum and Mastomys natalensis both reported as plague hosts comprised 50% of all the animals trapped. Trap success increased with altitude. Species diversity was higher in plantation forest followed by shrub, compared to other habitats, regardless of landscape type. It would therefore seem that chances of plague transmission from small mammals to humans are much higher under shrub, natural and plantation forest habitats.