The shortened version of Fennema-Sherman test is used to measure attitudes toward mathematics in several international testing settings like TIMSS and PISA. On the basis of Classical item analysis and Confirmatory Factor Analysis in the different achievement levels and sets of countries, it is suggested that there are two items on the Fennama-Sherman test which should be discarded due to cultural and achievement considerations. Items “Mathematics is more difficult for me than for many of my classmates” and “Mathematics is not one of my strengths” are too complicated for test takers who belong to the lowest quartile of achievement. These items also seem to carry culturally sensitive elements especially in East Asian countries where fairly good students answer illogically due to such negative wordings. Alternative possibilities for test items are recommended.