A standard method of determining whether a construction is well-formed is a grammaticality judgment test, where subjects make an intuitive pronouncement on the accuracy of form and structure in individual decontextualized sentences. An empirical study of the validity of grammaticality judgment tests is presented. Teachers of English (n = 52) were asked to rate the grammaticality of ten constructions which are contentious, e.g. center-embedding. The results varied with considerable agreement by the subjects on some sentences, e.g. the perceived incorrectness of Who did you quit college because you hated?, and a very divided response on others, e.g. John angered while Susan amused the woman. In evaluating the degree of (non)-consensus, there is some clash between evidence on the frequency of usage, from corpora, and internal linguistic criteria, e.g., from constraints in Universal Grammar. It is concluded that grammaticality judgment tests offer a probabilistic rather than definitive answer to the question, 'Is x well-formed?'