We explored the effects of chondroitin sulfate on knee osteoarthritis in a one-year, randomized, double-blind, dose-comparison study. Patients with painful, Kellgren–Lawrence grade 2–3, osteoarthritis of the knee were treated with oral chondroitin sulfate at a dose of either 260 mg/d (low-dose group, control group) or 1560 mg/d (high-dose group). Symptoms were evaluated by the Lequesne’s index and visual analog scale for pain. We made subgroup analyses according to background symptom severity (Lequesne’s index ≥8 or <8) in 73 patients. Serum level of cartilage oligomeric matrix protein and hyaluronic acid were also determined. In the subgroup with severe symptoms (Lequesne’s index ≥8), the chondroitin sulfate dose of 1560 mg/d improved pain faster after 6 and 9 months’ therapy. However, no dose-related effects were found on cartilage oligomeric matrix protein or hyaluronic acid levels. Chondroitin sulfate also had good tolerability. We conclude that chondroitin sulfate is useful for pain control in knee osteoarthritis.