Update: Dana Goldstein, who is an actual education reporter instead of a poor imitation of one, drops some knowledge. The most widely-used number is 5-10 percent of the teacher workforce. See this chapter from Eric Hanushek for more detail
The New York Times bemoans the results of recent teacher evaluation systems, which seem to give an overwhelming majority of teachers passing grades. The tone of the article seems to imply that this is a failing of the evaluation system; that principals do not have sufficient emotional detachment to rate teachers poorly; and that in general the efforts to deviate from more quantitative methods of teacher evaluation have stymied education reform. As a followup, it would be instructive if the New York Times examined its own 2012 Performance Evaluations and published an article on how many employees were deemed ineffective.
The evaluation systems seem to place an extremely high number in the highest-two rating categories, and only identifying a handful of teachers--2-3% seems to be a common number--as ineffective. The reformers seem to have this hazy, not-very-well-thought-out view that perhaps 10, 20, or 50% of teachers (no one is willing to put a number on this) are inadequate to the task of teaching, which would be astonishing for any large enterprise. The 2% figure lines up roughly with the number of LAPD cops who averaged at least one excessive force complaint per year during the late '80s (keep in mind this covers allegations, not proven cases or even disciplinary actions). Over that same timeframe, 0.5% of officers averaged about 1.5 complaints per year, and a handful of cops were truly disastrous. Even if America's public schools were to have double or triple the number of bad apples as the LAPD, that would still mean the overwhelming majority of teachers are at least doing their jobs adequately.
The real problem is that inadequate teachers are more likely to be found in environments where their inadequacy hurts. If half a city or state's weak teachers are concentrated in high-poverty schools, then those schools are going to fit the mental picture that reformers seem to ascribe to the public school system writ large. And the students at those schools won't have the same growth opportunities outside the classroom that the children of middle-class parents have. If we can find a way to get high-quality teachers to take jobs in poor schools and stay there, it would probably result in a significant net boost in student outcomes.
The Center for American Progress has a decent writeup on America's teacher workforce if you're curious.