The problem of central banks trying to shape policy is an especially big deal in an environment where wealthy interests have a big role in choosing central bankers. Then you get the awesome power of central bankers deployed on behalf of wealthy interests against ordinary working people.
First: I see no real reason to think that as a rule unaccountable central bank bureaucracies are going to make better long-term economic policy choices than a democratic process. The very same people who seem enthusiastic about this model of policymaking would likely be the first to explain to you that it's unwise to have a centrally planned economy run by unelected bureaucrats.
Second: that there actually is good reason to think that giving elected officials direct control over short-term macroeconomic stabilization is a bad idea, namely that they'll try to time the business cycle to the election cycle, which is exactly how central bank independence came to be enshrined as a principle of good governance. But in the medium term once it becomes clear that central banks are using their "independence" to encroach on other areas of policymaking, the central banks are (rightly) going to wind up having their independence taken away.
I'm wondering why we don't eliminate the role of human central bankers in setting interest rates and replace them with a computer program that just follows the Taylor rule or whatever formula the consensus of academic economists suggests. If we've got a reasonably good formula for determining what values interest rates should take, a computer would be better at following it than a human, since a human might be tempted to diverge for the purpose of threatening elected policymakers into making policies they prefer in other places or supporting a favored political leader, as Alan Greenspan seems to have done for Bush in 2004. And if we don't have a good formula, it's unclear how central bankers are supposed to decide what they should do anyway, and it's time to not have central banks. I don't think we're in this unfortunate situation -- we know enough about inflation/unemployment tradeoffs and other economic relationships to have good quantitative rules that humans can follow, and that robots can follow better.
In any event, I don't see why setting interest rates would be like writing novels or throwing parties or recognizing faces, where the best humans are going to do better than impartial robots following formulas programmed in by smart professors. If well-programmed computers can beat us at math and now chess and Jeopardy, why not at monetary policy?