Michael, I don't think the issue is the ability to track performance, but more the decision on what minimal set of metrics to track.
Even if you can provide a graph, someone needs to analyze the results, and report to it on human terms.
I'm thinking for this to be at all effective, it needs to be managed on a case-by-case basis, like how Ubuntu papercuts is managed.
That needs buy-in from the people upstream.
I suspect the issue is more of a political one, than a technical one. Policy & process need to be defined, else what happens is that the people in-fight as to whom is at blame, or whom is right, whom's metric is more important (especially if a change affects one metric positively and another negatively)
Trust me, political a nightmare, people are.
Argue they do, on the same side of the fence, and none dares admit it.