I've always found the notion of peer review of philosophy journal articles a bit suspect. I also find it ludicrous that the phrase 'peer reviewed' is taken by some people to be a mark of quality assurance. A good editor is surely better than an incompetent anonymous peer in this respect.
One basic flaw in the peer review process is the selection of people to do the peer reviewing: are they competent, biased, awake, rivalrous, too busy with their own work to read to the end? The trouble is, we can't get reliable information on this sort of thing as they are almost always anonymous and unaccountable. Do we trust that editors always get great reviewers? It seems unlikely that they will be able to get many eminent thinkers to devote their time to sifting through the huge slush pile of articles generated by RAE pressure in the UK. Judging by the huge quantity of tedious and poorly written drivel that gets through the process of peer review and into print in philosophy journals, something isn 't working here.
Obviously if the reviews contain good arguments and criticisms these
can be judged more or less independently of their source. But such
reviews are not particularly common. Praise and acceptance from a mediocrity who doesn't completely grasp the topic is very likely as meaningless as condemnation and rejection by the same individual. But praise or rejection from someone who is a major player in the field - that might mean something. It might also sharpen up the process if named people could be held to account for their views.
Better still, why don't we bypass journals altogether and just publish all our views on the Internet as we choose. Let our peers review these after the event (if they feel like it) and in the open (or before, by private arrangement, if we are shy about making fools of ourselves). That might re-invigorate the subject and the ways in which we express ourselves.
A programme that just went out on BBC Radio 4,'Peer Review in the Dock' and which you can listen to again here for the next 7 days, revealed how much more dangerous faith in the peer review process is in areas such as medicine. It also refers to evidence that peer review is not a reliable mark of quality...far from it.
See also some useful links in Tim Crane's comments below.