I recently read Citations are not enough: Academic promotion panels must take into account a scholar’s presence in popular media. I do understand that sometimes articles that should be accepted aren’t. Sometimes in a peer reviewed journal in a very select field everyone knows whose article it is even in the case of a blind review. Some archives are very restrictive and they have the power to refuse the use of their material in anything except a journal of their choice. Those archives do not want to see any of the material they paid for be available online for free. They maintain their prestige as an archive by having documents that no one else has. This would definitely prevent an article using those sources from being printed online.
With an online publication that is not peer-reviewed the thin edge of the wedge appears. By this I mean, an article that might have slightly questionable methods of research could be placed in a non-peer reviewed journal. Some people likely would not read the article from beginning to end and then we would be using questionable research to support ours. We could then slant our reading of results and publish our results online. The end result could be erroneous. I think the peer review process, although tedious, is best. Having spoken in favour of peer-reviewed articles being used to judge a scholar there are other ways. There are conferences that the scholar can present at. This does not carry as much weight as an article but a number of conference publications do eventually add up to success. The British system includes something called a REF (Research Excellence Framework) where article, complete book, chapter in a book, article, etc. all carry a separate weight.The total is what is important. (Remind me I said this when I am on the other end of the process).