Draft papers and abstracts submitted to 2024's conferences, being organized by the International Institute of Informatics and Systemics (WMSCI 2024 and the symposia to be organized in its context, CCCT 2024, ISAS 2024, EISTA 2024, PISTA 2024, SOIC 2024, etc), will all have multi-modal and multi-methodological reviewing processes. Multi-methodological reviewing is due to the multi- and inter-disciplinary nature of these conferences. The reasoning which supports the multi-methodological approach can be found at "A Multi-Methodological Reviewing Process for Multi-Disciplinary Conferences".
This multi-methodological approach might be achieved by:
Formal, linear, systematic methods, for the achievement of what is called top-down quality, and/or
Informal, nonlinear, systemically interactive methods, for the achievement of what is called bottom-up quality
A usual conventional Formal, linear, systematic and top-down method that we will use for IIIS' 2024 conferences is the double-blind reviewing.
Another Formal, linear, systematic and top-down method that we will also use for IIIS' 2024 conferences is a non-blind, open reviewing.
We implemented a Participative Peer-to-Peer Reviewing (PPPR), as an Informal, nonlinear, systemically interactive and bottom-up reviewing method, where each draft paper or abstract will be reviewed, evaluated and constructively commented by other authors who submitted draft papers or abstract in the same area, sub-area or topic.
Consequently, each draft paper or abstract submitted to any of the 2024' conferences being organized by IIIS will go through these three kinds of reviewing:
It will be sent to at least three reviewers, randomly selected, for its double blind reviewing.
Draft papers and extended abstracts will also have non blind, open reviewing by means of 1-3 reviewers suggested by the submitting authors.
It will be posted, without previous screening, in the conference website in a way that it could be accessed, reviewed, commented and evaluated just by the authors who sent draft papers or abstracts in the same area or topic. Authors will get a login and a password in order to have this kind of access.
The reasoning supporting this multi-methodological approach has been included in another document ("A Multi-Methodological Reviewing Process for Multi-Disciplinary Conferences"). Our purpose in this document is to provide some reasoning to support the third kind of reviewing given above, i.e. the Participative Peer-to-Peer Reviewing (PPPR).
Peer reviewing/refereeing in scholarly publishing is an instrument of quality control and assurance by means of which journal's editors, conference's organizers, books publishers, etc. achieve the quality level required by their objectives. Since the functions of different means of scholarly publishing are not necessarily the same, indeed they might be highly diverse, peer reviewing may differ among different means of scholarly publishing.
Regarding this issue Walker and Hurt (1990, Scientific and Technical Literature), for example, affirm that "it is not practical to require the same kind of refereeing of conference papers as for journal articles because of time constraints, and the process would modify, if not eliminate, one of the most desirable characteristics of such meetings: informal exchange of ideas and preliminary findings of new research as well speculative and even nonconventional presentations of information, both intended to promote innovation and creativity". This is why "Even for conferences sponsored by societies that have high standards for their journal publications, there is no assurance high standards are also applied to the publications of all conference contributions" (p. 97; emphasis added).
Although the informal or the semi-formal nature of conferences, as well as their time constraints, might be the cause of a less effective quality control and a lower level of quality assurance, communications and information technologies are making feasible some reviewing methods that might help in increasing the level of quality control and assurance. Communications and Information Technologies are making Bottom-up quality a real and practical possibility. Participative peer-to-peer reviewing, via computing mediated communications, is a means for achieving bottom up quality in papers' peer reviewing or refereeing.
Bottom-up methods and processes are opposites to top-down ones, but, in our opinion these both opposites are not necessarily contradictory. Indeed, they might complement each other creating synergic relationships where the whole is more than the sum of its parts. They might even be polar opposites, where each opposite requires each other as a necessary condition for its own existence.
Quality control used methods has been oscillating between these two opposite methods. The movement of Total Quality is fundamentally based on participative peer-to-peer bottom-up methods, which emerged in opposition to the supervisory top-down approach based on the applications of rules and systematic and formal procedures. Bottom up quality processes are highly interactive, informal, non-linear and systemic (but not necessarily systematic) where personal creativities are merged in a collective web of team creativity. Analogously, papers reviewing/refereeing may be made by means of interactive, informal, non-linear and systemic bottom-up methods as opposed to formal, linear and systematic top-down methods.
Bottom-up quality has been achieved not just in manufacturing, and in other areas where Total Quality methods were applied with a significant effectiveness, but also in participative management of organizations, information systems development and software engineering. Ed Yourdon, for example, the very well known consultant in Software Engineering, and creator of the best known top-down methods in programming and software development is actually embracing and advocating top-down quality in the areas of software engineering and information systems development. Interviewed by Carol Deckers (from Quality Plus Technologies) in a Quality Plus E-talk affirmed that "in the context of quality assurance in the computer field, that is a bottom-up grass roots approach to making things better as opposed to the top-down approach that you see in most business organizations". Regarding Software Engineering Institute's Capability Maturity Model, oriented to quality assurance, Yourdon affirm that "It is interesting that, at this point [November, 2000], after a full decade that only 15% of American IT organizations have even bothered going through an assessment to find out where they are on that scale. When it happens, it is usually done on a top-down basis, that is a senior vice president or a CIO says we better do it and it is very important for the long-term good of the company to achieve a level 3, 4 or 5 on this scale…while this may be very beneficial for the company as a whole, it often has short-term negative consequences for the practitioners and the computer professionals and the engineers down at the bottom because they end up having to work harder and longer in order to achieve these worthy goals. One of the interesting things that was done four or five years ago by the same organization, the SEI, was to develop a bottom-up approach. This is something that could be practiced at the grass roots level by individual computer engineers". (Transcripts of the Quality Plus E-talk where Ed Yourdon made these statements can be found at http://www.stickyminds.com/sitewide.asp?Function=edetail&ObjectType=ART&ObjectId=2244. Yourdon, creator, about 30 years ago, of the Yourdon's Top-Down Structured Analysis and Design in Software Engineering affirms convincingly that most software engineering projects are feasible just with a bottom-up quality approach. The Software Engineering Institute, developer and promoter of the Top-Down Capability Maturity Model also conceived a bottom-up quality model. Information Communication Technologies (ICT) are providing the means to make possible and feasible the bottom up approaches to quality control and assurance. Bottom-up approaches to quality control are being increasingly applied in an expanding diversity of areas.
In our opinion, bottom-up approaches to peer reviewing/refereeing should also be designed, implemented and tested in the context of action-design, action-learning and action-reflection, in order to take advantage of the opportunities being generated by ICT. Consequently, The Institute of Systemics, Cybernetics and Informatics (IIIS) is implementing a Participative Peer-to-Peer Reviewing (PPPR) as a complement to the traditional top-down processes where journal editors and conference organizers select reviewers for submitted papers and whose comments and evaluation regarding the papers they are reviewing support the decisions to be made regarding the acceptance or non-acceptance of the submitted papers. Integrating systemically traditional top-down and bottom-up reviewing method will, very probably, improve the total quality of peer reviewing and might help in overcoming the frequently reported weaknesses of peer reviewing/refereeing.
In this systemic framework, IIIS is implementing the Participative Peer-to-Peer Reviewing for 2024's conferences being organized. Accordingly, draft papers and abstracts submitted to 2024`s conferences will be posted as received, without any previous screening, by the Organizing Committee, on the conference website where it can be accessed, via password, by the authors who made submissions in the same area, sub-area or topic. This kind of bottom-up quality control and assurance needs the real and effective participation of the authors submitting papers in order to provide a workable solution for increasing the quality level of the papers to be accepted for their conference presentation. Yourdon used, at the 10th International Conference in Software quality, the phrase "Pay it Forward" to refer to what it is necessary in bottom-up quality.
Interviewed by Carol Dekkers regarding this phrase or idea, Yourdon affirmed "I want to point out a thought that is something that I picked up elsewhere; I did not invent it at all. It is the title of a book and actually a movie…It is a very simple idea that if someone does you a favor rather than paying it back or ignoring it altogether, that you might reciprocate by paying it forward. You know, passing it on but in kind of an expanding chain. If somebody does you one favor, you pass on the favor forward to three other people and each of those three passes it on to three others and so on. The reason that I was suggesting it, particularly in the context of quality assurance in the computer field, that is a bottom-up grass roots approach to making things better as opposed to the top-down approach that you see in most business organizations, and frankly in many government and social movements as well. The idea that the president, or the boss, or the CEO is going to figure out how to make things better and then the issue of edicts and orders that will ripple downward through the hierarchy to cause things to be done in a different fashion. Sometimes, that is important, particularly if you have a charismatic leader who can help break some kind of stalemate or paralysis in an organization. But I think in a lot of cases, it is going to have to come from the bottom upwards, and that was what I was trying to suggest in that conference and to help reinforce it. I made sure that everybody in the conference had a copy of the book. I also told them that I was prepared to follow my own advice by offering a "Pay It Forward" favor to two or three people in the conference". (Emphasis added. Transcript of this interview can be found at the URL we informed about above).
Participative Peer-to-Peer Reviewing requires a Pay it Forward attitude from the authors submitting draft papers and abstracts in order to be effective. Paraphrasing Yourdon we can say that each author receiving, or to receive, constructive comment for the draft paper or abstract he, or she, submitted to the conference, he, or she, should reciprocate and "pay it forward" making constructive reviews for three draft papers or abstracts. The authors of these three papers would "pay it forward" reviewing nine draft papers or abstract, and so on, in a kind of expanding chain that would generate a continuous quality increasing of each paper, and the quality of all the papers to be presented at the conference, as a whole.
