{"id":1402,"date":"2024-11-06T01:09:44","date_gmt":"2024-11-06T01:09:44","guid":{"rendered":"https:\/\/iticse.acm.org\/2025\/?page_id=1402"},"modified":"2025-12-08T12:19:54","modified_gmt":"2025-12-08T12:19:54","slug":"paper-review-process","status":"publish","type":"page","link":"https:\/\/iticse.acm.org\/2026\/paper-review-process\/","title":{"rendered":"Paper Review Process"},"content":{"rendered":"\n

Review process<\/h2>\n\n\n\n

ITiCSE papers are reviewed using a dual-anonymous process managed through EasyChair. The reviewing process has three phases: bidding, reviewing, and discussion\/metareviewing. <\/p>\n\n\n\n

Program committee (PC) members <\/strong>are expected to provide high-quality reviews to provide authors with feedback that will help to improve the work for publication or for future submission. Senior<\/strong> program committee members<\/strong> lead the discussion among PC members and metareview each paper, providing a recommendation and feedback to the program chairs. Since ITiCSE 2024, all PC members are allocated papers from only one track<\/a>. PC members will be asked to nominate the track or tracks for which they would prefer to review or metareview and every effort will be made to give them their preference. <\/p>\n\n\n\n

Each paper submission is expected to receive three or four reviews and a metareview. This number depends on the number of submissions and the number of ordinary and senior PC members. <\/p>\n\n\n\n

All reviews are submitted through EasyChair. All paper submissions are anonymous. PC members are and should remain anonymous to one another and to the authors of the submissions they review. When writing comments on a review during a discussion, please refer to yourself and other PC members by the number of their review (R1, R2, etc). <\/p>\n\n\n\n

Reviewing timeline<\/strong><\/h2>\n\n\n\n

The following dates describe the timeline for the program committee work on ITiCSE 2026. Note that the review period is just two and a half weeks. <\/p>\n\n\n\n

All PC members are expected to engage with the process throughout the five weeks from Monday 12 January to Wednesday 18 February, as described below. Please consider your workload and other commitments around these dates before offering your services. Those who cannot commit to engagement throughout this period should not volunteer  for ITiCSE 2026. The program chairs will only accommodate requests for lighter loads in exceptional circumstances. <\/p>\n\n\n\n

Activity<\/strong> <\/td>Start date<\/strong> <\/td>End date<\/strong> (Anywhere on Earth, UTC-12)<\/strong> <\/td><\/tr>
Bidding (1 week) <\/td>Mon 12 January <\/td>Mon 19 January <\/td><\/tr>
Reviewing (2.5 weeks) <\/td>Sat 24 January <\/td>Tuesday 10 February <\/td><\/tr>
Discussion and meta reviewing (7 days) <\/td>Wed 11 February <\/td>Wed 18 February <\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n

Expected workload<\/strong><\/h2>\n\n\n\n

Program committee members<\/strong>: 4-6 full papers (each up to 6 pages + references) <\/p>\n\n\n\n

Senior program committee members<\/strong>: 7-10 full papers (each up to 6 pages + references) <\/p>\n\n\n\n

These estimates are based on the workloads over the past few years. Of course, they are subject to the number of submissions, so actual workloads for ITiCSE 2026 might be a little higher or lower. <\/p>\n\n\n\n

Bidding<\/h2>\n\n\n\n

Both ordinary and senior PC members will bid for papers that they are interested in within their allocated paper track during the week between abstract submission and full paper submission. Please take the bidding process seriously, as this reduces the chance of weak reviews arising from a lack of familiarity with the content of a submission. As part of the bidding process, you will be required to note papers with which you have a conflict of interest (papers that you recognise are from colleagues or project partners). Separately from the bidding, you will be shown a list of authors and asked to indicate which of those you have a conflict of interest with. <\/p>\n\n\n\n

It is important to understand that bidding does not mean choosing the papers you will review. That choice will be made by EasyChair, using an algorithm that tries to fairly allocate the papers. Ideally, bidding is a process of identifying all<\/strong> the papers that you would like to review or are willing to review. The more papers you bid for, the more likely you are to be allocated papers from your list. <\/p>\n\n\n\n

Bidding on just a small number of papers influences the algorithm unfairly in your favour. To counter this, anyone who bids on too few papers will have their bids supplemented with an arbitrary selection of papers. If you do not wish this to happen to you, please ensure that you bid \u2018yes\u2019 for at least 20 papers and \u2018yes\u2019 or \u2018maybe\u2019 for at least 20 more. <\/p>\n\n\n\n

Once you are notified that bidding is open, you should log in to your EasyChair account and access the ITiCSE 2026 conference with the appropriate role (\u2018ordinary<\/strong> PC member<\/strong>\u2019 or \u2018senior PC member<\/strong>\u2019). From the menu, click on Paper Bidding<\/strong>. You will see a list of the submissions, and you can click links to indicate your level of interest in reviewing each submission. To see the abstracts as well as the titles, click either Show full abstracts<\/strong> or Show abstract summaries<\/strong> at the upper right of the screen. <\/p>\n\n\n\n

There is no need to click No<\/strong> for papers you do not wish to review; No is the default option. You should click Conflict<\/strong> for a paper if you believe you have a conflict of interest, for details on what constitutes a conflict please visit the ACM Conflict of Interest Policy.<\/a> <\/p>\n\n\n\n

The papers are presented in a random order for each bidder, to help ensure a good spread of bids. However, this order is preserved for each bidder, so if you spread your bidding over several sessions, the papers will appear in the same order each time you access them.<\/p>\n\n\n\n

Reviewing<\/strong> <\/h2>\n\n\n\n

Program committee responsibilities<\/strong><\/h4>\n\n\n\n

Shortly after the close of bidding, PC members will be notified which papers they have been assigned, and paper reviewing will begin. <\/p>\n\n\n\n

PC members are urged to download their assigned papers immediately and take a first look at them. If, for example, you discover that you cannot review a paper because you have discovered a conflict of interest, we would far rather be informed in the first few days of reviewing than on the last day. <\/p>\n\n\n\n

When you log in to the appropriate role on EasyChair you will see a Reviews<\/strong> menu. On this page, you can download the papers assigned to you. You should also click Add Review<\/strong> for one of the papers, to get a first look at the review form \u2013 please don\u2019t submit the review until you\u2019ve written it! <\/p>\n\n\n\n

Please aim to complete your reviews before<\/strong> the deadline; it is a deadline, not a target submission time. When you leave the task till the last minute, things can go wrong (illness, work emergency, family emergency, reviewing taking longer than you anticipated), and then the reviews are not done by the deadline \u2013 which means that we are left asking for volunteers to review extra papers at very short notice. <\/p>\n\n\n\n

There will be periodic reminders during the review period. If at any time during this period you become aware that you will not have your reviews written by the deadline, please contact us immediately so that we can decide how to deal with it. <\/p>\n\n\n\n

Once you have submitted your review for a paper, you will be able to see the other reviews already submitted for that paper, and EasyChair will notify you by email when further reviews are submitted. Please consider these other reviews, in preparation for the discussion period. <\/p>\n\n\n\n

Here are some points that you should consider when reviewing. <\/p>\n\n\n\n

    \n
  1. Is the paper within the scope of ITiCSE? Is it about the education of students who are studying computing? Is it pertinent to the track for which it has been submitted? <\/li>\n\n\n\n
  2. Your review must be written by you; please do not farm your reviews out to other people. If you believe that other people should be reviewing for ITiCSE, please encourage them to do so in their own right. If you do not have the time or the inclination to do your own reviewing for ITiCSE, please do not volunteer to do so. There is a ACM policy on the use of generative AI (GenAI)<\/a> for reviewers, which all PC members should familiarise themselves with prior to reviewing.  <\/li>\n\n\n\n
  3. Your review should indicate whether the paper as submitted is acceptable for ITiCSE. There is no review-revise-review cycle, so papers should be marked as accept if they are currently acceptable, not if they have the potential to become acceptable after substantial revision. In particular, ITiCSE does not have a conditional acceptance. If a paper cannot be made acceptable with grammatical corrections and\/or minor revisions of content, you should recommend that it be rejected. <\/li>\n\n\n\n
  4. Whether or not a paper is accepted, the authors will want to know why. Please ensure that your review gives reasons, explanations, and examples of the points you make. Reviews just a few lines long are generally not helpful to the authors, the program committee members, or the program chairs. Authors should be able to see the text parts of the reviews as a constructive critique of their submission. <\/li>\n\n\n\n
  5. Be sure that your overall recommendation tallies with your impression of the paper. If you do not believe that the paper should be accepted, please do not give it a 4 just to be nice, or to err on the side of generosity. It is more helpful to the program chairs, the PC members, and the authors if your review is honest and transparent and your overall recommendation reflects your impression. Please do not<\/strong> <\/strong>present or even allude to numeric scores in the free text boxes, or to your overall recommendation score.  <\/li>\n\n\n\n
  6. ITiCSE reviewing is dual-anonymous. You should not know who wrote the papers that you review, and the authors should not know who reviewed their paper. Please take care not to divulge your identity in your review, for example by saying \u2018This paper that I wrote might be helpful to you\u2019 or by recommending the addition of references to several papers all of which you have (co-)authored. <\/li>\n\n\n\n
  7. Please appreciate that authors sometimes must omit or disguise references to their own work for their paper to be anonymous. As a reviewer, you may give the authors the benefit of the doubt. Use the \u2018Confidential comments to the committee\u2019 box to indicate this to the program chairs; for example, \u201cThis paper should reference Kim Lee\u2019s work, unless those references were removed for anonymity\u201d. <\/li>\n\n\n\n
  8. If you find that a paper you are reviewing is not completely anonymous, you may point this out to the authors. However, please do not divulge the authors\u2019 identity to the other PC members, and please continue to review the paper as though it were anonymous. If you feel that you cannot review it objectively, please notify the program chairs immediately. <\/li>\n\n\n\n
  9. Most papers have some flaws in their writing. You are not required to be an editor, pointing out every flaw that requires correction. However, some PC members choose to do this, and many authors appear to appreciate the additional service. The review form has a specific field for such suggestions, keeping them distinct from the strengths and weaknesses and the overall evaluation of the paper. PC members should cater for all levels of writing, from new undergraduate student to experienced researcher, and for those whom English is not a first language. If the work is publishable as it is except that the language could be improved, please consider not rejecting this on those grounds alone. <\/li>\n\n\n\n
  10. Please try to ensure that reviews are grammatically correct \u2013 especially if they criticise the grammar of the paper. Just as a poorly written paper gives the reader a poor first impression, a poorly written review can make the authors less inclined to accept its content. Please read your own reviews carefully, and, if necessary, submit revised versions with the spelling and grammar errors corrected. If you choose to draw attention to spelling issues in a paper, please be aware that many words have different spellings in different parts of the English-speaking world. Neither \u2018analyze\u2019 nor \u2018analyse\u2019 is wrong: they are different acceptable spellings; likewise, \u2018program\u2019 and \u2018programme\u2019, \u2018behavior\u2019 and \u2018behaviour\u2019, and many more such pairs. <\/li>\n\n\n\n
  11. Please keep in mind the track for which you are a PC member. There are three tracks: computing education research; experience reports and tools; and position papers and proposals. Different guidelines apply to each track, and it would be unreasonable, for example, to review an experience report as though it were a research paper. See Paper track guidelines<\/strong><\/a> below for the guidelines for each track. <\/li>\n\n\n\n
  12. Papers and studies differ for a multitude of reasons, and this should be considered when reviewing. One size does not fit all, and reviews should consider this. For example, one study might require a large sample size. Another may only have a very small sample size, but the depth of treatment is still suitable. Thus, please avoid generic review comments (such as this paper needs to gather more data) that may affect the overall scores that reduce the chance of acceptance where it may be a very suitable study.  <\/li>\n\n\n\n
  13. Good papers will be relevant to an international audience. ITiCSE certainly accepts papers based on work from specific countries. Such papers are more valuable if they have clear insights for people from other countries. For example, a paper that writes about making it easier for people from the mountain regions to succeed in a computing degree would be better received if it made a clear generalisation to other underrepresented minorities \u2013 or perhaps to people with better lung capacity \u2013 regardless of country. <\/li>\n\n\n\n
  14. The limit of six pages (with references possibly extending onto a seventh page) is a maximum, not a minimum. There is nothing wrong with shorter papers if they are good and pertinent; they do not need to be padded out to six pages. At the same time, if a shorter paper clearly lacks something that would have strengthened it, feel free to point that out to the authors. <\/li>\n\n\n\n
  15. Authors are not permitted to provide links to supplementary materials in a review submission. Author guidelines make it clear that the paper should stand on its own merits. If authors overlook this guideline and provide links to supplementary material, reviewers are not expected to access those links to examine the additional material. <\/li>\n\n\n\n
  16. It is acceptable to point out formatting issues. Do not make them the deciding factor in a recommendation to reject. Some authors are still struggling to come to grips with the ACM templates. One version of the template even results in a single-column pdf. If in doubt, leave it to the proceedings chair to decide on a paper\u2019s compatibility with the template. <\/li>\n\n\n\n
  17. Please be aware that different versions of the ACM template have different placeholders for the title and the authors\u2019 names. There is nothing insidious about a paper that says it is being submitted to WOODSTOCK\u201918, June, 2018, El Paso, Texas USA, or that its authors are F. Surname et al. or G. Gubbiotti et al.; these are all defaults in different versions of the template. As with other aspects of the formatting, assume that it will be fixed if and when the paper is accepted for publication and revised. <\/li>\n<\/ol>\n\n\n\n

    Senior program committee member responsibilities<\/strong><\/strong><\/h4>\n\n\n\n

    Each submission will be assigned to one senior PC member. As a senior PC member, please carefully read each submission at the start of the review period so that you will be able to relate the reviews to it as they are submitted. You are expected to watch the reviews for your assigned papers as they come in. To do this, you will need to explicitly add them to your watchlist so that you will receive email notifications. From the ITiCSE 2026<\/strong> menu choose My watchlist<\/strong>, choose the Select all<\/strong> option, then Save watchlist<\/strong>. As the reviews arrive, please keep an eye open for obvious problems: a review might have been submitted for the wrong paper, or might identify the author or the PC member, or might be so brief as to be unhelpful. While we will do our best to notice such problems, your attention will also help. <\/p>\n\n\n\n

    Unlike some other conferences, ITiCSE keeps track of review completion, sends several review reminders, and chases unresponsive PC members. PC members can become confused if they receive multiple reminders from the same EasyChair email address \u2013 for example, they might respond to a reminder from the chairs, and then a few hours later receive another reminder from a senior PC member. Therefore, we explicitly ask senior PC members not<\/strong> to send reminders to PC members, and indeed not to send any emails to them, but to limit their communications to the comment feature of EasyChair and their eventual metareviews. <\/p>\n\n\n\n

    Emergency reviewing (optional)<\/h2>\n\n\n\n

    Despite the best of intentions, some PC members find that they cannot review the papers assigned to them and others simply drop out of contact for various reasons. The program chairs often discover this only at the end of the reviewing period. They then need to find PC members who have completed their assigned reviews and are able to review one or two more papers within a day or two. <\/p>\n\n\n\n

    The call for emergency reviewers generally goes to all PC members, but there is no expectation that all of them will respond. We understand that most PC members now have other calls on their time, and we are appreciative and grateful that a few can step in and take on this additional load. <\/p>\n\n\n\n

    Discussion and metareviewing<\/h2>\n\n\n\n

    The discussion period facilitates communication among the PC members and senior PC members to ensure that the senior PC member makes the best recommendation to the program chairs, and the submission is given full consideration in the review process. <\/p>\n\n\n\n

    Program committee member responsibilities during discussion and metareview<\/strong> <\/em><\/strong><\/h4>\n\n\n\n

    As a reviewer, you are expected to engage with the discussion on each of your papers during the discussion period. Read the reviews from the other reviewers and take part in the discussion using the Comments feature in EasyChair (beneath all the reviews). Your comments will not be sent to the paper\u2019s authors, but the final version of your review will. If, after reading the other reviews and taking part in the discussion, you are inclined to revise your review, please do so. There is no obligation to revise your reviews, but your review should reflect your final impression of the paper, and compatible reviews from all reviewers make matters easier for the program committee members, the program chairs, and the authors of the submission.<\/p>\n\n\n\n

    If you do revise your review, please do not refer explicitly to other reviews. It is not appropriate to write, for example, \u201cI agree with Reviewer 3 about the paper\u2019s shortcomings\u201d. It should be possible to read each review independently of the others. If you do wish to explain why you have revised your review, please say it in the Confidential comments part, where it will be seen by the other reviewers, the program committee members, and the program chairs, but not by the paper’s authors.<\/p>\n\n\n\n

    Senior program committee member responsibilities during discussion and metareview<\/strong> <\/em><\/h4>\n\n\n\n

    As a senior program committee member, you will lead and encourage discussion among the PC members, especially for submissions with divergent reviews. This discussion will help you to form an opinion on whether the paper should be accepted. <\/p>\n\n\n\n

    Ideally, the PC members will come close to consensus on the quality of the submission. Different PC members might see different strengths and weaknesses in the paper or might weight those strengths and weaknesses differently. However, it would be good if all PC members, after reading the other reviews and taking part in the discussion, have similar opinions about the paper overall, and are willing to revise their reviews if necessary to reflect those opinions. <\/p>\n\n\n\n

    On the other hand, PC members should never feel coerced to change their reviews, scores, or viewpoints in this process. <\/p>\n\n\n\n

    The senior PC member will write a metareview, summarising the reviews, adding their own opinion if appropriate, and providing a recommendation for the program chairs. If the reviews are divergent, it is important for the metareview to summarise both positions. This metareview will be sent to the authors along with the regular reviews. The metareview should also include feedback from the senior PC member, not just a repeat of the reviews, as this is an important piece of feedback for the authors (irrespective if the paper is accepted or not). Thus, a high quality, suitable and detailed metareview can support the authors on top of the reviews. <\/p>\n\n\n\n

    The text of the metareview should not explicitly include the senior PC member\u2019s accept\/reject recommendation; there is a separate field for that. As a senior PC member, you will only see a few of the submitted papers; please understand that while your recommendation will be carefully considered, the program chairs might choose the opposite outcome when considering the full set of submissions and working to put together a broad yet coherent program. At the same time, please remember to select a recommendation. Recommendations to accept or reject are far more helpful to the program chairs than \u2018undecided\u2019, which should be reserved only for the very difficult<\/em> papers. <\/p>\n\n\n\n

    Please do not wait until the very end of the discussion period to write your metareviews. <\/p>\n\n\n\n

    Senior PC members who are unsure what to put into a metareview might consider starting with a simple four-paragraph template: what the paper is about; what\u2019s good about it; what\u2019s not good about it; your overall opinion. An alternative approach would be to briefly note the key points made in each review, then to add your own opinions. Remember that you are a senior PC member: your role is not to repeat what the PC members have already said, but to coalesce it into a single overall impression, which can certainly include your own thoughts on the paper. <\/p>\n\n\n\n

    Note that confidential comments in the metareview are visible to all the paper\u2019s reviewers as well as the program chairs. If you wish to communicate just with the program chairs, please do so by email. <\/p>\n\n\n\n

    Additionally, the program chairs will ask senior PC members for feedback on the quality of reviews, to help inform decisions about future invitations to review for ITiCSE. <\/p>\n\n\n\n

    Paper track guidelines<\/h2>\n\n\n\n

    There are three tracks for papers.  <\/p>\n\n\n\n

    Authors are asked to choose the track that they feel best fits their submission. Review the submission using the guidelines for the track the submission is in; not the track you would prefer it to be in. <\/p>\n\n\n\n

    Computing education research track<\/strong><\/p>\n\n\n\n

    Papers submitted to the computing education research track typically describe an empirical computing education project. <\/p>\n\n\n\n

    Computing education research papers should adhere to rigorous standards, describing research questions or hypotheses, methods, and results as is typical for research studies. These will normally focus on topics relevant to computing education with emphasis on educational goals and knowledge units or topics relevant to computing education with statistical rigour; methods or techniques in computing education; evaluation of pedagogical approaches; and studies of the many different populations that are engaged in computing education, including (but not limited to) students, instructors, and issues of gender, diversity, and underrepresentation. Papers need not present novel research and positive results to be accepted at ITiCSE: we also welcome replication papers and papers that present null or negative results, so long as they meet the criteria below. <\/p>\n\n\n\n

    For a typical paper in this track, here are some factors to include (as an author) and to look for (as a PC member):<\/p>\n\n\n\n

      \n
    1. Are there one or more clearly stated research questions? Since the rest of the paper will be organised around these, it\u2019s often good to put them in the abstract and early in the body of the paper.<\/li>\n\n\n\n
    2. Are the questions of interest to the ITiCSE audience? Is the focus on the education of computing students, as opposed to education in general?<\/li>\n\n\n\n
    3. Related work in computing education<\/span>\n