The proposed working groups:
- Exploring Computing Science Programs’ Admission Procedures with a Diversity & Inclusion Lens
- Building Recommendations for Conducting Equity-Focused, High Quality K-12 Computer Science Education Research
- Multi-institutional Studies of the Effectiveness and Impacts of Parsons Problems in Python
- Transformed by Transformers: Navigating the AI Coding Revolution for CS Education
- Arguments for and Approaches to Adding Computing Education to Undergraduate Computer Science Programmes
- Modeling Women’s Module Choices in Computer Science
Applying to join a working group
Applications now closed
WG 1: Exploring Computing Science Programs’ Admission Procedures with a Diversity & Inclusion Lens
WG Leaders:
Ouldooz Baghban Karimi (ouldooz@sfu.ca)
Giulia Toti (giulia.toti@ubc.ca)
Mirela Gutica (Mirela_Gutica@bcit.ca)
Abstract:
Motivation
Computing science education has experienced low attendance and historic declines in registration from different minority groups. The past decade of enrollment surge in computer science undergraduate programs has increased the number of women and minorities in the field, but the improvements are inconsistent and less than expected. An increase in the use of computing science and in the demand of technology workforce is expected in the upcoming years. Thus, computing science is set to shape the future of technology for a diverse set of technology users. Therefore, it is important to analyze how undergraduate program admission procedures are affecting the Equity, Diversity, and Inclusion of historically marginalized groups in computing science.
To our knowledge, the impact of how indicators of success are formulated and advertised through admissions processes has not been studied. It is possible that some underlying message in admission processes may affect student demographics. Exploring this possibility is especially important because, while registrations in computer science programs have enjoyed a considerable increase, all studies report a lesser growth in diversity when compared to the growth in overall enrollment, even with such affirmative processes implemented.
We aim to explore the factors considered by different computer science programs for admitting a student to a program. We plan to identify the differences among such processes among institutions successful in closing the gender gap and including more diverse student backgrounds with counterpart programs. We also aim to find out if there is a meaningful correlation between admissions procedures and gender and racial diversity among admitted students.
Goals
We attempt to identify the role of admission processes in gender and racial diversity in the student population and the inclusion of historically marginalized groups in undergraduate computer science programs. We are asking:
- What are common approaches to admissions in computer science undergraduate programs?
- What are the indicators of success considered in such admission procedures?
- What are the different outcomes of current admissions processes in terms of cultivating diversity and inclusion in admitted student population?
- What are possible ways in which admission processes can promote diversity and inclusion?
Methodology
We intend to identify undergraduate computer science programs across the globe that have shown relative success in attracting women and historically marginalized groups and perform (1) content analysis on their advertisement of admission processes, (2) data analysis on their student population. We will formulate success indicators based on the approach of these programs for achieving diversity and inclusion, and their identified differences within admission procedures. We plan to identify the differences among such processes among institutions successful in closing the gender gap and including more diverse student backgrounds with counterpart programs. We will then proceed with designing survey questions and conducting surveys, [possibly] interviews, and case studies to further our knowledge of the impact of the success indicators in the admissions procedures.
Expectations of members
We estimate 3-4 hours per week from April to July including a one-hour online meeting for pre-conference work. We should complete data collection, design and conduct surveys (and interviews), and proceed with some data analysis before we meet at the conference. We expect all members to attend the pre-conference working days. We also expect 3-4 hours per week from July to October for addressing the review comments.
If we collectively approve interviews as a required part of this work, each member is expected to take an equal workload of performing interviews (e.g., performing 2 interviews) with selected participants during the process.
All members are expected to contribute to data collection and receive required ethics approvals on the same unified application from their host institutions.
We welcome group members with expertise and involvement experience in the undergraduate admissions process.
We expect members to bring their experience, expertise, enthusiasm and commitment to achieve meaningful results, take and own a part of the work they are passionate about, and contribute to our work and learning together.
WG 2: Building Recommendations for Conducting Equity-Focused, High Quality K-12 Computer Science Education Research
WG Leaders:
Monica McGill (monica@csedresearch.org)
Sarah Heckman (sarah_heckman@ncsu.edu)
Abstract:
To investigate and identify promising practices in equitable K-12 computer science (CS) education, the capacity for education researchers to conduct this research must be rapidly built globally. Simultaneously, concerns have arisen over the last few years about the quality of research that is being conducted and the lack of research that examines CS education among systematically marginalized students in computing (e.g., rural students, girls, students with disabilities, students from low-income families). In this working group, we will tackle the research question: In what ways can previous research standards inform high-quality, equity-focused K-12 CS education research?
We will use existing research and various standards bodies (e.g., European Educational Research Association, 2019; Australian Education Research Organisation, 2023; CONSORT, 2010; American Psychological Association, 2020) to synthesize key features in the context of equity-focused K-12 CS education research. We will then vet these attributes with experts who can provide feedback and refine our recommendations and guidelines, synchronously and asynchronously. Our working group will select the experts using a strata reflecting a diversity of backgrounds and experiences to support our focus on systematically marginalized student populations.
Our recommendations will directly impact future equitable computing education research by providing guidance on conducting high-quality research such that the findings can be aggregated and impact future policy with evidence-based results. While we recognize that different countries and regions may yield differing answers to this question, our recommendations will be robust enough that researchers in each country or region may choose to use those most appropriate to their context.
Our carefully-scoped project plan consists of 6 phases, with the majority of work to be completed on or before we meet at ITiCSE. For more details including a schedule of planned activities, please visit https://csedresearch.org/iticse-2023-working-group/.
WG 3: Multi-institutional Studies of the Effectiveness and Impacts of Parsons Problems in Python
WG Leaders:
Barbara Ericson (barbarer@umich.edu)
Janice Pearce (jan_pearce@berea.edu)
Susan Rodger (rodger@cs.duke.edu)
Abstract:
We encourage people who are interested in the challenges that novices have when writing code to apply to join our working group. This working group will plan, conduct, and analyze data from multi-institutional studies on the effectiveness of Parsons problems. In Parsons problems, students place mixed-up fragments of code or an algorithm into the correct order to solve a problem. This working group is leveraging the work of the 2022 working group which conducted an extensive literature review on Parsons problems and designed and piloted several studies based on gaps in the literature. We will review/revise these studies and research questions, conduct one or more of the studies before the ITiCSE conference, analyze the collected data, and finalize a draft of the paper during the intensive three days before the ITiCSE conference.
Three of the current studies are between-subject experiments where the participants will be randomly assigned to one of two conditions. These studies will compare the learning performance on a post-test from 1) solving adaptive Parsons problems with and without distractors (extra fragments that are not needed in a correct solution), 2) solving adaptive Parsons problems versus writing the equivalent code, and 3) solving write-code problems without any scaffolding versus write-code problems with the ability to pop-up the equivalent Parsons problem as scaffolding (toggle problems). Another set of studies will investigate if students can successfully write code for common algorithms, such as swapping the values of two variables or the rainfall problem, after solving a set of Parsons problems. We plan to conduct the studies on the open-source Runestone ebook platform. This platform is used by tens of thousands of students and supports some newer types of Parsons problems such as adaptive Parsons problems, toggle problems, and micro Parsons problems.
We plan to meet weekly via Zoom at a mutually convenient time. If needed, we will meet multiple days a week so that we can accommodate a variety of schedules and time zones. We are looking for members who can run studies, and/or refine the studies, and/or analyze the data. Instructors who have access to 50+ undergraduate students learning Python programming are particularly encouraged to apply.
In your application, please explain why you are interested in this working group and how you plan to contribute. Be sure to include your background as it relates to the goals of this working group. Also, if you can run a study, please include the number of undergraduate students you have access to and anticipated timeframe for running one or more studies.
WG 4: Transformed by Transformers: Navigating the AI Coding Revolution for CS Education
WG Leaders:
James Prather (jrp09a@acu.edu)
Paul Denny (paul@cs.auckland.ac.nz)
Juho Leinonen (juho.2.leinonen@aalto.fi)
Brett Becker (brett.becker@ucd.ie)
Abstract:
The recent advent of highly accurateand scalable large language models (LLMs) have taken the world by storm. From art to essays to computer code, models like ChatGPT, Midjourney and Codex are creating novel content previously thought only producable by humans. Recent work in computing education has sought to understand the capabilities of LLMs for solving tasks such as writing code, explaining code, creating novel coding assignments, interpreting programming error messages, and more. However, these technologies continue to evolve at an astonishing rate leaving educators little time to adapt. Recent work in our community has shown that models like Codex can solve typical CS1 and CS2 exam questions with greater accuracy than most students and can be used to generate novel learning resources including line by line and high-level explanations of code. There is now a pressing need for computing educators to understand the capabilities of these models and adapt their instruction to take advantage of the opportunities and mitigate the challenges they present.
As a member of our working group, you’ll have the chance to delve into:
– Reviewing the latest, state-of-the-art research on the use of large language models in computing education
– Collecting and analysing data on the effectiveness of these technologies in teaching and learning computer programming
– Contributing to the development of actionable approaches for integrating these technologies into computing curricula.
We will meet online twice a month from March to July to discuss logistics, research, and progress on writing. We will discuss the timeline and set deadlines for completion of individually assigned work. Our ultimate goal is to have a compelling introduction and a thorough literature review ready before the in-person work at ITiCSE 2023 in Turku, Finland. We welcome you to join us on this exciting opportunity to shape the future of computing education.
WG 5: Arguments for and Approaches to Adding Computing Education to Undergraduate Computer Science Programmes
WG Leaders:
Quintin Cutts (quintin.cutts@glasgow.ac.uk)
Maria Kallia (Maria.Kallia@glasgow.ac.uk)
Abstract:
Motivation. This working group will address the adoption of computing education (CE) in computer science (CS) programmes. There are many possible arguments for why this is important: we need more graduates with both CS and CE knowledge to populate the CE initiatives in our school systems as well as upwards into society, including design, delivery, evaluation and research; a more subtle argument is that computing practice and research have the exercise of learning at their heart – a software engineer will be learning new languages, systems, paradigms and problem domains throughout their career and so it is appropriate to explicitly equip them for this; at a more functional level, we need courses in our own field to support the development of the CS education research ecosystem in universities.
Goal. Broadly, the goal is two-fold: to develop the arguments for such courses, and to provide broad curricular outlines to suit different possible modes of adoption. A multi-national working group team is required to develop a wide range of arguments given the highly situated nature of education: an argument fit for UK academia may not work in US institutions, for example. Considering curricular outlines, there are many issues and alternatives to work through: does the curriculum principally concern best CE practices, or research frontiers, or both; what are the key topics; is there an appropriate developmental sequence; is this compulsory or optional material; and is it embedded in existing modules, or delivered stand-alone?
Proposed methodology:
– Literature review to collate prior work on delivering CS education content, e.g. via Masters, PhD and tutor/TA training.
– Attitudes to CS education will be collated by working group members using a semi-structured interview format with up to five colleagues each to better understand the issues concerning institutional support or otherwise for CS education courses.
– CS education course content. We will run a Delphi-like process with a number of cycles to surface a range of potential topic areas and coalesce onto the ones that are most popular.
– On the in-person WG days prior to the conference, we will hone the arguments and draw up curricula structures in discussion with each other, ready for inclusion in the final paper.
We have only around 2-2.5 months to do the preparatory work. We won’t all do all the activities above – but will share out the work in our first meeting. We’d expect to meet to review progress every 2-3 weeks, with meetings running via Zoom and respecting members’ timezones.
WG 6: Modeling Women’s Module Choices in Computer Science
WG Leaders:
Steven Bradley (s.p.bradley@durham.ac.uk)
Miranda Parker (mcparker@sdsu.edu)
Abstract:
At ITiCSE 2021, Working Group 3 examined the evidence for teaching practices that broaden participation for women in computing, based on the National Center for Women & Information Technology (NCWIT) Engagement Practices framework. One of the report’s recommendations was “Make connections from computing to your students’ lives and interests (Make it Matter) but don’t assume you know what those interests are; find out!”. The goal of this working group is to find out what interests women students by bringing together data from our institutions on undergraduate module enrollment, seeing how they differ for women and men, and what drives those choices. We will code published module content based on ACM curriculum guidelines and combine these data to build a hierarchical Bayesian model of factors affecting student choice.
This model should be able to tell us how interesting different topics are to women, and to what extent topic affects choice of module – as opposed to other factors such as the instructor, the timetable, the mode of assessment. Equipped with this knowledge we can advise departments how to focus curriculum development on areas that are of interest to women. Computing is a very broad discipline and no program or even department could offer courses in all areas. In choosing which subjects are offered, departments are implicitly defining what is important within the field. We hope to be able to help departments change their curriculum to be more inclusive of diverse interests.
We are looking for participants who are able to source data from their institution on the numbers of men and women taking elective undergraduate computing modules. This may require institutional ethical approval, but would not require individual consent of students because all personal data (gender only) would be aggregated. In many cases this would be exempt from a full ethics approval submission, but it would be the responsibility of group members to gain appropriate consent for using their data in the study – with support from the group leaders where required. All group members will also be involved in reviewing published module descriptions from their own institution, and from one or more other institutions, to assign ACM curriculum areas to each module.