Developing and Validating the Competency Profile for Teaching and Learning Research Integrity
Abstract
Since research integrity is not external to research but an integral part of it, it should be integrated into research training. However, several hindrances regarding contemporary research integrity education exist. To address them, we have developed a competency profile for teaching and learning research integrity based on four assumptions: 1) to include all levels of study (BA, MA, and PhD); 2) to integrate research integrity into research education itself; 3) to address research integrity issues in context-specific practices; and 4) to pay particular attention to the ‘grey zone’ or questionable research practices. To assess the validity of the content of the competency profile and to determine if some adjustments to the profile are needed, we translated the competencies of the profile into items of a measurement instrument (a questionnaire) and conducted a survey amongst University of Ljubljana students that allowed us to 1) obtain information about students’ attitudes toward issues of integrity in research; 2) analyse differences in these attitudes among BA, MA, and PhD students; and 3) statistically validate the competency profile and suggest possible improvements. The results showed that 1) students are highly aware of research integrity issues, as scores were high on all items assessed. However, there were some deviations to lower scores, especially in relation to questionable research practises, confirming our assumption that the ‘grey zone’ issues are those that should be particularly addressed and given special attention in contemporary research integrity education. 2) The differences in the attitudes of BA, MA, and PhD students showed that higher-level students showed significantly more awareness of integrity issues than lower-level students did, suggesting that research integrity issues should be given special attention at the BA study level. 3) The measurement characteristics showed that the reliability of the questionnaire was very high, suggesting a good overall structure of the competency profile. The principal component analysis also confirmed the four-field structure of the Competency profile (Values and Principles, Research Practise, Publication and Dissemination, and Violations). However, the analysis also showed that the substructure of the four main areas of the profile did not fully match the results of the factor analysis, suggesting that the distribution of competencies in the competency profile could be reconsidered, especially in the area of Research Practice. The most recent developments in the field of research integrity also suggest that the competency profile should be updated with issues regarding the impact of artificial intelligence on research integrity.
Downloads
References
Alemu, S. K. (2020). Transnational mobility of academics: Some academic impacts. Center for Educational Policy Studies Journal, 10(2), 77–99. https://doi.org/10.26529/cepsj.464
Antes A., & DuBois J. M. (2014). Aligning objectives and assessment in responsible conduct of research instruction. Journal of Microbiology & Biology Education, 15(2), 108–116. https://doi.org/10.1128/jmbe.v15i2.852
Bebeau, M. J. (2002a). Influencing the moral dimensions of professional practice: Implications for teaching and assessing for research integrity. In N. A., Steneck, & M. H. Sheetz (Eds.), Proceedings of the first ORI research conference on research integrity (pp. 179–187). Office of Research Integrity, Department of Health and Human Services. https://ori.hhs.gov/documents/proceedings_rri.pdf
Bebeau, M. J. (2002b). The defining issues test and the four component model: Contributions to professional education. Journal of Moral Education, 31(3), 271–295. https://doi.org/10.1080/0305724022000008115
Bebeau, M. J. (2002c). Outcome measures for assessing integrity in the research environment (Appendix B). In US National Research Council. Integrity in scientific research: Creating an environment that promotes responsible conduct (pp. 143-166). The National Academies Press. https://www.nap.edu/read/10430/chapter/11
Bebeau, M. J., & Thoma, S. J. (1999). Intermediate concepts and the connection to moral education. Educational Psychology Review, 11(4), 343–360. https://doi.org/10.1023/A:1022057316180
Bison, R. (2023, July 19). Rise of AI stokes fears research misconduct could accelerate. Research Professional News. https://www.researchprofessionalnews.com/rr-news-world-2023-7-rise-of-aistokes-fears-research-misconduct-could-accelerate/
Böttcher, F., & Thiel, F. (2018), Evaluating research-oriented teaching: A new instrument to assess university students’ research competences. Higher Education, 75(1), 91–110. https://doi.org/10.1007/s10734-017-0128-y
Brent, T. (2023, June 27). European research integrity code updated to reflect advances in artificial intelligence. Science Business. https://sciencebusiness.net/news/AI/european-research-integritycode-updated-reflect-advances-artificial-intelligence
Butler, N., Delaney. H., & Spoelstra S. (2017). The gray zone: questionable research practices in the business school. Academy of Management Learning & Education, 16(1), 94–109. https://doi.org/10.5465/amle.2015.0201
Council of Europe (2023, April 26). EduTalks@Council of Europe – Artificial Intelligence and Academic Integrity. Council of Europe. https://www.coe.int/en/web/education/-/edutalks-council-of-europe-artificial-intelligence-andacademic-integrity
Currie, G. M. (2023). Academic integrity and artificial intelligence: is ChatGPT hype, hero or heresy? Seminars in Nuclear Medicine, 53(5), 719–730. https://doi.org/10.1053/j.semnuclmed.2023.04.008
Davis, M., & Feinerman A. (2010). Assessing graduate student progress in engineering ethics. Science and Engineering Ethics, 18(2), 351–367. https://doi.org/10.1007/s11948-010-9250-2
Davis, M., & Riley K. (2008). Ethics across the graduate engineering curriculum: An experiment in teaching and assessment. Teaching Ethics, 9(1), 25–42. https://doi.org/10.5840/tej20089115
Dawson, A. G. (2023). Artificial intelligence and academic integrity. Aspen Publishing.
Diaz-Martinez, L. A., Fisher, G. R., Esparza, D., Bhatt, J. M., D’Arcy, C. E., Apodaca, J., Brownell, S., Corwin, L., Davis, W. B., Floyd, K. W., Killion, P. J., Madden, J., Marsteller, P., Mayfield-Meyer, T., McDonald, K. K., Rosenberg, M., Yarborough, M. A., & Olimpo, J. T. (2019). Recommendations for effective integration of ethics and responsible conduct of research (E/RCR) education into coursebased undergraduate research experiences: A meeting report. CBE—Life Sciences Education, 18(2), 1–10. https://doi.org/10.1187/cbe.18-10-0203
Eaton, S. E. (2023, March 4). Artificial intelligence and academic integrity, post-plagiarism. University World News. https://www.universityworldnews.com/post.php?story=20230228133041549
Eke, D. O. (2023). ChatGPT and the rise of generative AI: Threat to academic integrity? Journal of Responsible Technology, 13. https://doi.org/10.1016/j.jrt.2023.100060
Fanelli, D. (2013). Redefine misconduct as distorted reporting. Nature, 494(7436), 149. https://doi.org/10.1038/494149a
Gonzáles J., & Wagenaar R. (Eds.) (2008). Universities’ contribution to the Bologna Process: An introduction (2nd Edition). Universidad de Deusto. https://www.unideusto.org/tuningeu/publications/278-universitiesacontribution-to-the-bolognaprocess-an-introduction-english-version.html
Hall, J., & Martin, B. R. (2019). Towards a taxonomy of research misconduct: The case of business school research. Research Policy, 48(2), 414–427. https://doi.org/10.1016/j.respol.2018.03.006
Hauser, F., Reuter, R., Gruber, H., & Mottok, J. (2018). Research competence: Modification of a questionnaire to measure research competence at universities of applied sciences. In C. S. González González, M. Castro & M. Llamas Nistal (Eds.), Proceedings of 2018 IEEE Global Engineering Education Conference (EDUCON 2018) (pp. 109–117). Institute of Electrical and Electronics Engineers. https://doi.org/10.1109/EDUCON.2018.8363216
Hussain, M. M. (2023). The policy efforts to address racism and discrimination in higher education institutions: the case of Canada. Center for Educational Policy Studies Journal, 13(2), 9–29. https://doi.org/10.26529/cepsj.965
Kohlberg, L. (1969). Stage and sequence: The cognitive development approach to socialization. In D. A. Goslin (Ed.), Handbook of socialization theory (pp. 347–480). Rand McNally.
Kohlberg, L. (1976). Moral stages and moralization: The cognitive-developmental. In T. Lickona (Ed.), Moral development and behavior: Theory, research and social issues (pp. 31–53). Holt, Rinehart and Winston.
LinkedIn (2023, August 3). What are some best practices for validating and updating competency models? https://www.linkedin.com/advice/0/what-some-best-practices-validating-updating-1e
Nanda, N. (2021, December 7). Is artificial intelligence a threat to academic integrity? Original by Turnitin. https://www.ouriginal.com/is-ai-a-threat-to-academic-integrity/
NPA Core Competencies Committee (2007-2009). The NPA postdoctoral core competencies. https://www.nationalpostdoc.org/page/CoreCompetencies
OECD Future of Education and Skills 2030 (2019). OECD learning compass 2030: A series of concept notes. http://www.oecd.org/education/2030-project/teaching-and-learning/learning/learningcompass-2030/OECD_Learning_Compass_2030_Concept_Note_Series.pdf
Olatunde Oduoye, M., Javed, B., Gupta, N., & Valentina Sih, C. M. (2023). Algorithmic Bias and Research integrity; the role of non-human authors in shaping scientific knowledge with respect to Artificial Intelligence (AI); a perspective. International Journal of Surgery. Advance online publication. https://doi.org/10.1097/JS9.0000000000000552
Rest, J. (1983). Morality. In P. H. Mussen, J. Flavell, and E. Markman (Eds.), Handbook of Child Psychology, Vol. 3, Cognitive Development (4th ed.) (pp. 556–629). Wiley.
Schaper, N. (2017). Why is it necessary to validate models of pedagogical competency? GMS Journal for Medical Education, 34(4), 1–8. https://doi.org/10.3205/zma00112
Selan, J., Metljak, M., Berčnik, S., & Dagarin Fojkar, M. (2021). Competency profile for teaching and learning research integrity. Faculty of Education University of Ljubljana. https://zalozba.pef.uni-lj.si/index.php/zalozba/catalog/book/176
Steneck, N. (2006). Fostering integrity in research: Definition, current knowledge, and future directions. Science and Engineering Ethics, 12(1), 53–74. https://doi.org/10.1007/PL00022268
The European Centre for the Development of Vocational Training (Cedefop) (2011). Glossary: Quality in education and training. Publications Office of the European Union. https://www.cedefop.europa.eu/files/4106_en.pdf
The Ohio State University (2023, April 6). Artificial intelligence and academic integrity. The Ohio State University. https://oaa.osu.edu/artificial-intelligence-and-academic-integrity
The US National Academies of Sciences, Engineering, and Medicine (2017). Fostering integrity in research. The National Academies Press. https://doi.org/10.17226/21896
The US National Research Council (2002). Integrity in scientific research: Creating an environment that promotes responsible conduct. The National Academies Press. https://doi.org/10.17226/10430
Trachtenberg, B. (2023). ChatGPT, artificial intelligence, and academic integrity. Office of Academic Integrity. University of Missouri. https://oai.missouri.edu/chatgpt-artificial-intelligence-and-academic-integrity/
Turnitin. (2023). Academic integrity in the age of AI. Turnitin. https://www.turnitin.com/resources/academic-integrity-in-the-age-of-AI
University of Cambridge (2023). Artificial Intelligence. University of Cambridge. https://www.plagiarism.admin.cam.ac.uk/what-academic-misconduct/artificial-intelligence
York University (2023, August 16). AI technology and academic integrity for instructors. York University. https://www.yorku.ca/unit/vpacad/academic-integrity/ai-technology-and-academic-integrity/
Zobel, J. (2023, May 19). We need to retain research integrity in the AI era. Pursuit and Research, University of Melbourne. https://pursuit.unimelb.edu.au/articles/we-need-to-retain-research-integrity-in-the-ai-era
In order to ensure both the widest dissemination and protection of material published in CEPS Journal, we ask Authors to transfer to the Publisher (Faculty of Education, University of Ljubljana) the rights of copyright in the Articles they contribute. This enables the Publisher to ensure protection against infringement.
