Dr Donghwan Shin
School of Computer Science
Lecturer in Testing
School Student Experience Lead
Member of the Testing Research Group
D.Shin@sheffield.ac.uk
Regent Court (DCS)
Full contact details
Dr Donghwan Shin
School of Computer Science
Regent Court (DCS)
211 Portobello
Sheffield
S1 4DP
School of Computer Science
Regent Court (DCS)
211 Portobello
Sheffield
S1 4DP
- Profile
-
Dr Donghwan Shin has been a Lecturer in Software Testing in the School of Computer Science, University of Sheffield since 2022. He did his BSc (2006-2010), MSc (2010-2012), and PhD (2012-2018) at KAIST, South Korea. This was followed by four years as a research associate (2018-2020) and research scientist (2020-2022) at SnT, University of Luxembourg. His work has been funded by the EPSRC (UK) and the National Research Foundation of Korea (South Korea).
- Research interests
-
His research and teaching interests lie in mutation testing, testing for ML-enabled cyber-physical systems (e.g., ML-enabled automated driving systems), and log analysis (e.g., model inference and anomaly detection). He has published research papers at top venues such as ICSE, ICST, ISSTA, and MODELS and journals such as TSE, EMSE, and STVR.
- Publications
-
Journal articles
- Software testing for extended reality applications: a systematic mapping study. Automated Software Engineering, 32(2). View this article in WRRO
- Impact of log parsing on deep learning-based anomaly detection. Empirical Software Engineering, 29(6). View this article in WRRO
- The 5thWorkshop on Testing for Deep Learning and Deep Learning for Testing (DeepTest 2024). Proceedings 2024 IEEE ACM International Workshop on Deep Learning for Testing and Testing for Deep Learning Deeptest 2024, VII.
- Systematic evaluation of deep learning models for log-based failure prediction. Empirical Software Engineering, 29. View this article in WRRO
- Rigorous assessment of model inference accuracy using language cardinality. ACM Transactions on Software Engineering and Methodology, 33(4), 1-39. View this article in WRRO
- Virtual environment model generation for CPS goal verification using imitation learning. ACM Transactions on Embedded Computing Systems, 23(1), 1-29. View this article in WRRO
- An extensible modeling method supporting ontology-based scenario specification and domain-specific extension. International Journal of Software Engineering and Knowledge Engineering, 34(1), 91-162. View this article in WRRO
- Identifying the hazard boundary of ML-enabled autonomous systems using cooperative co-evolutionary search. IEEE Transactions on Software Engineering, 49(12), 5120-5138. View this article in WRRO
- Summary of the fourth international workshop on deep learning for testing and testing for deep learning (DeepTest 2023). ACM SIGSOFT Software Engineering Notes, 48(4), 39-40. View this article in WRRO
- PRINS : scalable model inference for component-based system logs. Empirical Software Engineering, 27(4).
- Can offline testing of deep neural networks replace their online testing?. Empirical Software Engineering, 26(5).
- Empirical evaluation of mutation‐based test case prioritization techniques. Software Testing, Verification and Reliability, 29(1-2).
- A Theoretical and Empirical Study of Diversity-Aware Mutation Adequacy Criterion. IEEE Transactions on Software Engineering, 44(10), 914-931.
- An Applicability Study of Action-Benefit-Cost Model and Statistical Model Checking for System of Systems Goal Achievement Verification. KIISE Transactions on Computing Practices, 23(4), 256-261.
- A Case Study for Mutation-based Fault Localization for FBD Programs. KIISE Transactions on Computing Practices, 22(3), 145-150.
- Comprehensive analysis of FBD test coverage criteria using mutants. Software & Systems Modeling, 15(3), 631-645.
- Quality-Based Software Project Staffing and Scheduling with Project Deadline. Journal of KIISE, 42(7), 878-888.
- Human Resource Allocation in Software Project with Practical Considerations. International Journal of Software Engineering and Knowledge Engineering, 25(01), 5-26.
- Automated test case generation for FBD programs implementing reactor protection system software. Software Testing, Verification and Reliability, 24(8), 608-628.
- Identification of quality improvement strategies using COPQ in software industry. International Journal of Performability Engineering, 8(6), 679-688.
Book chapters
- Autonomous Driving System Testing: Traffic Density Does Matter, Lecture Notes in Computer Science (pp. 315-331). Springer Nature Switzerland
- Towards log slicing In Lambers L & Uchitel S (Ed.), Fundamental Approaches to Software Engineering (pp. 249-259). Springer Nature Switzerland
- A Theoretical Framework for Understanding the Relationship Between Log Parsing and Anomaly Detection, Lecture Notes in Computer Science (pp. 277-287). Springer International Publishing
- Empirical Evaluation on FBD Model-Based Test Coverage Criteria Using Mutation Analysis, Lecture Notes in Computer Science (pp. 465-479). Springer Berlin Heidelberg
Conference proceedings
- Empirically evaluating flaky tests for autonomous driving systems in simulated environments. 2025 IEEE/ACM International Flaky Tests Workshop (FTW). Ottawa, Ontario, Canada, 27 April 2025 - 27 April 2025. View this article in WRRO
- Can test generation and program repair inform automated assessment of programming projects?. 2025 IEEE Conference on Software Testing, Verification and Validation (ICST) (pp 699-710). Naples, Italy, 31 March 2025 - 31 March 2025. View this article in WRRO
- Towards simplification of failure scenarios for machine learning-enabled autonomous systems. 2024 IEEE 24th International Conference on Software Quality, Reliability, and Security Companion (QRS-C) (pp 1089-1090). Cambridge, United Kingdom, 1 July 2024 - 1 July 2024. View this article in WRRO
- Toward automated compliance checking of fund activities using runtime verification techniques. FinanSE '24: Proceedings of the 1st IEEE/ACM Workshop on Software Engineering Challenges in Financial Firms, Vol. 18 (pp 19-20). New York, United States, 16 April 2024 - 16 April 2024. View this article in WRRO
- Mutation-based Consistency Testing for Evaluating the Code Understanding Capability of LLMs. Proceedings of the IEEE/ACM 3rd International Conference on AI Engineering - Software Engineering for AI (pp 150-159)
- Tuning the feedback controller gains is a simple way to improve autonomous driving performance. 2024 UKACC 14th International Conference on Control (CONTROL) (pp 72-77). Winchester, United Kingdom, 10 April 2024 - 10 April 2024. View this article in WRRO
- Many-objective reinforcement learning for online testing of DNN-enabled systems. 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE) (pp 1814-1826). Melbourne, Australia, 14 May 2023 - 14 May 2023. View this article in WRRO
- Guidelines for assessing the accuracy of log message template identification techniques. ICSE '22: Proceedings of the 44th International Conference on Software Engineering (pp 1095-1106). Pittsburgh, Pennsylvania, USA, 21 May 2022 - 21 May 2022. View this article in WRRO
- Efficient online testing for DNN-enabled systems using surrogate-assisted and many-objective optimization. Proceedings of the 44th International Conference on Software Engineering (pp 811-822). Pittsburgh, Pennsylvania, USA, 21 May 2022 - 21 May 2022. View this article in WRRO
- Automatic test suite generation for key-points detection DNNs using many-objective search (experience paper). Proceedings of the 30th ACM SIGSOFT International Symposium on Software Testing and Analysis (pp 91-102)
- Log-based slicing for system-level test cases. Proceedings of the 30th ACM SIGSOFT International Symposium on Software Testing and Analysis (pp 517-528)
- Digital Twins Are Not Monozygotic – cross-replicating ADAS testing in two industry-grade automotive simulators. 2021 14th IEEE Conference on Software Testing, Verification and Validation (ICST) (pp 383-393). Porto de Galinhas, Brazil, 12 April 2021 - 12 April 2021. View this article in WRRO
- Comparing Offline and Online Testing of Deep Neural Networks: An Autonomous Car Case Study. 2020 IEEE 13th International Conference on Software Testing, Validation and Verification (ICST), 24 October 2020 - 28 October 2020.
- Formal Modeling and Verification of a Federated Byzantine Agreement Algorithm for Blockchain Platforms. 2019 IEEE International Workshop on Blockchain Oriented Software Engineering (IWBOSE) (pp 11-21), 24 February 2019 - 24 February 2019.
- Are mutation scores correlated with real fault detection?. Proceedings of the 40th International Conference on Software Engineering (pp 537-548)
- ABC
+. Proceedings of the 33rd Annual ACM Symposium on Applied Computing (pp 1698-1701)
- Mutation Analysis for System of Systems Policy Testing. 2017 IEEE/ACM Joint 5th International Workshop on Software Engineering for Systems-of-Systems and 11th Workshop on Distributed Software Development, Software Ecosystems and Systems-of-Systems (JSOS) (pp 16-22), 23 May 2017 - 23 May 2017.
- Diversity-aware mutation adequacy criterion for improving fault detection capability. 2016 IEEE Ninth International Conference on Software Testing, Verification and Validation Workshops (ICSTW) (pp 122-131). Chicago, IL, USA, 11 April 2016 - 11 April 2016. View this article in WRRO
- A theoretical framework for understanding mutation-based testing methods. 2016 IEEE International Conference on Software Testing, Verification and Validation (ICST) (pp 299-308). Chicago, IL, USA, 11 April 2016 - 11 April 2016. View this article in WRRO
- Modeling and verification for different types of system of systems using PRISM. Proceedings of the 4th International Workshop on Software Engineering for Systems-of-Systems (pp 12-18)
- A runtime verification framework for dynamically adaptive multi-agent systems. 2016 International Conference on Big Data and Smart Computing (BigComp) (pp 509-512), 18 January 2016 - 20 January 2016.
- Quality Based Software Project Staffing and Scheduling with Cost Bound. 2015 Asia-Pacific Software Engineering Conference (APSEC) (pp 269-276), 1 December 2015 - 4 December 2015.
- Efficient Testing of Self-Adaptive Behaviors in Collective Adaptive Systems. 2015 IEEE 39th Annual Computer Software and Applications Conference (pp 216-221), 1 July 2015 - 5 July 2015.
- Quality based software project staffing and scheduling with budget and deadline. Ceur Workshop Proceedings, Vol. 1519 (pp 58)
- Practical human resource allocation in software projects using genetic algorithm. Proceedings of the International Conference on Software Engineering and Knowledge Engineering Seke, Vol. 2014-January(January) (pp 688-694)
- On the Promises and Challenges of AI-Powered XR Glasses as Embodied Software. Proceedings of the 2nd ACM International Conference on AI-Powered Software
- XRintTest: An Automated Framework for User Interaction Testing in Extended Reality Applications. Proceedings of the 40th IEEE/ACM International Conference on Automated Software Engineering
- Unseen data detection using routing entropy in mixture-of-experts for autonomous vehicles. Proceedings of the 40th IEEE/ACM International Conference on Automated Software Engineering, ASE 2025. Seoul, South Korea, 16 November 2025 - 16 November 2025. View this article in WRRO
- Using causal inference to test systems with hidden and interacting variables: an evaluative case study. EASE '25: Proceedings of the 29th International Conference on Evaluation and Assessment in Software Engineering. Istanbul, Turkey, 17 June 2025 - 17 June 2025. View this article in WRRO
Preprints
- Using Causal Inference to Test Systems with Hidden and Interacting Variables: An Evaluative Case Study, arXiv.
- Software Testing for Extended Reality Applications: A Systematic Mapping Study, arXiv.
- Taxonomy of Software Log Smells, arXiv.
- Tuning the feedback controller gains is a simple way to improve autonomous driving performance, arXiv.
- Mutation-based Consistency Testing for Evaluating the Code Understanding Capability of LLMs, arXiv.
- Impact of Log Parsing on Deep Learning-Based Anomaly Detection, arXiv.
- Systematic Evaluation of Deep Learning Models for Log-based Failure Prediction, arXiv.
- Identifying the Hazard Boundary of ML-enabled Autonomous Systems Using Cooperative Co-Evolutionary Search, arXiv.
- Rigorous Assessment of Model Inference Accuracy using Language Cardinality, arXiv.
- Many-Objective Reinforcement Learning for Online Testing of DNN-Enabled Systems, arXiv.
- Literature Review to Collect Conceptual Variables of Scenario Methods for Establishing a Conceptual Scenario Framework, arXiv.
- Environment Imitation: Data-Driven Environment Model Generation Using Imitation Learning for Efficient CPS Goal Verification, arXiv.
- PRINS: Scalable Model Inference for Component-based System Logs, arXiv.
- Can Offline Testing of Deep Neural Networks Replace Their Online Testing?, arXiv.
- Digital Twins Are Not Monozygotic -- Cross-Replicating ADAS Testing in Two Industry-Grade Automotive Simulators, arXiv.
- Automatic Test Suite Generation for Key-Points Detection DNNs using Many-Objective Search (Experience Paper), arXiv.
- Effective Removal of Operational Log Messages: an Application to Model Inference, arXiv.
- Comparing Offline and Online Testing of Deep Neural Networks: An Autonomous Car Case Study, arXiv.
- Scalable Inference of System-level Models from Component Logs, arXiv.
- Empirical Evaluation of Mutation-based Test Prioritization Techniques, arXiv.
- A Theoretical Framework for Understanding Mutation-Based Testing Methods.
- Software testing for extended reality applications: a systematic mapping study. Automated Software Engineering, 32(2). View this article in WRRO
- Grants
-
- Simulation-Based Testing for Mobility Cyber-Physical Systems, National Research Foundation of Korea, 04/2025 - 12/2026, £50,011, as PI
- SimpliFaiS: Simplification of Failure Scenarios for Machine Learning-enabled Autonomous Systems, UKRI, 01/2024 - 07/2026, £464,344, as PI