The field of infrastructure as code is moving towards ensuring the reliability and reproducibility of IaC scripts. Recent studies have focused on developing defect taxonomies and testing frameworks to identify and classify defects in IaC scripts. Additionally, there is a growing emphasis on reproducibility in systems and HPC computer science research, with efforts to develop practical and cost-effective methods for making computational experiments reproducible. Noteworthy papers include: A Defect Taxonomy for Infrastructure as Code, which confirms the generalizability of a previously developed defect taxonomy across a broader landscape of IaC tools. OODTE: A Differential Testing Engine for the ONNX Optimizer, which detects issues associated with optimizer crashes and accuracy deviations in ONNX models. WATCH: Weighted Adaptive Testing for Changepoint Hypotheses via Weighted-Conformal Martingales, which proposes a weighted generalization of conformal test martingales for online monitoring and change-point detection.
Advances in Infrastructure as Code and Reproducibility
Sources
A Defect Taxonomy for Infrastructure as Code: A Replication Study
Report on Challenges of Practical Reproducibility for Systems and HPC Computer Science
OODTE: A Differential Testing Engine for the ONNX Optimizer
Context-Aware Online Conformal Anomaly Detection with Prediction-Powered Data Acquisition
Adaptive Scoring and Thresholding with Human Feedback for Robust Out-of-Distribution Detection
Improving the Reproducibility of Deep Learning Software: An Initial Investigation through a Case Study Analysis
Detecting Concept Drift in Neural Networks Using Chi-squared Goodness of Fit Testing
WATCH: Weighted Adaptive Testing for Changepoint Hypotheses via Weighted-Conformal Martingales
The Design Space of Lockfiles Across Package Managers