Advances in Text and Code Readability Assessment

The field of text and code readability assessment is moving towards more nuanced and human-aligned approaches. Recent studies have highlighted the limitations of traditional metrics and the importance of considering context, information content, and topic in evaluating readability. The use of large language models and machine learning techniques is becoming increasingly prominent in this area, with applications in automatic essay scoring, code readability assessment, and invoice information extraction. Noteworthy papers in this area include: Readability Reconsidered, which found that model-based metrics outperform traditional metrics in capturing human perceptions of readability. Human-Aligned Code Readability Assessment with Large Language Models, which introduced a large-scale benchmark for evaluating LLM-based code readability assessment and found that developer-guided prompting improves alignment with human judgments. ImpossibleBench, which introduced a benchmark framework for measuring LLMs' propensity to exploit test cases and provides a versatile tool for studying model behaviors and developing monitoring tools.

Sources

Readability Reconsidered: A Cross-Dataset Analysis of Reference-Free Metrics

Automatic essay scoring: leveraging Jaccard coefficient and Cosine similaritywith n-gram variation in vector space model approach

Invoice Information Extraction: Methods and Performance Evaluation

Human-Aligned Code Readability Assessment with Large Language Models

An Empirical Study of Bitwise Operators Intuitiveness through Performance Metrics

ImpossibleBench: Measuring LLMs' Propensity of Exploiting Test Cases

Built with on top of