Advances in Code Generation and Understanding with Large Language Models

The field of code generation and understanding is rapidly advancing with the help of Large Language Models (LLMs). Recent studies have focused on improving the performance of LLMs in code generation tasks, such as code completion, code summarization, and code execution. One of the key challenges in this area is the lack of robust semantics understanding in LLMs, which can lead to generated code that is not correct or efficient. To address this challenge, researchers have proposed various approaches, including the use of formal semantics, retrieval-augmented generation, and equivalence scores for evaluating the quality of generated code. Another important area of research is the investigation of code smells in LLM-generated code, which can help identify areas for improvement in code quality. Furthermore, studies have shown that LLMs can be used to generate high-level test cases from requirements, and to predict the relative comprehensibility of code snippets. Noteworthy papers in this area include GramTrans, which proposes a novel approach to code representation that improves the performance of LLMs in code generation tasks, and VeriEquivBench, which introduces a new benchmark for evaluating the quality of formally verifiable code generated by LLMs. Overall, the field of code generation and understanding with LLMs is rapidly evolving, with new approaches and techniques being proposed to improve the performance and quality of generated code.

Sources

GramTrans: A Better Code Representation Approach in Code Generation

Investigating The Smells of LLM Generated Code

When Names Disappear: Revealing What LLMs Actually Understand About Code

PLSEMANTICSBENCH: Large Language Models As Programming Language Interpreters

Relative Code Comprehensibility Prediction

Generating High-Level Test Cases from Requirements using LLM: An Industry Study

EvoEngineer: Mastering Automated CUDA Kernel Code Evolution with Large Language Models

Designing Empirical Studies on LLM-Based Code Generation: Towards a Reference Framework

Retrieval-Augmented Code Generation: A Survey with Focus on Repository-Level Approaches

PLSemanticsBench: Large Language Models As Programming Language Interpreters

VeriEquivBench: An Equivalence Score for Ground-Truth-Free Evaluation of Formally Verifiable Code

Vibe Checker: Aligning Code Evaluation with Human Preference

Built with on top of