Advances in Question Answering for Sustainability and Multimodal Reasoning

The field of question answering is moving towards more complex and nuanced tasks, incorporating multimodal reasoning and external tools to enhance problem-solving capabilities. Recent developments have focused on creating comprehensive datasets for specific domains, such as corporate sustainability and lifelogging, to support the development of advanced knowledge assistants. These datasets often require innovative approaches to data generation and annotation, such as the use of semantic chunk classification and hybrid span extraction pipelines. The integration of large language models and retrieval-augmented generation systems has also shown promising results in improving question answering performance. Notably, the use of real-world visual contexts and challenging implicit multi-step reasoning tasks has been shown to better align with real user interactions. Overall, the field is advancing towards more realistic and practical applications, with a focus on developing systems that can navigate complex sustainability compliance and provide insights into daily life. Noteworthy papers include: SustainableQA, which introduces a comprehensive dataset for corporate sustainability question answering, and ToolVQA, which proposes a large-scale multimodal dataset for multi-step reasoning VQA with external tools. CF-RAG and OpenLifelogQA also present innovative approaches to carbon footprint question answering and lifelog question answering, respectively.

Sources

SustainableQA: A Comprehensive Question Answering Dataset for Corporate Sustainability and EU Taxonomy Reporting

ToolVQA: A Dataset for Multi-step Reasoning VQA with External Tools

CF-RAG: A Dataset and Method for Carbon Footprint QA Using Retrieval-Augmented Generation

OpenLifelogQA: An Open-Ended Multi-Modal Lifelog Question-Answering Dataset

Built with on top of