Abstract:With the rapid advancement of large language models (LLMs), foundational models (FMs) have seen significant advancements. Healthcare is one of the most crucial application areas for these FMs, given the significant time and effort required for physicians to analyze large volumes of patient data. Recent efforts have focused on adapting multimodal FMs to the medical domain through techniques like instruction-tuning, leading to the development of medical foundation models (MFMs). However, these approaches typically require large amounts of training data to effectively adapt models to the medical field. Moreover, most existing models are trained on English datasets, limiting their practicality in non-English-speaking regions where healthcare professionals and patients are not always fluent in English. The need for translation introduces additional costs and inefficiencies. To address these challenges, we propose a \textbf{J}apanese \textbf{Radi}ology report generation model enhanced by \textbf{Evo}lutionary optimization of model merging (JRadiEvo). This is the first attempt to extend a non-medical vision-language foundation model to the medical domain through evolutionary optimization of model merging. We successfully created a model that generates accurate Japanese reports from X-ray images using only 50 translated samples from publicly available data. This model, developed with highly efficient use of limited data, outperformed leading models from recent research trained on much larger datasets. Additionally, with only 8 billion parameters, this relatively compact foundation model can be deployed locally within hospitals, making it a practical solution for environments where APIs and other external services cannot be used due to strict privacy and security requirements.

Ruri: Japanese General Text Embeddings

Japanese SimCSE Technical Report

Character n-gram Embeddings to Improve RNN Language Models

Multilingual E5 Text Embeddings: A Technical Report

Towards Better Monolingual Japanese Retrievers with Multi-Vector Models

Radical-level Ideograph Encoder for RNN-based Sentiment Analysis of Chinese and Japanese

Building a Japanese Document-Level Relation Extraction Dataset Assisted by Cross-Lingual Transfer

mGTE: Generalized Long-Context Text Representation and Reranking Models for Multilingual Text Retrieval

The Russian-focused embedders' exploration: ruMTEB benchmark and Russian embedding model design

MMM: Multilingual Mutual Reinforcement Effect Mix Datasets & Test with Open-domain Information Extraction Large Language Models

RakutenAI-7B: Extending Large Language Models for Japanese

Constructing Multimodal Datasets from Scratch for Rapid Development of a Japanese Visual Language Model

Joint Model Using Character and Word Embeddings for Detecting Internet Slang Words

CodeXEmbed: A Generalist Embedding Model Family for Multiligual and Multi-task Code Retrieval

jina-embeddings-v3: Multilingual Embeddings With Task LoRA

2D Matryoshka Sentence Embeddings

Multilingual Extractive Reading Comprehension by Runtime Machine Translation

Synthetic Scene Character Generator and Ensemble Scheme with the Random Image Feature Method for Japanese and Chinese Scene Character Recognition

Improving Text Embeddings with Large Language Models

Morphological Analysis of Japanese Hiragana Sentences using the BI-LSTM CRF Model

JRadiEvo: A Japanese Radiology Report Generation Model Enhanced by Evolutionary Optimization of Model Merging