Abstract:Large language models (LLMs) mark a key shift in natural language processing (NLP), having advanced text generation, translation, and domain-specific reasoning. Closed-source models like GPT-4, powered by proprietary datasets and extensive computational resources, lead with state-of-the-art performance today. However, they face criticism for their "black box" nature and for limiting accessibility in a manner that hinders reproducibility and equitable AI development. By contrast, open-source initiatives like LLaMA and BLOOM prioritize democratization through community-driven development and computational efficiency. These models have significantly reduced performance gaps, particularly in linguistic diversity and domain-specific applications, while providing accessible tools for global researchers and developers. Notably, both paradigms rely on foundational architectural innovations, such as the Transformer framework by Vaswani et al. (2017). Closed-source models excel by scaling effectively, while open-source models adapt to real-world applications in underrepresented languages and domains. Techniques like Low-Rank Adaptation (LoRA) and instruction-tuning datasets enable open-source models to achieve competitive results despite limited resources. To be sure, the tension between closed-source and open-source approaches underscores a broader debate on transparency versus proprietary control in AI. Ethical considerations further highlight this divide. Closed-source systems restrict external scrutiny, while open-source models promote reproducibility and collaboration but lack standardized auditing documentation frameworks to mitigate biases. Hybrid approaches that leverage the strengths of both paradigms are likely to shape the future of LLM innovation, ensuring accessibility, competitive technical performance, and ethical deployment.

Fully Open Source Moxin-7B Technical Report

MAP-Neo: Highly Capable and Transparent Bilingual Large Language Model Series

LLM360: Towards Fully Transparent Open-Source LLMs

Evaluation of Open-Source Large Language Models for Metal-Organic Frameworks Research

The Model Openness Framework: Promoting Completeness and Openness for Reproducibility, Transparency, and Usability in Artificial Intelligence

OpenMoE: An Early Effort on Open Mixture-of-Experts Language Models

The Open Source Advantage in Large Language Models (LLMs)

OLMoE: Open Mixture-of-Experts Language Models

BLOOM: A 176B-Parameter Open-Access Multilingual Language Model

MOSS: an Open Conversational Large Language Model

OLMo: Accelerating the Science of Language Models

JetMoE: Reaching Llama2 Performance with 0.1M Dollars

Molmo and PixMo: Open Weights and Open Data for State-of-the-Art Vision-Language Models

How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suites

Baichuan-Omni Technical Report

Code Llama: Open Foundation Models for Code

OpenCoder: The Open Cookbook for Top-Tier Code Large Language Models

2 OLMo 2 Furious

Can Open-Source LLMs Compete with Commercial Models? Exploring the Few-Shot Performance of Current GPT Models in Biomedical Tasks