Open Science for Foundation Models (SCI-FM)

About the Workshop

Foundation models (FMs) have transformed AI research but lack scientific transparency. The SCI-FM workshop aims to address this by fostering open science, reproducibility, and the sharing of open-source models and datasets. We invite contributions that explore key aspects of FMs, such as dataset curation, evaluation methodologies, and innovative training strategies. Join us in advancing the accessibility and transparency of foundation models for the global research community.

Call for Papers

The SCI-FM workshop invites submissions of research papers and industrial papers, ranging from preliminary research results and visionary papers to full-length papers. Submissions should follow the ICLR proceedings format and fall into the following categories:

• Short paper: up to 6 pages + unlimited references and appendix.
• Regular paper: up to 9 pages + unlimited references and appendix.

We invite papers on topics including, but not limited to:

• Open Datasets: Acquisition, curation, and synthesis of pretraining, instruction, and preference datasets through manual or algorithmic methods. Open access to instruction and preference datasets for alignment research.
• Open Foundation Models: Pretraining strategies including data scaling, model architecture, multi-modal, and multi-task pretraining. Learning algorithms such as meta-learning, model fusion, model merging, and continual learning designed for open, scalable models. Inference algorithms like decoding, reasoning, search, and planning, tailored for foundation models.
• Open Training Protocols: Training dynamics research on scaling laws, interpretability, complexity analysis, emergent capabilities, and phenomena like grokking. Alignment techniques including prompt tuning, prefix tuning, instruction tuning, and reinforcement learning with human/AI feedback.
• Open Evaluation: Benchmark development and the creation of transparent evaluation protocols and metrics, including the open sharing of benchmark datasets and evaluation results across different foundation models.
• Open Compute Efficiency Techniques: Focus on model distillation, compression, quantization, and optimizing attention or memory mechanisms for improved compute efficiency in open foundation models.
• Open Multi-Modal Foundation Models: Expanding to modalities like vision, audio, and multi-modal foundation models, with extra emphasis on underexplored areas such as chemistry, medicine, and education.
• Open Interactive and Agent Systems: Open development of conversational AI, interactive learning models, multi-agent systems, and integration with external tools and APIs.
• Open Replication of Proprietary Systems: Efforts to replicate and openly share foundation models and systems that were previously proprietary, ensuring transparency and reproducibility for broader research and development.

All accepted papers will be presented as posters. Additionally, we will select around 3 papers for short oral presentations and 2 papers for outstanding paper awards with potential cash incentives.

Novelty and Openness

For novelty, the workshop does not accept submissions that have previously been published at ICLR or other machine learning or related venues. For openness, we encourage submissions with sufficient open-source resources (e.g., checkpoint, code, data, training details). Accepted papers will be published on the website, but the workshop is non-archival.

Key Dates

• Submission Open: January 22, 2025
• Submission Deadline: February 10, 2025
• Notifications: March 15, 2025
• Camera-Ready: April 5, 2025

Submission Instructions

• Submission format: use the ICLR 2025 paper template. Appendix should be included in the same PDF as the main paper.
• Submit you paper through OpenReview.

This year, ICLR is discontinuing the separate “Tiny Papers” track, and is instead requiring each workshop to accept short (3–5 pages in ICLR format, exact page length to be determined by each workshop) paper submissions, with an eye towards inclusion; see https://iclr.cc/Conferences/2025/CallForTinyPapers​​ for more details. Authors of these papers will be earmarked for potential funding from ICLR, but need to submit a separate application for Financial Assistance that evaluates their eligibility. This application for Financial Assistance to attend ICLR 2025 will become available on https://iclr.cc/Conferences/2025/ at the beginning of February and close on March 2nd.

Review

We are inviting reviewers for workshop submissions. If you are interested in reviewing, please register through:

Workshop Schedule (Tentative)

Invited Speakers

Stella Biderman (EleutherAI)

Shayne Longpre (MIT)

Luca Soldaini (Ai2)

Wenhao Huang (ByteDance)

Zhengzhong Liu (Petuum)

Junyang Lin (Alibaba)

Organizing Committee

Jiaheng Liu

(Alibaba, M-A-P)

Riza Batista-Navarro

(University of Manchester)

Qian Liu

(TikTok)

Niklas Muennighoff

(Stanford University)

Ge Zhang

(ByteDance, M-A-P)

Yizhi Li

(University of Manchester, M-A-P)

Xinyi Wang

(UC Santa Barbara)

Willie Neiswanger

(USC)

Our Sponsor