BOOM - Beyond Only One Modality

EACL 2026 System demostration

Beyond Only One Modality - KIT’s Multimodal Multilingual Lecture Companion

Lecture Translator Slide Translator OmniFusion Instructions

MIT

Key Highlights

Open-source models and pipelines for

Multimodal speech translation: Ambinguity resolution with visual context
Slide/Image translation: Enable multilingual lecture comprehension
Summarization, QA and more

Quickstart Guide

Install using pyParrot framework →

# Download
git clone https://github.com/ai4lt/pyparrot.git

# Enter the repo
cd pyparrot

# Initialize and update submodules
git submodule update --init --recursive

# Install PyParrot
pip install -e .

# Configure BOOM
pyparrot configure boom --config example/boom.yaml

# Build docker container
pyparrot build boom

# Start BOOM
pyparrot start boom

Download

git clone https://github.com/ai4lt/pyparrot.git
Enter the repo

cd pyparrot
Initialize submodules

git submodule update --init --recursive
Install PyParrot

pip install -e .
Configure BOOM

pyparrot configure boom --config example/boom.yaml
Build docker container

pyparrot build boom
Start BOOM

pyparrot start boom

Authors

Sai Koneru Author

Fabian Retkowski Author

Christian Huber Author

Lukas Hilgert Author

Seymanur Akti Author

Enes Yavuz Ugan Author

Alexander Waibel Author

Jan Niehues Author

Acknowledgements

Funded by

EU Project DVPS Funding

Visit homepage →

Want to collaborate?

We welcome issues, pull requests, and research collaborations.

Contact us

@inproceedings{boom,
  title     = {BOOM : Beyond Only One Modality KIT’s Multimodal Multilingual Lecture Companion},
  author    = {Koneru, Sai and Retkowski,Fabian and Huber, Christian and Hilgert, Lukas and Akti, Seymanur and Ugan, Enes Yavuz and Waibel, Alexander and Niehues, Jan},
 booktitle = "Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics: System Demonstrations",
    month = mar,
    year = "2026",
    address = "Rabat, Marroco",
    publisher = "Association for Computational Linguistics",
}