Open VLM Retrieval Leaderboard

This leaderboard presents the performance of various visual embedding models across different business sectors and languages. The evaluation is based on retrieval accuracy for visual search tasks.

Structure

• Sectors: Each column represents a different business sector (e.g., Energy, Education) with documents in either English (_EN) or French (_FR)
• Models: Each row shows a different model's performance
• Scores: Values range from 0 to 1, where higher is better (1.000 being perfect retrieval)
• Average: Overall mean performance across all sectors for each model
• Colors: Blue backgrounds indicate EU models, red backgrounds indicate Chinese models

The leaderboard was created in collaboration with the Intelligence Lab of the ECE - Ecole centrale d'électronique.

How to Read the Results

• Select a language tab to see how models perform with queries in that language
• All scores are normalized retrieval accuracy metrics
• Background colors indicate model origins (Blue = EU, Red = Chinese)

Additional Information

• Scores are updated regularly as new models are evaluated
• All evaluations use the same test set for fair comparison
• Models are evaluated on both English and French datasets to assess cross-lingual capabilities
• Color coding indicates model origin (Blue = EU, Red = Chinese)

Citation

If you use these benchmarks in your research, please cite:

@article{visual_embeddings_benchmark_2025,
    title={Cross-lingual Visual Embeddings Benchmark},
    author={racine.ai},
    year={2025}
}