Following the existing vision-language pre-training models, we employ a dual-encoder architecture for vision-language representation learning. Since the encoders of visual and textual modalities are decoupled, we explore different encoder architectures. Benchmark code is available at Mindspore and Pytorch project.

We evaluate our models on several tasks, for zero-shot classification task:

For zero-shot image text retrieval task:

And for finetuned retrieval task:

Below is some visualization examples of our models.