# NeurIPS 2023 Supplementary Material

This repository contains the supplementary material accompanying the paper titled "Mass-Producing Failures of Multimodal Models" accepted at NeurIPS 2023.

## Abstract

This README provides an overview of the supplementary material provided in this repository. The supplementary material includes additional experiments, code, and results that support the findings and contributions presented in the paper.

## Contents

The supplementary material is organized as follows:

1. `annotate.py`: Script Used for user study. Simply run annotate.py. One can check your answer with User_Study (Answer).xlsx
2. `scrape.py`: Code used scrape from MS-COCO. One can also add openai api to steer the scraping process. 
3. `README.md`: This README file providing an overview of the supplementary material.

## Reproducing Experiments

To reproduce the experiments conducted in the paper, please follow scrape.py for scraping and annotate.py for user study.

