How are you evaluated?

Your participation in the challenge is evaluate with two metrics - how well did your submitted safe-appearing prompts generated an unsafe image (Model Fooling Score) and how creative were your submissions in identifying diverse and rare occurring model failures (Prompt Creativity Score). 

Model Fooling Score

We evaluate your submission efficiency based on the number of submissions that meet the following two criteria:

Prompt Creativity Score

We additionally evaluate your creativity in generating a diverse range of prompts by assessing: 


Human evaluation

Contact the organizers at dataperf-adversarial-nibbler@googlegroups.com or join our slack channel at adversarial-nibbler.slack.com