Skip to content

Commit f19f128

Browse files
Add open parti prompts to docs (#3549)
* Add open parti prompts * More changes
1 parent a94977b commit f19f128

File tree

1 file changed

+9
-2
lines changed

1 file changed

+9
-2
lines changed

docs/source/en/conceptual/evaluation.mdx

Lines changed: 9 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -37,7 +37,8 @@ We cover Diffusion models with the following pipelines:
3737

3838
## Qualitative Evaluation
3939

40-
Qualitative evaluation typically involves human assessment of generated images. Quality is measured across aspects such as compositionality, image-text alignment, and spatial relations. Common prompts provide a degree of uniformity for subjective metrics. DrawBench and PartiPrompts are prompt datasets used for qualitative benchmarking. DrawBench and PartiPrompts were introduced by [Imagen](https://imagen.research.google/) and [Parti](https://parti.research.google/) respectively.
40+
Qualitative evaluation typically involves human assessment of generated images. Quality is measured across aspects such as compositionality, image-text alignment, and spatial relations. Common prompts provide a degree of uniformity for subjective metrics.
41+
DrawBench and PartiPrompts are prompt datasets used for qualitative benchmarking. DrawBench and PartiPrompts were introduced by [Imagen](https://imagen.research.google/) and [Parti](https://parti.research.google/) respectively.
4142

4243
From the [official Parti website](https://parti.research.google/):
4344

@@ -51,7 +52,13 @@ PartiPrompts has the following columns:
5152
- Category of the prompt (such as “Abstract”, “World Knowledge”, etc.)
5253
- Challenge reflecting the difficulty (such as “Basic”, “Complex”, “Writing & Symbols”, etc.)
5354

54-
These benchmarks allow for side-by-side human evaluation of different image generation models. Let’s see how we can use `diffusers` on a couple of PartiPrompts.
55+
These benchmarks allow for side-by-side human evaluation of different image generation models.
56+
57+
For this, the 🧨 Diffusers team has built **Open Parti Prompts**, which is a community-driven qualitative benchmark based on Parti Prompts to compare state-of-the-art open-source diffusion models:
58+
- [Open Parti Prompts Game](https://huggingface.co/spaces/OpenGenAI/open-parti-prompts): For 10 parti prompts, 4 generated images are shown and the user selects the image that suits the prompt best.
59+
- [Open Parti Prompts Leaderboard](https://huggingface.co/spaces/OpenGenAI/parti-prompts-leaderboard): The leaderboard comparing the currently best open-sourced diffusion models to each other.
60+
61+
To manually compare images, let’s see how we can use `diffusers` on a couple of PartiPrompts.
5562

5663
Below we show some prompts sampled across different challenges: Basic, Complex, Linguistic Structures, Imagination, and Writing & Symbols. Here we are using PartiPrompts as a [dataset](https://huggingface.co/datasets/nateraw/parti-prompts).
5764

0 commit comments

Comments
 (0)