12/5/23, 9:05 PM Mass Produce XY Plots with ComfyUI API

Mass Produce XY Plots with ComfyUI API

Save time by queuing up hundreds of XY plots rather than doing it 1 by 1
FOLLOWFOX.AI
OCT 1, 2023

3 2 Share

Hello FollowFox Community!

In this post, we will look at ComfyUIs API and automate the process of XY graph
creation for significant time savings. As usual, we will do a step-by-step guide on
achieving this and share all the code and resources used in the process.

We will not cover Comfy basics here, but for that, you can check our four-part series on
starting from zero to doing more advanced workflows:

Part 1 (link), we implemented the simplest SDXL Base workflow and generated our
first images

Part 2 (link)- We added SDXL-specific conditioning implementation + tested the

impact of conditioning parameters on the generated images.

Part 3 (link) - we added the refiner for the full SDXL process.

Part 4 (link) - We installed custom nodes and built out workflows with img2img,
controlnets, and LoRAs.

You can support us by joining and testing our image generation service on Discord - Distillery.
You will get daily free generations there! It has had some serious upgrades lately, and we aspire
to become one of the most flexible and powerful image generation services powered by open-
source progress.

https://followfoxai.substack.com/p/mass-produce-xy-plots-with-comfyui 1/11
12/5/23, 9:05 PM Mass Produce XY Plots with ComfyUI API

https://discord.gg/7pda9z6btE

XY Plots in Comfy
In this post, we will be a little less granular on ComfyUI node parts - the four posts linked above
are the material that should develop a general understanding of how these nodes work.

XY plots are powerful when testing different parameters and combinations for image
generation. However, unlike Automatic1111, ComfyUI has less support for such plots
and takes some time to get used to it.

In this post, we will show examples of testing LoRAs and LoRA weights with XY plots,
but the approach is transferrable, and you can apply it to whatever parameters you
intend to test. Let’s start with our setup right away (download this image and drag it on
your ComfyUI canvas to copy it).

https://followfoxai.substack.com/p/mass-produce-xy-plots-with-comfyui 2/11
12/5/23, 9:05 PM Mass Produce XY Plots with ComfyUI API

Let’s quickly review key things that are happening here.

The Efficient Loader combines a few nodes typically responsible for loading a model (we
use the Bloodymary in this case), creating input empty latent images and text prompts.
The interesting part is the Dependencies output slot - a new thing needed for XY
plotting; it ‘tells’ the next node what the levers can change during the image generation
process.

https://followfoxai.substack.com/p/mass-produce-xy-plots-with-comfyui 3/11
12/5/23, 9:05 PM Mass Produce XY Plots with ComfyUI API

The next node - XY Plot is what powers the plot creation process. It takes as inputs the
Dependencies that we just mentioned and the X and Y inputs that we will discuss.
These input slots can be connected to a bunch of XY Input nodes - each allowing to
iterate over a different set of parameters.

In this case, we are using XY Input: LoRA Plot node since we want to iterate over
different LoRA versions and LoRA weights.

https://followfoxai.substack.com/p/mass-produce-xy-plots-with-comfyui 4/11
12/5/23, 9:05 PM Mass Produce XY Plots with ComfyUI API

We have three LoRA files placed in the folder ‘ComfyUI\models\loras\xy_loras’. That’s

why we need to set the path to the folder on this node and set X_Batch_count to three.
The Y batch count will determine how many different strength values will be tested in
the range we can define.

https://followfoxai.substack.com/p/mass-produce-xy-plots-with-comfyui 5/11
12/5/23, 9:05 PM Mass Produce XY Plots with ComfyUI API

Connect everything to the Ksampler Adv. Efficient, ensure that the sampler state is set
to script and we are ready to generate our XY Plot.

https://followfoxai.substack.com/p/mass-produce-xy-plots-with-comfyui 6/11
12/5/23, 9:05 PM Mass Produce XY Plots with ComfyUI API

As you can see, we tested three different LoRAs at three different strength values in this
case.

Limitations and Time Requirements

This is awesome but not perfect. First, one can only control two parameters through the
X and Y axes. More importantly, though, you have to generate one XY plot, update
prompts/parameters, and generate the next one, and when doing this at scale, it takes
hours. We solved this for Automatic1111 through API in this post (link), and we will do
something similar here.

Running ComfyUI with API

What’s great is that ComfyUI comes with a built-in API. Let's start by figuring out how
to run things via API, and then we can automate it.

To start, launch ComfyUI as usual and go to the WebUI. From the settings, make sure to
enable Dev mode Options.

https://followfoxai.substack.com/p/mass-produce-xy-plots-with-comfyui 7/11
12/5/23, 9:05 PM Mass Produce XY Plots with ComfyUI API

Now you should be able to see the Save (API Format) button, pressing which will
generate and save a JSON file.

You can see that we have saved this file as xyz_tempate.json (link). The file itself is not
extremely readable, but with some attention, it becomes clear that each ‘paragraph’ in
JSON corresponds to one of the nodes in ComfyUI.

https://followfoxai.substack.com/p/mass-produce-xy-plots-with-comfyui 8/11
12/5/23, 9:05 PM Mass Produce XY Plots with ComfyUI API

Let’s figure out how to run the job from this file. With ComfyUI, it is extremely easy. We
just need to load the JSON file to a variable and pass it as a request to ComfyUI. Here is
an example script that does that (link). You can test this by ensuring your Comfy is
running and launching this script using a terminal. You’ll see that Comfy’s terminal will
run this job as usual.

This gives us all the necessary clues to automate XY plot mass production. We will use
this API and, from another script, just modify things like prompts and other parameters
as needed.

Automating XY Plots with API

As the first step, we created a Google Sheet to make inputting all the different prompts
and parameters easy. In this case, we wanted to test 45 different prompts with custom
parameters each; each prompt was run with 15 different LoRA versions, 6 LoRA weights
each. So, a total of ~4000 images.

If you want to use our Google Sheets template, just make a copy of this file to your drive
and edit it (link).

Once the Sheet was ready, we downloaded and saved it as a .csv file in our working
directory (link).

https://followfoxai.substack.com/p/mass-produce-xy-plots-with-comfyui 9/11
12/5/23, 9:05 PM Mass Produce XY Plots with ComfyUI API

The final step was writing a script using a loop to read that .csv file and launching a
ComfyUI API request for each row. You can see the script here - simply run it with your
terminal. (link)

The result is amazing and saves us a ton of time. LoRA testing is just one simple use
case; you can get creative to use it in many more ways.

As always, let us know what you think!

3 Likes

2 Comments

https://followfoxai.substack.com/p/mass-produce-xy-plots-with-comfyui 10/11
12/5/23, 9:05 PM Mass Produce XY Plots with ComfyUI API

Write a comment...

dellatm Oct 13
Great method for evaluating different model at different steps! Can't you do the same with
Automatic1111 API?
LIKE REPLY SHARE

1 reply by followfox.ai

1 more comment...

© 2023 followfox.ai ∙ Privacy ∙ Terms ∙ Collection notice

Substack is the home for great writing

https://followfoxai.substack.com/p/mass-produce-xy-plots-with-comfyui 11/11