Summer brings many wonderful activities and traditions and one of my personal favorites is the summer farmer's market. While many enjoy the fresh fruits and vegetables these markets offer, I also enjoy the baked goods! One of my favorites comes from an elderly woman who sells baked goods (and gently used books) to raise money for various charities. Perhaps her best offering is a homemade cinnamon roll, which—while it might not be good for my waistline—sure seems worth the price.

The End of Summer

Of course, when summer ends, my local farmer's market shuts down. Realizing this would mean the end of my cinnamon roll supply, I decided I needed to be creative.

Have you ever looked at a homemade food label—like one from a local farmer’s market—and wondered how to recreate it yourself? All I had was a picture of the cinnamon roll with the list of ingredients printed on a sticky label.

⊕A cell phone picture showing the list of ingredients from the cinnamon roll.

As I am not a cook nor a baker, I wasn’t sure how I might tackle this problem. But being scientifically-minded, I thought there must be at least three steps:

1. Extract the list of ingredients from the label
2. Find a recipe that matches this list
3. Try the recipe out

Focusing on the first two steps, I was curious if a generative AI tool might be able to help. I uploaded the image of the label to ChatGPT and simply asked:

Do you think you could reverse engineer the recipe from the list of ingredients 
in this picture?

To my surprise⊕In hindsight, I suppose I shouldn't have been so naive!, the AI processed the image, extracted the text, identified the list of ingredients⊕The image I provided above was cropped from the full photo showing the cinnamon roll wrapped in plastic with a sticky label that included the ingredients, the baker’s contact info, weight in grams, and price., and cleaned it up by removing any irrelevant text. Then, it inferred a suitable recipe.

Label Processing

The label itself offers very little detail—just a list of ingredients:

Bread Flour, Potatoes, Margarine, Sugar, Eggs, Milk, Salt, Yeast, Water, Butter, Cinnamon, Powdered Sugar, Vanilla, Maple Flavoring

No amounts. No steps. No temperatures. Just the raw components.

ChatGPT’s response?

Great! Based on the label in your photo, here’s a reverse-engineered cinnamon roll recipe using the listed ingredients.

Without any direction on my part, CHatGPT structured the cinnamon roll recipe into three components:

• A yeast-based dough using mashed potato for tenderness (which ChatGPT happily informed me is a common trick in old-fashioned recipes)
• A cinnamon-sugar filling made with softened butter
• A maple-flavored icing made with powdered sugar, vanilla, and maple flavoring

Why Does This Work?

Generative AI is an amazing tool that can process images, extract text, and transform that text based on your instructions. Recipe reconstruction is just one use case. Other tasks that follow this pattern include:

• Translating menus
• Summarizing pages from books or manuscripts
• Extracting and verifying data from invoices
• Analyzing business cards
• Reading prescriptions
• Capturing code from screenshots

So next time you take a photo, remember you’re capturing more than just pixels. You might be holding a dataset waiting to be decoded.

Final Thoughts

Of course, you can adapt this approach to almost any baked good or food label. Just take a picture of the label, upload it to your favorite generative AI tool, and prompt:

Can you reverse engineer a recipe from this picture, which lists the ingredients?

Now, sharp-eyed readers will note I’ve only completed the first two steps in my plan. I’m still waiting on the third—remember, I said I’m neither a cook nor a baker! While I remain hopeful I’ll get to taste these AI-powered cinnamon rolls soon, for now I just ask the original creator for a few extra ones on the side. 🙂

If you'd like to try it yourself, here is the full generated recipe from ChatGPT. If you do bake it, be sure to enjoy one for me!