← back to SERVICE

AMI Shorts

An internal tool that analyzes webtoon pages with AI and drafts review shorts: scenario, storyboard, TTS, cut picks and a video draft. The first product of the AMI project. AI drafts, people finish.

stack
Python, Flask, React, TypeScript, PostgreSQL, pgvector, Gemini API, OpenCV, FFmpeg, Docker, AWS EC2, n8n, Google Antigravity
Time per short

8h → 3h

used by 12 planners and media producers

Episode analysis

~5 min

production average, 97.9% success

What I did
  • Built the first version as n8n workflows.
  • Moved from n8n to Flask code, starting from a first draft written with Google Antigravity.
  • Built the webtoon analysis engine that Shorts runs on.
  • Kept improving it based on feedback from the people who used it.

Problem

Webtoon review shorts took a long time to make: reading the work, shaping a scenario, picking cuts and making assets were mostly manual. Strong AI tools already existed, but using them meant people had to change how they worked to fit the tool. Shorts went the other way and fit AI to the existing work.

First try: n8n

The first version was a set of n8n workflows. n8n is a visual tool for connecting steps into automated workflows.

The wall

  • Logic: Some of the logic was hard to express in visual scripts.
  • Speed: Processing was slow.
  • Access: There was no GUI, so I was the only person who could use it.
  • Tracking: 14 webhooks ran with no status tracking and no tests, so when something broke it was hard to find where.

The switch: code

Google Antigravity wrote the first draft, and the code was refined from there into its current form. The rule stayed the same: keep the existing workflow, reflect how the work is actually done, and improve based on feedback from real users.

An analysis engine finds the characters in each cut, reads expressions and dialogue, and works out the context of the scene. From that it builds a scenario, a storyboard, TTS, cut picks and a vertical video draft. Shorts prepares the draft, and people finish the content.

Trial and error

  • Double round trip: Going through n8n meant Flask → n8n → external API → n8n → Flask, which made debugging and deployment hard. I moved the AI and TTS calls into Flask modules.
  • Errors that pile up: Feeding unverified analysis back into the next prompt as context made character-recognition errors accumulate. I added a reference-image step and a verification loop. A more complex two-stage observation pipeline was built, then rolled back to a single call (about 2,100 lines removed).
  • Video rendering: Rendering moved from n8n (37 nodes) to Creatomate to FFmpeg on the server, which removed the per-render cost and the external queue.
  • Cut detection: Cuts are split with OpenCV margin detection instead of a YOLO model. An audit found and fixed wrong progress labels, missed very tall cuts and high memory use.

Quality and operations

  • Analysis time: Production logs show about 5 minutes on average to analyze one episode, with a 97.9% success rate.
  • Cost tracking: Every call records an estimated cost from its tokens, and an admin screen shows usage.
  • Running and deploying: It runs on AWS EC2 in Docker, and the production database is the same PostgreSQL (with pgvector) as local development. Code pushed to git flows into deployment, with production and staging kept apart.

Results

  • Average production time per short went from 8 hours to 3 hours.
  • Besides me, 12 planners and media producers used it.