← back to SERVICE

AMI Search

A search tool that cuts long webtoon pages into panels, stores a text and an image vector for each, and finds the scene you want from one natural-language sentence. A product of the AMI project.

stack
Python, Flask, React, TypeScript, PostgreSQL, pgvector, Gemini Embedding 2, Gemini API, Docker, AWS EC2
Time to find a cut

30m → 5m

company-wide rollout planned, up to 500 users

Exclusion violations

7.2% → 2.2%

eval set of 41 queries and 408 judgments, v1 → v2

What I did
  • Started it as a personal project after the Gemini Embedding 2 preview was released and built it on my own, with no outside help (Flask, React, pgvector).
  • For the official version, rebuilt the search UI from the wireframe and designed an AI search that splits a query into search text and conditions.
What the team did
  • Another team is moving it to new infrastructure for the company-wide rollout.

Problem

To find one scene, for example a character image for a banner, someone had to look through dozens of episodes by eye. The old approach had Gemini read every cut’s metadata on each call, which took 2–3 minutes, cost a lot and gave unstable results.

What I built

It cuts long webtoon pages into single cuts and stores two vectors per cut: a text vector from the cut’s analysis and an image vector. A search blends the two scores and finds cuts from one natural-language sentence. It searches by visual features such as appearance, expression and drawing style, filters out features you don’t want, and also accepts an image as the query. Search started after the Gemini Embedding 2 preview was released.

Trial and error

  • Ignored task_type: gemini-embedding-2 returned the same vector whatever task_type value separated queries from documents. I separated them with different text prefixes instead.
  • Exclusions wiping out results: The prefixes raised the median similarity from 0.34 to 0.55, and the old exclusion thresholds then removed every result. Each embedding version now has its own thresholds.
  • Text and image weights: In a real scoring comparison, a text weight of 0.5 beat 0.7.
  • Text-only cuts: Cuts showing only text polluted the results, so they are filtered with a flag and a similarity check against a centroid vector.
  • Index limit: 3072-dimension vectors exceed the pgvector approximate-index limit (2,000 dimensions), so search runs as a sequential scan.
  • Reading the query: The search log showed 58% of queries were a single word, and sentences with conditions were embedded as they were. I designed an AI search that splits a query into search text and conditions.

Quality and operations

  • Eval set: A fixed eval set of 41 queries and 408 judgments compares versions. In v2, exclusion-term violations fell from 7.2% to 2.2% and precision@10 rose from 45.7% to 49.3%.
  • Speed and cost: Even as a sequential scan, results come back in about 1.1 seconds on average over the 41 queries. Image embedding moved to the Batch API, which halved its cost.
  • Running it: The first version ran on the same EC2 host as Shorts in Docker, with the same PostgreSQL (with pgvector) as local development.

Results

  • Finding a cut went from 30 minutes to about 5 minutes.
  • Teams used it to find images for platform banners, carousel posts and media content.
  • A company-wide rollout is planned, for up to 500 users.

Getting ready for the company-wide rollout

In the first version, users had to upload images themselves. For the company-wide rollout, Search is moving to infrastructure that pulls the original files directly from S3, and another team is handling that move. The new API accepts fewer fields, so filters it does not support are hidden in the UI.