How Sincera Categorizes 1–2M Records at 85%+ With AI
85%+
Not disclosed
Not disclosed
Sincera receives millions of unstructured records monthly describing products and services from diverse internet sources. The records lack consistency — some represent customer segments like 'Lipton purchaser' while others describe products like 'pork tenderloin' — and needed to be standardized into a usable format.
Fractional AI built an AI categorization system that maps records to Shopify's product taxonomy of 10,000 categories with up to seven levels of hierarchy.
A multi-step LLM pipeline runs five sequential agents: CLASSIFY determines whether a record is a brand, segment, or uncategorizable; ENRICH expands it with richer descriptions; DISCERN selects the best candidate category; JUDGE evaluates the decision and assigns a confidence score; and a final step returns categorization with confidence metrics. It uses Claude-3.5-Sonnet, Llama-3.1, and GPT-4o-mini, with Braintrust for evaluations, OpenAI Structured Output, and a modified-RAG vector search workflow.
Best fit for data or adtech companies that must normalize high volumes of messy, inconsistent records into a structured taxonomy in real time.
85%+ accuracy
1-2M records monthly
Real-time processing
Chris Taylor
Through a five-agent AI pipeline that classifies each record, enriches it with a fuller description, selects the best category via vector search, and judges the result with a confidence score. It maps records to Shopify's 10,000-category taxonomy in real time at accuracy consistently above 85%.
The pipeline uses Claude 3.5 Sonnet, Llama 3.1, and GPT-4o-mini across its stages, with OpenAI Structured Output for machine-readable results, a modified-RAG vector search to shortlist categories, and Braintrust for evaluation.
The system holds categorization accuracy consistently above 85%, processes one to two million unstructured records per month, and works in real time against a 10,000-category taxonomy.
Records are categorized in real time as they arrive, at a volume of one to two million per month. A separate implementation timeline was not disclosed.
It is best suited to data or adtech companies that must normalize high volumes of messy, inconsistent records into a structured taxonomy in real time.
Have a similar challenge?
