From scattered company signals to structured sales intelligence, built on a load-tested, multi-tenant Node.js architecture.
From 90+ seconds to under 3 seconds via Promise.all parallelization.
Sustained through the full spike profile with <5% error rate.
End-to-end including multi-source ingestion and LLM synthesis.
The Challenge
Sales teams need rich, current company intelligence, but the underlying data lives across incompatible sources (news feeds, job boards, SEC EDGAR, CRMs) arriving at different cadences. Pulling it together manually is slow; naively automating it creates noise, runaway LLM costs, and a system that collapses under concurrent multi-tenant load.
Approach
- 01Multi-source ingestion: news feeds, job boards, SEC EDGAR, and CRM data pulled into a unified pipeline
- 02Async worker architecture: dedicated BullMQ workers handle enrichment, monitoring, and CRM sync independently
- 03Layered AI filtering: deduplication and ICP screening before LLM synthesis to reduce noise and cost
- 04Pool-aware concurrency: explicit queue and connection limits prevent any tenant from degrading the system
Architecture & data model
Defined the multi-tenant schema, settled on stateless API + async worker topology, and wired Postgres + Redis + Supabase auth into the monorepo.
Enrichment pipeline
Built the fan-out pipeline — research intake, relevance filtering, ICP screening, and LLM synthesis — each stage its own BullMQ worker with an independent concurrency cap.
Concurrency & cost controls
Added per-tenant queue budgets and route-level PQueue limits to prevent pool exhaustion. Layered heuristic pre-filters before LLM calls to keep inference costs bounded.
Latency optimization + load testing
Rewrote bootstrap endpoints to parallelize independent fetches with Promise.all, cutting p95 from 90+ seconds to under 3 seconds. Validated with k6 smoke, default, and spike profiles up to 300 VUs.
Main intelligence hub showing active dossiers, enrichment status, and company signals across all tracked accounts.
Tracked company list with ICP fit scores, hiring signals, tech footprint, and last-touched timestamps for each account.
AI-generated ideal customer profile scoring that weighs company size, industry, tech stack, and buying signals against predefined criteria.
Competitive positioning brief auto-generated from signal data, surfacing differentiators and objection-handling talking points per account.
Real-time monitoring pipeline watching for hiring activity, news mentions, and tech stack changes that indicate buying intent.
