Work / 2.2 / LumpyLP

Normalizing 448,000 open-licensed tracks across eleven heterogeneous sources

Role
Sole engineer
Scale
~448,000 tracks · 11+ sources
Stack
Node.js · TypeScript · MySQL
Status
Paused · catalog preserved

1Problem

Openly licensed music is fragmented across archives and platforms, each expressing license terms in its own schema. The objective was a single discovery layer, ranked by listener consensus rather than algorithmic curation, in which license eligibility is determined in exactly one place.

2Architecture

  1. Sources
  2. Per-source resolvers
  3. License quality gate
  4. Catalog
  5. API
  6. Discover + voting

Source isolation

Each source is handled by a dedicated resolver that maps its metadata into a common schema, so a malformed upstream feed degrades one source rather than the catalog.

Centralized eligibility

All license logic resides in a single compliance gate, so policy changes apply uniformly across sources and are verifiable in one place.

Geographic discovery

Backfill jobs tagged and geocoded artist locations to support browsing by place; 8,207 of 11,834 artists were resolved to coordinates.

3Failure analysis

3.1 Silent rejection in the compliance gate

Sources serialized identical Creative Commons licenses with modifiers in differing orders, and the gate evaluated each permutation as a distinct license. No error was raised; eligible tracks were discarded silently. Canonicalizing modifier order prior to comparison resolved the defect.

3.2 Pagination ceiling on archive ingestion

Ingest counts for an Internet Archive collection plateaued well below its published size. The search endpoint enforces a maximum pagination depth, so the resolver was migrated to the scrape API's cursor interface, which traverses the complete result set.

3.3 Autonomous local agents underperformed

An autonomous agent for discovering new sources was iterated through six prompt architectures and evaluated across five local 8B-class models; none produced a valid candidate. The replacement decomposes the task: a small model proposes leads, deterministic scripts verify them, and a frontier model classifies the survivors. The revised pipeline completed in minutes and identified the next ingestion target.

4Status

Development is paused. The catalog, pipeline and database snapshots are preserved; this page documents the engineering rather than a live product.