Analysis
Turning raw lots into a catalog.
The hard part isn't scraping a price — it's deciding which bottle it belongs to, keeping stats fresh, and exposing history in a way that's useful on a chart and in a CSV download.
Bottle normalization
The normalization pipeline matches incoming titles to existing bottles via normalized names and aliases, or creates new bottle records when no match exists. Categories (bourbon, scotch, Irish, etc.) are inferred from title patterns.
bottles table
Canonical catalog with cached aggregates: price_count, avg_price, min/max, last sale, trend direction.
bottle_aliases
Alternate spellings and retailer-specific titles mapped to one bottle ID — critical when the same expression appears across six auction houses.
USD normalization
Every price stored in original currency plus price_usd for cross-source comparison and charting.
Price intelligence on the site
| Feature | Implementation | User-facing page |
|---|---|---|
| Price history charts | PriceService — daily / weekly / monthly aggregation | /bottles/{id} |
| Trending bottles | Recent activity scoring on price inserts | /trending |
| Market overview | market_stats monthly auction volumes | /market |
| Brand browse | Distinct brands with category filters | /brands |
| Search | PostgreSQL full-text + autocomplete API | /search |
| CSV / JSON export | Authenticated export routes | /api/export |
Market statistics
Monthly aggregates per auction house — mean winning bid, lot counts, trading volume — imported from Kaggle seed data and extended as live scrapers add new hammer prices.
market_stats
├── auction_name, auction_slug
├── period_date
├── winning_bid_min / max / mean
├── trading_volume
└── lots_countMaintenance tasks
Celery beat runs background jobs beyond scraping:
- refresh_bottle_stats — recomputes cached min/max/avg on bottles after bulk imports
- Cleanup tasks — old scrape run records, stale session data
- Export scripts —
export_dataset.sh→ Hugging Face + Kaggle via shared dataset cards