[sm].[offtherecord]

A music library management and DJ tooling platform — part of the samuelmarks.com tool suite. Designed to keep your collection organised, analysed, and in sync across every device you own, without touching your existing workflow.

Self-hosted Non-destructive Cross-platform Mac · Windows · Linux Traktor · Rekordbox · Serato

The problem

Your library lives in too many places

A professional DJ or serious collector typically has music spread across a primary machine, a NAS, one or more USB drives for gigs, and possibly a second location. Keeping all of these in sync is manual, error-prone, and time-consuming.

Tracks go missing after a sync. BPM analysis differs between tools. Cue points disappear when you move a file. There is no single view of what you own, where it lives, or what has been analysed.

offtherecord maintains a central manifest of your entire collection — where every file is, what has been analysed, and what needs syncing — and acts as a quiet background service that keeps everything consistent without breaking your existing setup.

Design principles

01
Non-destructive

Never modifies your files without permission. Works alongside Traktor, Rekordbox, and Serato — does not replace them.

02
Self-hosted

All data stays on your own infrastructure. No cloud dependency, no subscription, no third party with access to your library.

03
Modular

Features are independent modules. Disable what you do not need. New functionality can be added cleanly without touching the core.

04
Cross-platform

Electron desktop app runs on Mac, Windows, and Linux. Server runs in Docker — identical dev and production environments.

05
Extensible API

Well-documented REST API. New integrations can be added by any developer on the team without touching the core codebase.

06
Plays well with others

Reads and writes NML, Rekordbox XML, and Serato formats. Importing from or exporting to your DJ software is a first-class feature.

Technical stack

Desktop appElectron (Node.js + Chromium) — Mac, Windows, Linux
Central APIPHP 8.3 + Slim 4 — REST, hosted in Docker on Proxmox / Portainer
Central DBMariaDB 11 — on-premise, no cloud
Local DBSQLite — per-machine index, syncs with central API
Server agentNode.js — watches NAS storage, processes analysis jobs
Audio analysisaudiowaveform (BBC), aubio, ffmpeg — open source CLI tools
Waveform UIWavesurfer.js — renders from pre-generated peak data
Audio engineWeb Audio API — playback, EQ, tempo in the Electron renderer
AI integrationAnthropic Claude API — cataloging, playlist generation, QC (Phase 2)
InfrastructureDocker Compose — identical dev and prod, deployed via Portainer

Roadmap

Phase 1 — Foundation

Current build

Everything required for daily use. The working core that all future phases build on top of.

Library & storage

File scanning & classification
Watches your music root directory. Automatically classifies files into artist / album / various-artists / non-album (Beatport) / special based on your existing folder structure.
Phase 1
SHA-256 deduplication
The same audio file on multiple devices is one record with many locations — not duplicates. Deduplication is by content hash, not filename.
Phase 1
ID3 / tag metadata extraction
Title, artist, album, year, genre, BPM, key, label, ISRC — read on scan and stored in the local database.
Phase 1
Local SQLite database
Per-machine index of all tracks, cue points, analysis results. Fast queries with no network dependency.
Phase 1
Central MariaDB + REST API
Authoritative record of all tracks, nodes, sync state, and analysis queue. Hosted in Docker on your Proxmox / Portainer instance.
Phase 1
Waveform storage (content-addressed)
Waveform data stored by audio hash — one waveform per unique audio file regardless of how many copies exist across devices. Generated by the BBC audiowaveform tool.
Phase 1

Sync & nodes

Node registration
Every device registers with the central API — Mac, PC, NAS, and USB drives each get a stable UUID. Nodes heartbeat to mark themselves online.
Phase 1
Drift detection
Delta API returns everything changed since a given timestamp. The Electron app calls this on launch and whenever a drive connects.
Phase 1
Sync queue with store-and-forward
Pending transfers are queued per node. USB drives pick up their queue the next time they are connected — no transfers are lost because a device was offline.
Phase 1
NAS agent (Docker container)
Lightweight Node.js agent runs on your Proxmox / Portainer stack, watches NAS storage via NFS mount, reports new files to the API, and processes analysis jobs.
Phase 1

Analysis

BPM analysis (aubio)
Pitch-accurate tempo detection run by the NAS agent. BPM source is tracked (id3 / analysed / manual) so you always know where a value came from. Manual overrides are locked against re-analysis.
Phase 1
Waveform generation (BBC audiowaveform)
Generates .dat peak files used for display. Stored once per unique audio file and reused everywhere — no re-generation on copy or move.
Phase 1
Analysis job queue
New tracks are automatically queued for BPM, waveform, and structure analysis. Jobs are prioritised and processed by the NAS agent in the background.
Phase 1

Preview & playback

In-app audio preview
Double-click any track in the library to load it into the preview panel. Full playback via Web Audio API — no external player required.
Phase 1
Waveform display with beat grid
Rendered from pre-generated peak data via Wavesurfer.js. Beat grid overlay calculated from BPM. Click anywhere on the waveform to seek.
Phase 1
3-band EQ with kill switches
Low shelf (250Hz), mid peak (1kHz), high shelf (8kHz) — each with ±12dB range. Kill switches ramp to −40dB. All transitions are click-free using Web Audio time-ramp smoothing.
Phase 1
Pitch-corrected tempo nudge
±8% range via native Web Audio playbackRate. SoundTouchJS Web Worker included for full master-tempo pitch correction (phase 2 activation).
Phase 1
Hot cue jump (8 slots)
Cue markers displayed on the waveform. Click a marker or press keys 1–8 to jump instantly. Cue positions stored in the local database and synced to Traktor NML.
Phase 1
VU meter & gain control
Real-time level display from the Web Audio analyser node. Master gain control with smooth ramping.
Phase 1

DJ software integration

Traktor NML import
Parse Traktor's collection.nml — import BPM, key, cue points, loops, playlists, ratings, play counts, and colour tags into the central database.
Phase 1
Traktor path patching
After a sync moves files to a new location, automatically rewrite the file paths in collection.nml. Fixes Traktor's missing track problem without manual relocation.
Phase 1

Phase 2 — DJ Tools & Intelligence

Next

Analysis depth, AI integration, and the tools that turn the library into an active part of your preparation workflow.

AI integration (Claude API)

AI ◈ cataloging
Smart cataloging

Suggest genres, moods, energy levels, and custom tags based on metadata and audio analysis results. Flags inconsistencies — different artist spellings, missing album art, incomplete tagging.

AI ◈ playlists
Playlist generation

Natural language prompts: "2 hour warm-up set, 120–128 BPM, melodic, Camelot key-compatible" — Claude queries your library and builds a sequenced playlist with transition logic.

AI ◈ QC
Library quality control

Identify duplicate tracks with different filenames, flag unanalysed tracks before a gig, surface tracks with BPM values that look wrong relative to genre, suggest metadata corrections.

AI ◈ notes
Track notes & context

Generate or enrich track descriptions, label history, artist bios, and release context. Useful for building set notes or sharing tracklists with context attached.

Extended analysis

Musical key detection (keyfinder-cli)
Camelot wheel notation and standard notation. Stored with source flag. Used for key-compatible playlist sequencing.
Phase 2
Loudness analysis (LUFS)
Integrated loudness measurement for consistent level matching across a set. Flag tracks that are significantly louder or quieter than your library average.
Phase 2
Track structure detection
Auto-detect intro length, outro start, and first beat offset from waveform energy profile. Tag tracks with structural markers (e.g. "16 bar intro") to aid set planning.
Phase 2

DJ tools

Two-deck preview
Load two tracks simultaneously with independent transport, EQ, and tempo. Visual crossfader. Useful for pre-listening transitions before a set.
Phase 2
Pre-gig checklist
Verify that your USB drive contains every track in your planned setlist. Flag missing, unanalysed, or recently modified tracks before you leave the house.
Phase 2
Set timeline planner
Visual timeline of a planned set — waveforms laid end to end, energy curve, BPM flow, Camelot key transitions. Plan the arc of a set visually before committing.
Phase 2
MIDI controller mapping
Web MIDI API — map physical controller knobs and buttons to EQ bands, tempo nudge, cue jump, transport. Profile-based so different controllers can be saved and loaded.
Phase 2
Beatport download watcher
Auto-detect new Beatport downloads, classify into -non-album with today's date, trigger immediate analysis, and sync to all nodes.
Phase 2
Tag taxonomy
User-defined tag categories — vibe, occasion, energy, era, source. Many-to-many, distinct from genre. Fully queryable for playlist building and AI prompts.
Phase 2
Rekordbox XML import / export
Read and write Rekordbox collection XML — cue points, playlists, ratings. Allows bidirectional sync with Rekordbox without using Pioneer's cloud service.
Phase 2
Serato metadata read
Parse Serato's hidden _Serato_ folders stored alongside audio files. Import cue points, loops, and BPM values into the central database.
Phase 2

Phase 3 — Platform & Ecosystem

Future

Turning the platform outward — mobile access, advanced ML, ecosystem connections, and multi-user support.

Stem separation (Demucs agent job)
ML-based isolation of kick, bass, melody, and vocals. Runs as a Python agent job on the NAS. Stems stored alongside the original, never replacing it.
Phase 3
Transition compatibility scoring
Compare waveform energy at the outro of one track against the intro of another. Surface a compatibility score and power "what plays well after this?" recommendations.
Phase 3
Mobile companion (web UI over VPN)
Responsive web UI served by the central API — browse library, check sync status, approve jobs, view set plans from your phone without installing anything.
Phase 3
Release watchlist
Track new releases from followed artists on Beatport and Bandcamp. Flag when something you have been watching becomes available to buy.
Phase 3
Analytics dashboard
Play counts, gig history, most-used tracks, BPM and key distribution across your library, genre breakdown over time.
Phase 3
Multi-user / team library
Shared collection with per-user permissions. Useful for agencies or management companies handling multiple artists who share a pool of tracks.
Phase 3
Webhook events
Fire events on scan complete, sync complete, analysis done, new track added. Connect to Zapier, Make, or custom automations.
Phase 3
Public tracklist & set pages
Optional — shareable links for tracks or recorded sets, hosted on samuelmarks.com. For sharing set information with promoters, press, or fans.
Phase 3

Compatibility

Works alongside your existing tools

offtherecord reads and writes standard formats used by every major DJ platform. It is not a replacement for any of them — it is a layer that connects them and keeps your data consistent across all of them.

Traktor Pro
Read + Write
Full NML import and export. BPM, key, cue points, loops, playlists, ratings, play counts, colour tags. Path patching after sync fixes missing track errors automatically.
Rekordbox
Read + Write
XML collection import and export. Cue points, playlists, ratings. Phase 2 feature — no Pioneer cloud account required.
Serato DJ
Read only
Reads hidden _Serato_ folders stored alongside audio files. Imports cue points, loops, and BPM into the central database.
Beatport
Watcher
Monitors your downloads folder. Auto-classifies new purchases into -non-album with today's date and triggers immediate analysis.
iTunes / Music.app
Read only
Scans iTunes XML library for metadata. Useful for initial import if your collection is organised in iTunes.
Any MP3 / FLAC / AIFF
Read + Write
Reads all standard ID3, ID3v2, and APEv2 tags. Can write BPM, key, and custom tags back to files when instructed — always with explicit confirmation.

Architecture

How the pieces connect

Every component talks to the central API over your always-on VPN. USB drives have no agent — the Electron app on whichever machine they are plugged into acts as their proxy.

[ Mac / Windows / Linux ] Electron desktop app Local SQLite → REST API (Bearer token) File watcher (chokidar) USB proxy — syncs queue on connect │ │ HTTPS over VPN ▼ [ Proxmox VM — Portainer ] ┌─────────────────────────────────┐ │ PHP API (Slim 4) │ │ MariaDB 11 │ │ Node.js agent ←──────────┐ │ └────────────────────────────│───┘ │ NFS mount ▼ [ Synology NAS ] /volume1/music ← your actual files /volume1/waveforms ← generated .dat files /volume1/artwork ← deduplicated artwork

Contributing

Where help is needed

The core scaffold is built. The following areas are where contributors can pick up clearly scoped work without needing to understand the full codebase.

PHP / API
  • Traktor NML parser service
  • Rekordbox XML read/write
  • Waveform upload endpoint
  • Analysis result validation
  • API rate limiting + logging
JavaScript / Electron
  • Two-deck preview UI
  • MIDI controller mapping
  • Set timeline planner
  • Pre-gig checklist view
  • Keyboard shortcut layer
Node.js / Agent
  • keyfinder-cli integration
  • LUFS loudness analysis
  • Track structure detection
  • Beatport download watcher
  • Serato metadata parser
AI / Claude integration
  • Playlist generation prompt design
  • Cataloging suggestion pipeline
  • Library QC rule definitions
  • Natural language search layer
  • Track note generation
DevOps / Infrastructure
  • GitHub Actions CI pipeline
  • Electron build and signing
  • Portainer stack versioning
  • Backup strategy for MariaDB
  • Health monitoring setup
Design / UI
  • Two-deck layout design
  • Mobile companion UI
  • Onboarding / setup flow
  • Dark theme refinement
  • Icon set and branding assets

Key decisions already made

To avoid relitigating architecture in early PRs — these are settled:

SQLite locally, MariaDB centrally
Local SQLite for the Electron app — fast, portable, no service. MariaDB on Portainer for the authoritative central record. These sync via the REST API, not direct DB connection.
Content-addressed storage for waveforms and artwork
Waveforms and artwork are keyed by SHA-256 hash of the source audio. If you own the same track in multiple places, one waveform is generated and referenced everywhere.
Non-destructive by default
The tool never writes to your audio files unless you explicitly trigger a tag-write operation with confirmation. Analysis results are stored in the database, not embedded in files, unless requested.