Skinki
A local-first memory engine exploring how retrieval, provenance and long-term memory can make AI systems more capable.
01 / QUESTION
What problem was framed
Current frontier models rely either on bloated remote context windows (which degrade in retrieval needle recall and leak private data) or naïve vector chunking (which breaks multi-hop joins across conversations separated by weeks).
The core question: Can an intelligent, deterministic memory substrate handle indexing, entity consolidation, multi-hop joins, and causal grounding locally on consumer hardware—allowing a small ~4B parameter model to behave with the long-term context awareness of an enterprise assistant?
02 / CONSTRAINTS
Guiding technical constraints
HARDWARE BUDGET
M1 Air Baseline (~4B model)
Prefill speed on consumer silicon is expensive. The memory substrate must perform the joins and assembly so the small model only verbalizes.
MEMORY FOOTPRINT
<250 MB RAM for 5M Vectors
Index and graph structures must be mmap-backed with aggressive RaBitQ 1-bit / Int4 quantization to keep resident set size strictly bounded.
LATENCY & BATTERY
Zero-Cost Realtime Capture
Realtime capture is an instant append to an immutable log. All heavy consolidation (entity extraction, Leiden communities, graph traversal) happens interruptibly during "sleep" (idle on power).
PRIVACY & PROVENANCE
Zero Network Bytes
Strict local operation. Every surfaced fact, entity edge, or insight must point back to exact byte ranges in raw capture logs ("cite or stay silent").
03 / SYSTEM
Layered Architecture (L0 to L5)
Skinki is structured as a headless Rust engine with clear layer responsibilities, isolating fast append paths from background graph consolidation and agent context assembly.
SKINKI ENGINE SUBSYSTEM TOPOLOGY
┌────────────────────────────────────────────────────────────────────────┐
│ L0 CAPTURE: Append-only raw log (Source of Truth & Provenance Bytes) │
└───────────────────────────────────┬────────────────────────────────────┘
│
▼
┌────────────────────────────────────────────────────────────────────────┐
│ L1 UNITS: Atomic thoughts & facts referencing L0 byte slices │
└───────────────────┬────────────────────────────────┬───────────────────┘
│ │
▼ ▼
┌─────────────────────────────────────┐ ┌───────────────────────────────┐
│ L2a VECTOR INDEX │ │ L2b KNOWLEDGE GRAPH │
│ Model2Vec first-pass + RaBitQ quant │ │ Entities, relations, temporal │
└───────────────────┬─────────────────┘ └───────────────┬───────────────┘
│ │
└─────────────────┬──────────────────┘
│ (Deferred to idle on power)
▼
┌────────────────────────────────────────────────────────────────────────┐
│ L3 SLEEP CONSOLIDATION: Incremental extraction (LightRAG style), │
│ Leiden community hierarchies (RAPTOR), HippoRAG 2 / PPR associative │
└─────────────────────────────────────┬──────────────────────────────────┘
│
▼
┌────────────────────────────────────────────────────────────────────────┐
│ L4 INSIGHT ENGINE: Deterministic discovery ──▶ Stat validation (FDR) │
│ (Separating discovery from LLM narration: "Cite or stay silent") │
└─────────────────────────────────────┬──────────────────────────────────┘
│
▼
┌────────────────────────────────────────────────────────────────────────┐
│ L5 AGENT / QUERY / MCP CONTEXT ASSEMBLER │
│ Budgeted, cited facts + pre-joined multi-hop chains for small LLMs │
└────────────────────────────────────────────────────────────────────────┘ RUST CORE / HEADLESS MMAP-BACKED STORAGE
04 / DECISIONS
Architectural Trade-Off Matrix
| Decision | Why | Alternatives Rejected | Trade-Off Incurred |
|---|---|---|---|
| Rust Core Engine | Predictable RAM, `no_std` hot paths, zero garbage collection pauses on background consolidation. | Python (LangChain/LlamaIndex), Swift-only app | Higher upfront development cost and FFI boundary boilerplate for UI consumers. |
| Sleep-Time Consolidation | Keeps text capture instant (<1ms) and prevents battery drain while typing on laptop. | Realtime graph building on every input | New memories require idle time before appearing in deep community summaries. |
| Coarse-to-Fine Search Hierarchy | 46% relative retrieval gain (0.291 → 0.438) by scoping turn search to relevant dialogue instances. | Flat top-k dense search across all turns | Requires clustering step and two-phase scoring pipeline. |
| Deterministic Statistical Validation | Prevents agent hallucinations by subjecting discovered candidate links to false discovery rate (FDR) gates. | Direct LLM reflection / free-form summarization | Conservative output: engine stays silent if evidence confidence threshold is unmet. |
05 / EXPERIMENTS
Benchmarks & Quantitative Signals
Memory Footprint
<250 MB
5,000,000 vectors under RaBitQ 1-bit / Int4 mmap indexing.
Coarse-to-Fine Gain
0.291 → 0.438
Recall@10 improvement over flat EmbeddingGemma 256d baseline.
Synthetic Graph Recall
0.325 → 0.800
Multi-hop recall@10 on deterministic synthetic V2 memory corpus.
06 / FAILURES & NEGATIVE RESULTS
GraphRAG Falsification: When Graph Retrieval Fails to Transfer
CRITICAL RESEARCH SIGNAL Synthetic Win vs Real Dialogue Transfer Failure
On a deterministic synthetic corpus, adding a typed relation graph raised multi-hop recall@10 from 0.325 to 0.800. However, when tested on real human dialogue (pooled LongMemEval multi-session instances spanning 9,697 turns), the graph retriever scored 0.168 recall@10, falling below ordinary BM25 (0.193) and dense retrieval (0.291).
| Retriever | Recall@10 | Answer@10 | nDCG@10 |
|---|---|---|---|
| BM25 (Lexical Baseline) | 0.193 | 0.450 | 0.154 |
| Co-mention graph + BM25 | 0.168 | 0.450 | 0.112 |
| Typed-fact graph + BM25 | 0.168 | 0.450 | 0.109 |
| EmbeddingGemma (256 dims) | 0.291 | 0.450 | 0.187 |
| Coarse-to-Fine Dense Hierarchy | 0.438 | 0.500 | 0.245 |
Why the failure occurred:
- Schema Coupling: The synthetic generator and retriever shared a causal grammar. Real dialogue broke this with speaker references, paraphrases, and aliases.
- Reachability ≠ Ranking: Adding graph edges brought in relevant chains, but also flooded candidate pools with conversational hub nodes, displacing valid lexical hits during rank fusion.
- Extraction Errors: Mistaken entity resolution in a graph creates false topological shortcuts, redistributing relevance across irrelevant passages.
07 / CURRENT STATE
What is working right now
- L0/L1 Storage Substrate: Append-only log + memory unit indexing with zero-copy byte provenance.
- L2 Vector & Graph Index: Mmap-backed storage with Model2Vec and RaBitQ quantization.
- L3 Sleep Scheduler: Interruptible background consolidation gated by idle/power states.
- Eval Harness: Rigorous synthetic corpus generator with deterministic multi-hop ground truth and regression test suite.
- MCP Server Surface: Standard Model Context Protocol interface exposing memory queries to Claude/Cursor/OpenAI tooling.
08 / NEXT
Current R&D focus
Building the L4 Insight Engine with statistical discovery filters (FDR control) and causal skepticism, ensuring that multi-hop patterns surfaced to the user are mathematically grounded before being passed to small LLM verbalizers.
09 / LINKS & ARTIFACTS