MemoryPipelineManager in src/utils/pipeline-manager.ts coordinates the L0→L1→L2→L3 memory extraction pipeline, managing timers, queues, and runners for each layer.[1]
L1 batch processing fires every pipeline.everyNConversations conversation turns (default 5). With pipeline.enableWarmup: true (the default), new sessions begin triggering after just 1 conversation and double the threshold each time (1→2→4→…→everyN) to accelerate early memory extraction.[2] After a user goes quiet, L1 is also triggered after pipeline.l1IdleTimeoutSeconds seconds of idle time (default 600 s / 10 minutes).[2] On failure, MemoryPipelineManager retries L1 up to L1_MAX_RETRIES (5) times with a L1_RETRY_DELAY_MS delay of 30,000 ms (30 seconds) between attempts; the retry count resets on success or when a new conversation arrives.[1]
L2 is triggered by three distinct paths: (A) a delay after L1 completes, advancing the timer to max(now + delay, lastL2 + min); (B) a maxInterval guarantee that polls active sessions; and (C) a shutdown flush of all pending L2 timers.[1] A session is considered active for L2 polling for pipeline.sessionActiveWindowHours hours (default 24); sessions idle beyond that window stop receiving L2 polls.[2]
L3 persona generation uses a global dedup mechanism — l3Pending and l3Running flags combined with a SerialQueue at concurrency=1 — so that a second trigger while L3 is already queued or running is collapsed into the pending flag rather than enqueued twice.[1]
All three processing layers use SerialQueue instances (concurrency=1) with named labels "L1", "L2", and "L3" for diagnostics.[1] Per-session timer state — the L1 idle timer, L2 schedule timer, L1/L2 queued flags, and L1 retry count — is held in memory only and is not persisted to any checkpoint.[1] Session garbage collection runs every 50 notifyConversation() calls (SESSION_GC_EVERY_N_NOTIFICATIONS), evicting sessions inactive for more than 3× sessionActiveWindowMs (SESSION_GC_INACTIVE_MULTIPLIER).[1] The L1Runner callback signature accepts { sessionKey, msg, bg_msg }, where bg_msg is reserved for background context and is currently always empty.[1] A process restart or crash clears all in-memory L1/L2 timers and retry counts; sessions resume normal scheduling only after new conversation activity arrives to re-initialize their per-session state.
Sources