feat(memory): système de mémoire projet model-agnostic (L14, LOT A+B+C)
Base de connaissance persistante par projet, indépendante de tout modèle/CLU
et de git. Cadrage archi en §14.5 (ARCHITECTURE.md), cycle Archi→Dev→Test.
LOT A — étage 1 (.md, source de vérité)
- domaine: entité Memory (+ MemorySlug, MemoryType, MemoryFrontmatter,
MemoryLink, MemoryIndexEntry), liens [[slug]], index MEMORY.md dérivé
- port MemoryStore + MemoryError, adapter FsMemoryStore (.ideai/memory/)
- application: 7 use cases (Create/Update/List/Get/Delete/ReadIndex/
ResolveLinks), From<MemoryError> for AppError
- app-tauri: commandes + DTO, events MemorySaved/MemoryDeleted
- suppression de la variante morte DomainError::MalformedFrontmatter
LOT B — rappel adaptatif (étage 1)
- port MemoryRecall + MemoryQuery, adapter NaiveMemoryRecall (troncature
au budget de tokens, court-circuit budget-0), use case RecallMemory
LOT C — étage 2 vectoriel (structure complète, zéro dépendance lourde)
- port Embedder + EmbedderError, profils déclaratifs EmbedderProfile/
EmbedderStrategy (embedder.json)
- VectorMemoryRecall (cosinus, cache .ideai/memory/.index/ gitignoré)
- AdaptiveMemoryRecall (bascule pure should_use_vector), défaut none
- HashEmbedder (déterministe, tests), StubEmbedder (onnx/server/api)
Tests: 57 binaires verts, build + clippy --workspace sans warning.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
173
crates/infrastructure/src/store/embedder.rs
Normal file
173
crates/infrastructure/src/store/embedder.rs
Normal file
@ -0,0 +1,173 @@
|
||||
//! [`Embedder`] adapters (LOT C, étage 2 vectoriel — §14.5.3).
|
||||
//!
|
||||
//! The étage-2 semantic recall is driven by a declarative
|
||||
//! [`EmbedderProfile`]; this module turns such a profile into a concrete
|
||||
//! [`Embedder`] via [`embedder_from_profile`].
|
||||
//!
|
||||
//! ## Zero heavy dependency by design
|
||||
//!
|
||||
//! IdeA's founding posture is **default `none`, nothing imposed, zero
|
||||
//! dependency** (ARCHITECTURE §14.5.3). The concrete engines therefore ship as
|
||||
//! **documented stubs**:
|
||||
//!
|
||||
//! - [`EmbedderStrategy::LocalOnnx`] / [`EmbedderStrategy::LocalServer`] /
|
||||
//! [`EmbedderStrategy::Api`] map to [`StubEmbedder`], which returns a clean
|
||||
//! [`EmbedderError::Unsupported`] (never a panic) — the real ONNX/HTTP
|
||||
//! integration is an explicit follow-up that would add the heavy crate behind a
|
||||
//! feature, here and only here;
|
||||
//! - [`EmbedderStrategy::None`] yields no embedder (`None`): recall stays the
|
||||
//! dependency-free naïve étage 1.
|
||||
//!
|
||||
//! [`HashEmbedder`] is a small **deterministic, dependency-free** embedder used to
|
||||
//! exercise the whole vector pipeline (it is not a stub: it produces stable,
|
||||
//! repeatable vectors from a hashing bag-of-words, good enough for tests and for a
|
||||
//! trivial offline fallback).
|
||||
|
||||
use async_trait::async_trait;
|
||||
|
||||
use domain::ports::{Embedder, EmbedderError};
|
||||
use domain::profile::{EmbedderProfile, EmbedderStrategy};
|
||||
|
||||
/// Builds the concrete [`Embedder`] for a profile, or `None` when the strategy is
|
||||
/// [`EmbedderStrategy::None`] (recall stays naïve).
|
||||
///
|
||||
/// The concrete engines are stubs ([`StubEmbedder`]) until their real ONNX/HTTP
|
||||
/// integration lands; they fail cleanly with [`EmbedderError::Unsupported`] rather
|
||||
/// than panic, so a composing recall degrades to naïve.
|
||||
#[must_use]
|
||||
pub fn embedder_from_profile(profile: &EmbedderProfile) -> Option<Box<dyn Embedder>> {
|
||||
match profile.strategy {
|
||||
EmbedderStrategy::None => None,
|
||||
EmbedderStrategy::LocalOnnx => Some(Box::new(StubEmbedder::new(
|
||||
profile.id.clone(),
|
||||
profile.dimension,
|
||||
"localOnnx",
|
||||
))),
|
||||
EmbedderStrategy::LocalServer => Some(Box::new(StubEmbedder::new(
|
||||
profile.id.clone(),
|
||||
profile.dimension,
|
||||
"localServer",
|
||||
))),
|
||||
EmbedderStrategy::Api => Some(Box::new(StubEmbedder::new(
|
||||
profile.id.clone(),
|
||||
profile.dimension,
|
||||
"api",
|
||||
))),
|
||||
}
|
||||
}
|
||||
|
||||
/// A placeholder [`Embedder`] for a not-yet-implemented concrete strategy.
|
||||
///
|
||||
/// Every [`embed`](Embedder::embed) call returns [`EmbedderError::Unsupported`]
|
||||
/// (never a panic), naming the strategy. It exists so the wiring, profiles, and
|
||||
/// adapters are all in place and testable today; swapping in the real engine is a
|
||||
/// localised follow-up.
|
||||
pub struct StubEmbedder {
|
||||
id: String,
|
||||
dimension: usize,
|
||||
strategy: &'static str,
|
||||
}
|
||||
|
||||
impl StubEmbedder {
|
||||
/// Builds a stub for `strategy` advertising `dimension`.
|
||||
#[must_use]
|
||||
pub fn new(id: impl Into<String>, dimension: usize, strategy: &'static str) -> Self {
|
||||
Self {
|
||||
id: id.into(),
|
||||
dimension,
|
||||
strategy,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[async_trait]
|
||||
impl Embedder for StubEmbedder {
|
||||
fn id(&self) -> &str {
|
||||
&self.id
|
||||
}
|
||||
|
||||
async fn embed(&self, _texts: &[String]) -> Result<Vec<Vec<f32>>, EmbedderError> {
|
||||
Err(EmbedderError::Unsupported(format!(
|
||||
"embedder strategy `{}` is not implemented yet (follow-up: real ONNX/HTTP backend)",
|
||||
self.strategy
|
||||
)))
|
||||
}
|
||||
|
||||
fn dimension(&self) -> usize {
|
||||
self.dimension
|
||||
}
|
||||
}
|
||||
|
||||
/// A deterministic, dependency-free [`Embedder`] backed by a hashing
|
||||
/// bag-of-words. Pure std (FNV-1a hashing of whitespace tokens into `dimension`
|
||||
/// buckets, then L2-normalised). Stable across runs and platforms.
|
||||
///
|
||||
/// It is **not** a stub: it lets the étage-2 pipeline run end-to-end with no heavy
|
||||
/// dependency. Vector quality is crude (lexical overlap only), but the
|
||||
/// [`Embedder`] contract is fully honoured, so it is a valid offline embedder and
|
||||
/// the substrate for deterministic tests of [`crate::store::VectorMemoryRecall`].
|
||||
#[derive(Clone)]
|
||||
pub struct HashEmbedder {
|
||||
id: String,
|
||||
dimension: usize,
|
||||
}
|
||||
|
||||
impl HashEmbedder {
|
||||
/// Builds a hashing embedder producing `dimension`-length vectors.
|
||||
///
|
||||
/// # Panics
|
||||
/// Panics if `dimension` is `0` (an embedder must produce fixed-length
|
||||
/// non-empty vectors).
|
||||
#[must_use]
|
||||
pub fn new(id: impl Into<String>, dimension: usize) -> Self {
|
||||
assert!(dimension > 0, "HashEmbedder dimension must be non-zero");
|
||||
Self {
|
||||
id: id.into(),
|
||||
dimension,
|
||||
}
|
||||
}
|
||||
|
||||
/// Embeds one text into an L2-normalised `dimension`-length vector.
|
||||
fn embed_one(&self, text: &str) -> Vec<f32> {
|
||||
let mut v = vec![0f32; self.dimension];
|
||||
for token in text.split_whitespace() {
|
||||
let lower = token.to_ascii_lowercase();
|
||||
let bucket = (fnv1a(lower.as_bytes()) as usize) % self.dimension;
|
||||
v[bucket] += 1.0;
|
||||
}
|
||||
let norm = v.iter().map(|x| x * x).sum::<f32>().sqrt();
|
||||
if norm > 0.0 {
|
||||
for x in &mut v {
|
||||
*x /= norm;
|
||||
}
|
||||
}
|
||||
v
|
||||
}
|
||||
}
|
||||
|
||||
#[async_trait]
|
||||
impl Embedder for HashEmbedder {
|
||||
fn id(&self) -> &str {
|
||||
&self.id
|
||||
}
|
||||
|
||||
async fn embed(&self, texts: &[String]) -> Result<Vec<Vec<f32>>, EmbedderError> {
|
||||
Ok(texts.iter().map(|t| self.embed_one(t)).collect())
|
||||
}
|
||||
|
||||
fn dimension(&self) -> usize {
|
||||
self.dimension
|
||||
}
|
||||
}
|
||||
|
||||
/// FNV-1a 64-bit hash of a byte slice (deterministic, dependency-free).
|
||||
fn fnv1a(bytes: &[u8]) -> u64 {
|
||||
const OFFSET: u64 = 0xcbf2_9ce4_8422_2325;
|
||||
const PRIME: u64 = 0x0000_0100_0000_01b3;
|
||||
let mut hash = OFFSET;
|
||||
for &b in bytes {
|
||||
hash ^= u64::from(b);
|
||||
hash = hash.wrapping_mul(PRIME);
|
||||
}
|
||||
hash
|
||||
}
|
||||
Reference in New Issue
Block a user