blomios/IdeA
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(memory): système de mémoire projet model-agnostic (L14, LOT A+B+C)

Browse Source 
Base de connaissance persistante par projet, indépendante de tout modèle/CLU
et de git. Cadrage archi en §14.5 (ARCHITECTURE.md), cycle Archi→Dev→Test.

LOT A — étage 1 (.md, source de vérité)
  - domaine: entité Memory (+ MemorySlug, MemoryType, MemoryFrontmatter,
    MemoryLink, MemoryIndexEntry), liens [[slug]], index MEMORY.md dérivé
  - port MemoryStore + MemoryError, adapter FsMemoryStore (.ideai/memory/)
  - application: 7 use cases (Create/Update/List/Get/Delete/ReadIndex/
    ResolveLinks), From<MemoryError> for AppError
  - app-tauri: commandes + DTO, events MemorySaved/MemoryDeleted
  - suppression de la variante morte DomainError::MalformedFrontmatter

LOT B — rappel adaptatif (étage 1)
  - port MemoryRecall + MemoryQuery, adapter NaiveMemoryRecall (troncature
    au budget de tokens, court-circuit budget-0), use case RecallMemory

LOT C — étage 2 vectoriel (structure complète, zéro dépendance lourde)
  - port Embedder + EmbedderError, profils déclaratifs EmbedderProfile/
    EmbedderStrategy (embedder.json)
  - VectorMemoryRecall (cosinus, cache .ideai/memory/.index/ gitignoré)
  - AdaptiveMemoryRecall (bascule pure should_use_vector), défaut none
  - HashEmbedder (déterministe, tests), StubEmbedder (onnx/server/api)

Tests: 57 binaires verts, build + clippy --workspace sans warning.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Blomios
2026-06-08 08:47:23 +02:00
parent 3ed0f6b45f
commit 98a8b7292a
30 changed files with 4978 additions and 14 deletions
Download Patch File Download Diff File Expand all files Collapse all files

5
crates/infrastructure/src/lib.rs
View File

@ -40,5 +40,8 @@ pub use pty::PortablePtyAdapter;
pub use remote::{remote_host, LocalHost};
pub use runtime::CliAgentRuntime;
pub use store::{
FsProfileStore, FsProjectStore, FsSkillStore, FsTemplateStore, IdeaiContextStore,
embedder_from_profile, index_token_size, should_use_vector, AdaptiveMemoryRecall,
FsEmbedderProfileStore, FsMemoryStore, FsProfileStore, FsProjectStore, FsSkillStore,
FsTemplateStore, HashEmbedder, IdeaiContextStore, NaiveMemoryRecall, StubEmbedder,
VectorMemoryRecall,
};

173
crates/infrastructure/src/store/embedder.rs Normal file
View File

@ -0,0 +1,173 @@
//! [`Embedder`] adapters (LOT C, étage 2 vectoriel — §14.5.3).
//!
//! The étage-2 semantic recall is driven by a declarative
//! [`EmbedderProfile`]; this module turns such a profile into a concrete
//! [`Embedder`] via [`embedder_from_profile`].
//!
//! ## Zero heavy dependency by design
//!
//! IdeA's founding posture is **default `none`, nothing imposed, zero
//! dependency** (ARCHITECTURE §14.5.3). The concrete engines therefore ship as
//! **documented stubs**:
//!
//! - [`EmbedderStrategy::LocalOnnx`] / [`EmbedderStrategy::LocalServer`] /
//! [`EmbedderStrategy::Api`] map to [`StubEmbedder`], which returns a clean
//! [`EmbedderError::Unsupported`] (never a panic) — the real ONNX/HTTP
//! integration is an explicit follow-up that would add the heavy crate behind a
//! feature, here and only here;
//! - [`EmbedderStrategy::None`] yields no embedder (`None`): recall stays the
//! dependency-free naïve étage 1.
//!
//! [`HashEmbedder`] is a small **deterministic, dependency-free** embedder used to
//! exercise the whole vector pipeline (it is not a stub: it produces stable,
//! repeatable vectors from a hashing bag-of-words, good enough for tests and for a
//! trivial offline fallback).
use async_trait::async_trait;
use domain::ports::{Embedder, EmbedderError};
use domain::profile::{EmbedderProfile, EmbedderStrategy};
/// Builds the concrete [`Embedder`] for a profile, or `None` when the strategy is
/// [`EmbedderStrategy::None`] (recall stays naïve).
///
/// The concrete engines are stubs ([`StubEmbedder`]) until their real ONNX/HTTP
/// integration lands; they fail cleanly with [`EmbedderError::Unsupported`] rather
/// than panic, so a composing recall degrades to naïve.
#[must_use]
pub fn embedder_from_profile(profile: &EmbedderProfile) -> Option<Box<dyn Embedder>> {
match profile.strategy {
EmbedderStrategy::None => None,
EmbedderStrategy::LocalOnnx => Some(Box::new(StubEmbedder::new(
profile.id.clone(),
profile.dimension,
"localOnnx",
))),
EmbedderStrategy::LocalServer => Some(Box::new(StubEmbedder::new(
profile.id.clone(),
profile.dimension,
"localServer",
))),
EmbedderStrategy::Api => Some(Box::new(StubEmbedder::new(
profile.id.clone(),
profile.dimension,
"api",
))),
}
}
/// A placeholder [`Embedder`] for a not-yet-implemented concrete strategy.
///
/// Every [`embed`](Embedder::embed) call returns [`EmbedderError::Unsupported`]
/// (never a panic), naming the strategy. It exists so the wiring, profiles, and
/// adapters are all in place and testable today; swapping in the real engine is a
/// localised follow-up.
pub struct StubEmbedder {
id: String,
dimension: usize,
strategy: &'static str,
}
impl StubEmbedder {
/// Builds a stub for `strategy` advertising `dimension`.
#[must_use]
pub fn new(id: impl Into<String>, dimension: usize, strategy: &'static str) -> Self {
Self {
id: id.into(),
dimension,
strategy,
}
}
}
#[async_trait]
impl Embedder for StubEmbedder {
fn id(&self) -> &str {
&self.id
}
async fn embed(&self, _texts: &[String]) -> Result<Vec<Vec<f32>>, EmbedderError> {
Err(EmbedderError::Unsupported(format!(
"embedder strategy `{}` is not implemented yet (follow-up: real ONNX/HTTP backend)",
self.strategy
)))
}
fn dimension(&self) -> usize {
self.dimension
}
}
/// A deterministic, dependency-free [`Embedder`] backed by a hashing
/// bag-of-words. Pure std (FNV-1a hashing of whitespace tokens into `dimension`
/// buckets, then L2-normalised). Stable across runs and platforms.
///
/// It is **not** a stub: it lets the étage-2 pipeline run end-to-end with no heavy
/// dependency. Vector quality is crude (lexical overlap only), but the
/// [`Embedder`] contract is fully honoured, so it is a valid offline embedder and
/// the substrate for deterministic tests of [`crate::store::VectorMemoryRecall`].
#[derive(Clone)]
pub struct HashEmbedder {
id: String,
dimension: usize,
}
impl HashEmbedder {
/// Builds a hashing embedder producing `dimension`-length vectors.
///
/// # Panics
/// Panics if `dimension` is `0` (an embedder must produce fixed-length
/// non-empty vectors).
#[must_use]
pub fn new(id: impl Into<String>, dimension: usize) -> Self {
assert!(dimension > 0, "HashEmbedder dimension must be non-zero");
Self {
id: id.into(),
dimension,
}
}
/// Embeds one text into an L2-normalised `dimension`-length vector.
fn embed_one(&self, text: &str) -> Vec<f32> {
let mut v = vec![0f32; self.dimension];
for token in text.split_whitespace() {
let lower = token.to_ascii_lowercase();
let bucket = (fnv1a(lower.as_bytes()) as usize) % self.dimension;
v[bucket] += 1.0;
}
let norm = v.iter().map(|x| x * x).sum::<f32>().sqrt();
if norm > 0.0 {
for x in &mut v {
*x /= norm;
}
}
v
}
}
#[async_trait]
impl Embedder for HashEmbedder {
fn id(&self) -> &str {
&self.id
}
async fn embed(&self, texts: &[String]) -> Result<Vec<Vec<f32>>, EmbedderError> {
Ok(texts.iter().map(|t| self.embed_one(t)).collect())
}
fn dimension(&self) -> usize {
self.dimension
}
}
/// FNV-1a 64-bit hash of a byte slice (deterministic, dependency-free).
fn fnv1a(bytes: &[u8]) -> u64 {
const OFFSET: u64 = 0xcbf2_9ce4_8422_2325;
const PRIME: u64 = 0x0000_0100_0000_01b3;
let mut hash = OFFSET;
for &b in bytes {
hash ^= u64::from(b);
hash = hash.wrapping_mul(PRIME);
}
hash
}

492
crates/infrastructure/src/store/memory.rs Normal file
View File

@ -0,0 +1,492 @@
//! [`FsMemoryStore`] — file implementation of the [`MemoryStore`] port
//! (LOT A, étage 1).
//!
//! Memory notes are the project's persistent, model-agnostic knowledge base. Each
//! note is a single Markdown file with a YAML frontmatter header, stored under the
//! project's `.ideai/memory/`:
//!
//! ```text
//! <project_root>/.ideai/memory/
//! ├── MEMORY.md # aggregated index: one `- [Title](slug.md) — hook` line per note
//! └── <slug>.md # a note: YAML frontmatter + Markdown body
//! ```
//!
//! A note file looks like:
//!
//! ```text
//! ---
//! name: my-note
//! description: A one-line hook
//! metadata:
//! type: project
//! ---
//! # Body
//! ...
//! ```
//!
//! The `.md` files are the **single source of truth**; `MEMORY.md` is derived and
//! kept in sync on every [`save`](MemoryStore::save)/[`delete`](MemoryStore::delete)
//! (idempotent upsert / removal of the note's line). All I/O goes through the
//! [`FileSystem`] port, so the adapter is location-neutral (SSH/WSL work unchanged)
//! and Tauri-agnostic.
//!
//! Like the sibling stores, [`delete`](MemoryStore::delete) drops the note's line
//! from `MEMORY.md` and leaves the orphaned `<slug>.md` on disk (the [`FileSystem`]
//! port exposes no remove); since listing is index-driven, the note is effectively
//! gone.
use std::sync::Arc;
use async_trait::async_trait;
use domain::markdown::MarkdownDoc;
use domain::memory::{
Memory, MemoryFrontmatter, MemoryIndexEntry, MemoryLink, MemorySlug, MemoryType,
};
use domain::ports::{
FileSystem, FsError, MemoryError, MemoryQuery, MemoryRecall, MemoryStore, RemotePath,
};
use domain::project::ProjectPath;
/// The `.ideai/` directory name inside a project root.
const IDEAI_DIR: &str = ".ideai";
/// Sub-path of the memory store inside `.ideai/`.
const MEMORY_DIR: &str = "memory";
/// Aggregated index file name inside the memory dir.
const INDEX_FILE: &str = "MEMORY.md";
/// First line of the aggregated index.
const INDEX_HEADER: &str = "# Memory Index";
/// File-backed [`MemoryStore`], composing a [`FileSystem`] port. The project root
/// is supplied **per call**, so a single instance serves every open project
/// (mirroring [`crate::store::FsSkillStore`]).
#[derive(Clone)]
pub struct FsMemoryStore {
fs: Arc<dyn FileSystem>,
}
impl FsMemoryStore {
/// Builds the store from an injected [`FileSystem`]. Directories are created
/// on first write.
#[must_use]
pub fn new(fs: Arc<dyn FileSystem>) -> Self {
Self { fs }
}
/// `<root>/.ideai/memory`.
fn dir(&self, root: &ProjectPath) -> String {
let base = root.as_str().trim_end_matches(['/', '\\']);
format!("{base}/{IDEAI_DIR}/{MEMORY_DIR}")
}
/// `<memory-dir>/<slug>.md`.
fn md_path(&self, root: &ProjectPath, slug: &MemorySlug) -> RemotePath {
RemotePath::new(format!("{}/{}.md", self.dir(root), slug.as_str()))
}
/// `<memory-dir>/MEMORY.md`.
fn index_path(&self, root: &ProjectPath) -> RemotePath {
RemotePath::new(format!("{}/{INDEX_FILE}", self.dir(root)))
}
/// Reads and parses a note by slug.
async fn load(&self, root: &ProjectPath, slug: &MemorySlug) -> Result<Memory, MemoryError> {
let bytes = self
.fs
.read(&self.md_path(root, slug))
.await
.map_err(|e| match e {
FsError::NotFound(_) => MemoryError::NotFound,
other => MemoryError::Io(other.to_string()),
})?;
let text = String::from_utf8(bytes).map_err(|e| MemoryError::Io(e.to_string()))?;
parse_note(&text)
}
/// Reads the raw `MEMORY.md` text, or `None` if it does not exist yet.
async fn read_index_text(&self, root: &ProjectPath) -> Result<Option<String>, MemoryError> {
match self.fs.read(&self.index_path(root)).await {
Ok(bytes) => String::from_utf8(bytes)
.map(Some)
.map_err(|e| MemoryError::Io(e.to_string())),
Err(FsError::NotFound(_)) => Ok(None),
Err(e) => Err(MemoryError::Io(e.to_string())),
}
}
/// Rewrites `MEMORY.md` from the given entries, ensuring the dir exists.
async fn write_index(
&self,
root: &ProjectPath,
entries: &[MemoryIndexEntry],
) -> Result<(), MemoryError> {
self.fs
.create_dir_all(&RemotePath::new(self.dir(root)))
.await
.map_err(|e| MemoryError::Io(e.to_string()))?;
let text = render_index(entries);
self.fs
.write(&self.index_path(root), text.as_bytes())
.await
.map_err(|e| MemoryError::Io(e.to_string()))
}
/// Lists the slugs known to the index (index-driven listing).
async fn index_slugs(&self, root: &ProjectPath) -> Result<Vec<MemorySlug>, MemoryError> {
Ok(self
.read_index(root)
.await?
.into_iter()
.map(|e| e.slug)
.collect())
}
}
#[async_trait]
impl MemoryStore for FsMemoryStore {
async fn list(&self, root: &ProjectPath) -> Result<Vec<Memory>, MemoryError> {
let slugs = self.index_slugs(root).await?;
let mut out = Vec::with_capacity(slugs.len());
for slug in &slugs {
out.push(self.load(root, slug).await?);
}
Ok(out)
}
async fn get(&self, root: &ProjectPath, slug: &MemorySlug) -> Result<Memory, MemoryError> {
self.load(root, slug).await
}
async fn save(&self, root: &ProjectPath, memory: &Memory) -> Result<(), MemoryError> {
// (1) Write the note file.
self.fs
.create_dir_all(&RemotePath::new(self.dir(root)))
.await
.map_err(|e| MemoryError::Io(e.to_string()))?;
let text = render_note(memory);
self.fs
.write(&self.md_path(root, memory.slug()), text.as_bytes())
.await
.map_err(|e| MemoryError::Io(e.to_string()))?;
// (2) Upsert the index line idempotently (same slug => one line).
let mut entries = self.read_index(root).await?;
let row = memory.index_entry();
if let Some(slot) = entries.iter_mut().find(|e| e.slug == row.slug) {
*slot = row;
} else {
entries.push(row);
}
self.write_index(root, &entries).await
}
async fn delete(&self, root: &ProjectPath, slug: &MemorySlug) -> Result<(), MemoryError> {
let mut entries = self.read_index(root).await?;
let before = entries.len();
entries.retain(|e| &e.slug != slug);
if entries.len() == before {
return Err(MemoryError::NotFound);
}
// The orphaned `<slug>.md` is left on disk (no FileSystem delete); the
// index no longer references it, so it is effectively gone.
self.write_index(root, &entries).await
}
async fn read_index(&self, root: &ProjectPath) -> Result<Vec<MemoryIndexEntry>, MemoryError> {
match self.read_index_text(root).await? {
Some(text) => Ok(parse_index(&text)),
None => Ok(Vec::new()),
}
}
async fn resolve_links(
&self,
root: &ProjectPath,
slug: &MemorySlug,
) -> Result<Vec<MemoryLink>, MemoryError> {
let memory = self.load(root, slug).await?;
let known = self.index_slugs(root).await?;
// Keep only links whose target resolves to a known note (ignore broken).
Ok(memory
.outgoing_links()
.into_iter()
.filter(|link| known.contains(&link.target))
.collect())
}
}
// ---------------------------------------------------------------------------
// NaiveMemoryRecall — the default, dependency-free MemoryRecall (LOT B).
// ---------------------------------------------------------------------------
/// Heuristic divisor turning a character count into an approximate token count
/// (~4 characters per token, the usual rule of thumb). Shared by every
/// [`MemoryRecall`] adapter so the budget semantics stay identical (DRY).
pub(crate) const CHARS_PER_TOKEN: usize = 4;
/// Approximate token cost of an index entry's textual payload — the single
/// budget-cost function shared by every [`MemoryRecall`] adapter (naïve, vector,
/// adaptive), so truncation semantics are identical across them (Liskov / DRY).
pub(crate) fn entry_cost(entry: &MemoryIndexEntry) -> usize {
let chars = entry.title.len() + entry.hook.len();
chars.div_ceil(CHARS_PER_TOKEN)
}
/// Greedily takes entries in the given order while their accumulated
/// [`entry_cost`] stays within `budget`; stops at the first entry that would
/// exceed it (and drops every entry after). A `budget` of `0` yields an empty
/// vec. This is the shared truncation used by every recall adapter.
pub(crate) fn truncate_to_budget(
entries: impl IntoIterator<Item = MemoryIndexEntry>,
budget: usize,
) -> Vec<MemoryIndexEntry> {
if budget == 0 {
return Vec::new();
}
let mut spent = 0usize;
let mut out = Vec::new();
for entry in entries {
let cost = entry_cost(&entry);
if spent + cost > budget {
break;
}
spent += cost;
out.push(entry);
}
out
}
/// Total approximate token cost of an index — a **pure** function of the entries,
/// used by [`crate::store::AdaptiveMemoryRecall`] to decide naïve vs. vector
/// recall without any I/O.
#[must_use]
pub fn index_token_size(entries: &[MemoryIndexEntry]) -> usize {
entries.iter().map(entry_cost).sum()
}
/// The default [`MemoryRecall`]: dependency-free, ignores semantic relevance.
///
/// It composes an [`Arc<dyn MemoryStore>`], reads the aggregated index via
/// [`MemoryStore::read_index`], and returns the entries **in index order**,
/// truncated to fit the query's token budget. It is the baseline against which a
/// future `VectorMemoryRecall` (LOT C) is substitutable.
///
/// ## Budget semantics
/// `token_budget` is an *approximate* budget. Each entry's cost is estimated as
/// `ceil((title.len() + hook.len()) / 4)` tokens (≈ 4 chars/token, counting only
/// the index line's textual payload). Entries are taken in order, accumulating
/// their cost; the first entry whose inclusion would exceed the budget — and every
/// entry after it — is dropped. A budget of `0` therefore yields an empty list;
/// an empty or missing memory yields an empty list without error.
#[derive(Clone)]
pub struct NaiveMemoryRecall {
store: Arc<dyn MemoryStore>,
}
impl NaiveMemoryRecall {
/// Builds the recall adapter from a composed [`MemoryStore`].
#[must_use]
pub fn new(store: Arc<dyn MemoryStore>) -> Self {
Self { store }
}
}
#[async_trait]
impl MemoryRecall for NaiveMemoryRecall {
async fn recall(
&self,
root: &ProjectPath,
query: &MemoryQuery,
) -> Result<Vec<MemoryIndexEntry>, MemoryError> {
// Budget-0 short-circuits before any I/O: a zero budget can hold no entry,
// so there is nothing to read (homogeneous with every recall adapter).
if query.token_budget == 0 {
return Ok(Vec::new());
}
let entries = self.store.read_index(root).await?;
Ok(truncate_to_budget(entries, query.token_budget))
}
}
// ---------------------------------------------------------------------------
// On-disk format: YAML frontmatter + body, and the MEMORY.md index.
//
// We hand-roll a tiny, well-scoped YAML reader/writer for exactly the frontmatter
// shape we own (`name`, `description`, `metadata.type`). This keeps the crate free
// of a YAML dependency for a fixed, simple schema; any deviation surfaces as a
// `MemoryError::Frontmatter`.
// ---------------------------------------------------------------------------
/// Renders a note to its on-disk `---`-fenced frontmatter + body form.
fn render_note(memory: &Memory) -> String {
let fm = &memory.frontmatter;
format!(
"---\nname: {}\ndescription: {}\nmetadata:\n type: {}\n---\n{}",
fm.name.as_str(),
fm.description,
type_to_str(fm.r#type),
memory.body.as_str(),
)
}
/// Parses a note from its on-disk form.
fn parse_note(text: &str) -> Result<Memory, MemoryError> {
let rest = text
.strip_prefix("---\n")
.or_else(|| text.strip_prefix("---\r\n"))
.ok_or_else(|| MemoryError::Frontmatter("missing opening `---` fence".to_string()))?;
// Find the closing `---` fence at the start of a line.
let (fm_block, body) = split_frontmatter(rest)
.ok_or_else(|| MemoryError::Frontmatter("missing closing `---` fence".to_string()))?;
let frontmatter = parse_frontmatter(fm_block)?;
Memory::new(frontmatter, MarkdownDoc::new(body))
.map_err(|e| MemoryError::Frontmatter(e.to_string()))
}
/// Splits the post-opening-fence text into `(frontmatter_block, body)` at the
/// closing `---` line. Returns `None` if no closing fence is present.
fn split_frontmatter(rest: &str) -> Option<(&str, &str)> {
let mut offset = 0;
for line in rest.split_inclusive('\n') {
let trimmed = line.trim_end_matches(['\n', '\r']);
if trimmed == "---" {
let fm = &rest[..offset];
let body = &rest[offset + line.len()..];
return Some((fm, body));
}
offset += line.len();
}
None
}
/// Parses the frontmatter key/values into a validated [`MemoryFrontmatter`].
fn parse_frontmatter(block: &str) -> Result<MemoryFrontmatter, MemoryError> {
let err = |reason: &str| MemoryError::Frontmatter(reason.to_string());
let mut name: Option<String> = None;
let mut description: Option<String> = None;
let mut type_str: Option<String> = None;
let mut in_metadata = false;
for raw in block.lines() {
if raw.trim().is_empty() {
continue;
}
let indented = raw.starts_with(' ') || raw.starts_with('\t');
let (key, value) = raw
.split_once(':')
.ok_or_else(|| err("frontmatter line missing `:`"))?;
let key = key.trim();
let value = value.trim();
if !indented {
in_metadata = false;
match key {
"name" => name = Some(value.to_string()),
"description" => description = Some(value.to_string()),
"metadata" => {
in_metadata = true;
if !value.is_empty() {
return Err(err("`metadata` must be a nested block"));
}
}
_ => return Err(err("unknown frontmatter key")),
}
} else if in_metadata && key == "type" {
type_str = Some(value.to_string());
} else {
return Err(err("unexpected indented frontmatter line"));
}
}
let name = name.ok_or_else(|| err("missing `name`"))?;
let description = description.ok_or_else(|| err("missing `description`"))?;
let type_str = type_str.ok_or_else(|| err("missing `metadata.type`"))?;
let name = MemorySlug::new(name).map_err(|e| MemoryError::Frontmatter(e.to_string()))?;
let r#type = str_to_type(&type_str).ok_or_else(|| err("unknown `metadata.type` value"))?;
Ok(MemoryFrontmatter {
name,
description,
r#type,
})
}
/// Renders the aggregated `MEMORY.md` index.
fn render_index(entries: &[MemoryIndexEntry]) -> String {
let mut out = String::from(INDEX_HEADER);
out.push('\n');
if !entries.is_empty() {
out.push('\n');
for e in entries {
out.push_str(&format!(
"- [{}]({}.md) — {}\n",
e.title,
e.slug.as_str(),
e.hook
));
}
}
out
}
/// Parses the `MEMORY.md` index lines back into structured entries. Lines that do
/// not match the `- [Title](slug.md) — hook` shape are skipped (tolerant read);
/// the `type` is not stored in the index line and defaults to
/// [`MemoryType::Reference`].
fn parse_index(text: &str) -> Vec<MemoryIndexEntry> {
let mut out = Vec::new();
for line in text.lines() {
let line = line.trim();
if !line.starts_with("- [") {
continue;
}
if let Some(entry) = parse_index_line(line) {
out.push(entry);
}
}
out
}
/// Parses one `- [Title](slug.md) — hook` line.
fn parse_index_line(line: &str) -> Option<MemoryIndexEntry> {
let rest = line.strip_prefix("- [")?;
let (title, rest) = rest.split_once("](")?;
let (target, rest) = rest.split_once(')')?;
let slug_str = target.strip_suffix(".md").unwrap_or(target);
let slug = MemorySlug::new(slug_str).ok()?;
let hook = rest.trim_start().strip_prefix('—').unwrap_or(rest).trim();
Some(MemoryIndexEntry {
slug,
title: title.to_string(),
hook: hook.to_string(),
r#type: MemoryType::Reference,
})
}
/// Maps a [`MemoryType`] to its YAML/string form (matches serde camelCase).
fn type_to_str(t: MemoryType) -> &'static str {
match t {
MemoryType::User => "user",
MemoryType::Feedback => "feedback",
MemoryType::Project => "project",
MemoryType::Reference => "reference",
}
}
/// Parses a [`MemoryType`] from its string form.
fn str_to_type(s: &str) -> Option<MemoryType> {
match s {
"user" => Some(MemoryType::User),
"feedback" => Some(MemoryType::Feedback),
"project" => Some(MemoryType::Project),
"reference" => Some(MemoryType::Reference),
_ => None,
}
}

8
crates/infrastructure/src/store/mod.rs
View File

@ -5,13 +5,19 @@
//! JSON files in the app data directory (machine-local, outside any project).
mod context;
mod embedder;
mod memory;
mod profile;
mod project;
mod skill;
mod template;
mod vector;
pub use context::IdeaiContextStore;
pub use profile::FsProfileStore;
pub use embedder::{embedder_from_profile, HashEmbedder, StubEmbedder};
pub use memory::{index_token_size, FsMemoryStore, NaiveMemoryRecall};
pub use profile::{FsEmbedderProfileStore, FsProfileStore};
pub use project::FsProjectStore;
pub use skill::FsSkillStore;
pub use template::FsTemplateStore;
pub use vector::{should_use_vector, AdaptiveMemoryRecall, VectorMemoryRecall};

125
crates/infrastructure/src/store/profile.rs
View File

@ -24,7 +24,7 @@ use serde::{Deserialize, Serialize};
use domain::ids::ProfileId;
use domain::ports::{FileSystem, ProfileStore, RemotePath, StoreError};
use domain::profile::AgentProfile;
use domain::profile::{AgentProfile, EmbedderProfile};
/// File name of the profiles store inside the app-data dir.
const PROFILES_FILE: &str = "profiles.json";
@ -147,3 +147,126 @@ impl ProfileStore for FsProfileStore {
self.write_doc(&doc).await
}
}
// ---------------------------------------------------------------------------
// FsEmbedderProfileStore — declarative embedder profiles (`embedder.json`, LOT C).
// ---------------------------------------------------------------------------
/// File name of the embedder-profiles store inside the app-data dir.
const EMBEDDER_FILE: &str = "embedder.json";
/// Current schema version of the embedder-profiles file.
const EMBEDDER_VERSION: u32 = 1;
/// On-disk shape of `embedder.json` (LOT C, §14.5.3), mirroring [`ProfilesDoc`].
#[derive(Debug, Clone, Serialize, Deserialize)]
#[serde(rename_all = "camelCase")]
struct EmbedderDoc {
/// Schema version.
version: u32,
/// All configured embedder profiles.
profiles: Vec<EmbedderProfile>,
}
impl Default for EmbedderDoc {
fn default() -> Self {
Self {
version: EMBEDDER_VERSION,
profiles: Vec::new(),
}
}
}
/// JSON-file store for declarative [`EmbedderProfile`]s (LOT C, étage 2 vectoriel),
/// calqué on [`FsProfileStore`]: `embedder.json` in the global IDE app-data dir,
/// mirroring `profiles.json`.
///
/// No domain `EmbedderProfileStore` port exists yet — embedder profiles are pure
/// configuration loaded at the composition root, so this is a concrete loader. A
/// dedicated port (parallel to [`ProfileStore`]) is an easy follow-up the day the
/// UI needs CRUD over embedder profiles.
///
/// Cheap to clone (everything behind `Arc`).
#[derive(Clone)]
pub struct FsEmbedderProfileStore {
fs: Arc<dyn FileSystem>,
app_data_dir: String,
}
impl FsEmbedderProfileStore {
/// Builds the store from an injected [`FileSystem`] and the app-data dir.
#[must_use]
pub fn new(fs: Arc<dyn FileSystem>, app_data_dir: impl Into<String>) -> Self {
Self {
fs,
app_data_dir: app_data_dir.into(),
}
}
/// `<app-data-dir>/embedder.json`.
fn path(&self) -> RemotePath {
let base = self.app_data_dir.trim_end_matches(['/', '\\']);
RemotePath::new(format!("{base}/{EMBEDDER_FILE}"))
}
async fn read_doc(&self) -> Result<EmbedderDoc, StoreError> {
match self.fs.read(&self.path()).await {
Ok(bytes) => {
serde_json::from_slice(&bytes).map_err(|e| StoreError::Serialization(e.to_string()))
}
Err(domain::ports::FsError::NotFound(_)) => Ok(EmbedderDoc::default()),
Err(e) => Err(StoreError::Io(e.to_string())),
}
}
async fn write_doc(&self, doc: &EmbedderDoc) -> Result<(), StoreError> {
let dir = RemotePath::new(self.app_data_dir.trim_end_matches(['/', '\\']).to_owned());
self.fs
.create_dir_all(&dir)
.await
.map_err(|e| StoreError::Io(e.to_string()))?;
let bytes =
serde_json::to_vec_pretty(doc).map_err(|e| StoreError::Serialization(e.to_string()))?;
self.fs
.write(&self.path(), &bytes)
.await
.map_err(|e| StoreError::Io(e.to_string()))
}
/// Lists all configured embedder profiles (empty when `embedder.json` is
/// absent — the default `none` posture).
///
/// # Errors
/// [`StoreError`] on an I/O or deserialisation failure.
pub async fn list(&self) -> Result<Vec<EmbedderProfile>, StoreError> {
Ok(self.read_doc().await?.profiles)
}
/// Saves (creates or replaces by id) an embedder profile.
///
/// # Errors
/// [`StoreError`] on failure.
pub async fn save(&self, profile: &EmbedderProfile) -> Result<(), StoreError> {
let mut doc = self.read_doc().await?;
if let Some(slot) = doc.profiles.iter_mut().find(|p| p.id == profile.id) {
*slot = profile.clone();
} else {
doc.profiles.push(profile.clone());
}
self.write_doc(&doc).await
}
/// Deletes an embedder profile by id.
///
/// # Errors
/// [`StoreError::NotFound`] if no profile carries that id.
pub async fn delete(&self, id: &str) -> Result<(), StoreError> {
let mut doc = self.read_doc().await?;
let before = doc.profiles.len();
doc.profiles.retain(|p| p.id != id);
if doc.profiles.len() == before {
return Err(StoreError::NotFound);
}
self.write_doc(&doc).await
}
}

347
crates/infrastructure/src/store/vector.rs Normal file
View File

@ -0,0 +1,347 @@
//! Étage-2 semantic recall (LOT C, §14.5.3): [`VectorMemoryRecall`] and the
//! [`AdaptiveMemoryRecall`] switch.
//!
//! [`VectorMemoryRecall`] composes an [`Embedder`] + a [`MemoryStore`] + a small
//! **derived vector store** under `.ideai/memory/.index/`, ranks the memory index
//! entries by cosine similarity to the query, and truncates to the token budget —
//! the **same** budget/emptiness semantics as [`super::NaiveMemoryRecall`]
//! (Liskov), only the relevance strategy differs.
//!
//! [`AdaptiveMemoryRecall`] composes both and routes between them by an
//! **objective, pure, I/O-free** decision ([`should_use_vector`]): tiny memories
//! or a `none` strategy go to the naïve recall; otherwise to the vector recall,
//! with an automatic **fallback to naïve** whenever the embedder is unavailable.
//! No path of this module ever fails hard on a missing/unavailable embedder.
//!
//! ## Derived vector store format
//!
//! `.ideai/memory/.index/vectors.json` holds, per slug, the embedding of that
//! note's index line (title + hook), tagged with the producing embedder id and
//! its dimension:
//!
//! ```json
//! {
//! "version": 1,
//! "embedderId": "hash-embedder",
//! "dimension": 64,
//! "vectors": { "my-note": [0.1, 0.0, ...] }
//! }
//! ```
//!
//! It is **derived data, fully reconstructible** from the `.md` source of truth,
//! and is **gitignored** (`.ideai/memory/.index/`). A stale or absent file is not
//! an error: missing vectors are recomputed on demand and the file is refreshed
//! best-effort. A change of embedder id/dimension invalidates the whole file.
use std::collections::HashMap;
use std::sync::Arc;
use async_trait::async_trait;
use serde::{Deserialize, Serialize};
use domain::memory::MemoryIndexEntry;
use domain::ports::{
Embedder, EmbedderError, FileSystem, FsError, MemoryError, MemoryQuery, MemoryRecall,
MemoryStore, RemotePath,
};
use domain::profile::EmbedderStrategy;
use domain::project::ProjectPath;
use super::memory::{index_token_size, truncate_to_budget};
/// `.ideai/` directory name (mirrors [`super::memory`]).
const IDEAI_DIR: &str = ".ideai";
/// Memory sub-dir.
const MEMORY_DIR: &str = "memory";
/// Derived vector-store sub-dir (gitignored).
const INDEX_DIR: &str = ".index";
/// Derived vector-store file.
const VECTORS_FILE: &str = "vectors.json";
/// Schema version of the derived vector store.
const VECTORS_VERSION: u32 = 1;
/// On-disk shape of `.ideai/memory/.index/vectors.json`.
#[derive(Debug, Clone, Serialize, Deserialize)]
#[serde(rename_all = "camelCase")]
struct VectorDoc {
/// Schema version.
version: u32,
/// Id of the embedder that produced these vectors (a mismatch invalidates).
embedder_id: String,
/// Vector length (a mismatch invalidates).
dimension: usize,
/// Per-slug embedding of the note's index line.
vectors: HashMap<String, Vec<f32>>,
}
impl VectorDoc {
fn new(embedder_id: String, dimension: usize) -> Self {
Self {
version: VECTORS_VERSION,
embedder_id,
dimension,
vectors: HashMap::new(),
}
}
/// Whether this doc was produced by the given embedder (id + dimension).
fn matches(&self, embedder_id: &str, dimension: usize) -> bool {
self.version == VECTORS_VERSION
&& self.embedder_id == embedder_id
&& self.dimension == dimension
}
}
/// Semantic [`MemoryRecall`] (étage 2): ranks the memory index by cosine
/// similarity of each note's index line to the query text, truncated to the token
/// budget (identical budget semantics to [`super::NaiveMemoryRecall`]).
///
/// Composes an [`Embedder`], a [`MemoryStore`] (the index source of truth), and a
/// [`FileSystem`] for the derived vector cache. **Best-effort**: any
/// [`EmbedderError`] surfaces as [`EmbedderError`] to the caller
/// ([`AdaptiveMemoryRecall`] turns it into a naïve fallback); a missing/empty
/// memory yields an empty list, never an error.
#[derive(Clone)]
pub struct VectorMemoryRecall {
embedder: Arc<dyn Embedder>,
store: Arc<dyn MemoryStore>,
fs: Arc<dyn FileSystem>,
}
impl VectorMemoryRecall {
/// Builds the vector recall from its composed ports.
#[must_use]
pub fn new(
embedder: Arc<dyn Embedder>,
store: Arc<dyn MemoryStore>,
fs: Arc<dyn FileSystem>,
) -> Self {
Self {
embedder,
store,
fs,
}
}
/// `<root>/.ideai/memory/.index`.
fn index_dir(root: &ProjectPath) -> String {
let base = root.as_str().trim_end_matches(['/', '\\']);
format!("{base}/{IDEAI_DIR}/{MEMORY_DIR}/{INDEX_DIR}")
}
/// `<index-dir>/vectors.json`.
fn vectors_path(root: &ProjectPath) -> RemotePath {
RemotePath::new(format!("{}/{VECTORS_FILE}", Self::index_dir(root)))
}
/// The text embedded for an index entry (its index-line payload).
fn entry_text(entry: &MemoryIndexEntry) -> String {
format!("{} {}", entry.title, entry.hook)
}
/// Loads the derived vector doc, or a fresh empty one when absent, malformed,
/// or produced by a different embedder/dimension (best-effort: never errors on
/// a stale cache — it is simply rebuilt).
async fn load_doc(&self, root: &ProjectPath) -> VectorDoc {
let empty = || VectorDoc::new(self.embedder.id().to_owned(), self.embedder.dimension());
match self.fs.read(&Self::vectors_path(root)).await {
Ok(bytes) => match serde_json::from_slice::<VectorDoc>(&bytes) {
Ok(doc) if doc.matches(self.embedder.id(), self.embedder.dimension()) => doc,
_ => empty(),
},
Err(FsError::NotFound(_)) | Err(_) => empty(),
}
}
/// Persists the derived vector doc best-effort (a write failure is swallowed:
/// the cache is reconstructible, recall already has its result).
async fn save_doc(&self, root: &ProjectPath, doc: &VectorDoc) {
let dir = RemotePath::new(Self::index_dir(root));
if self.fs.create_dir_all(&dir).await.is_err() {
return;
}
if let Ok(bytes) = serde_json::to_vec_pretty(doc) {
let _ = self.fs.write(&Self::vectors_path(root), &bytes).await;
}
}
}
#[async_trait]
impl MemoryRecall for VectorMemoryRecall {
async fn recall(
&self,
root: &ProjectPath,
query: &MemoryQuery,
) -> Result<Vec<MemoryIndexEntry>, MemoryError> {
if query.token_budget == 0 {
return Ok(Vec::new());
}
let entries = self.store.read_index(root).await?;
if entries.is_empty() {
return Ok(Vec::new());
}
// Load the derived cache and compute any missing note vectors.
let mut doc = self.load_doc(root).await;
let missing: Vec<&MemoryIndexEntry> = entries
.iter()
.filter(|e| !doc.vectors.contains_key(e.slug.as_str()))
.collect();
if !missing.is_empty() {
let texts: Vec<String> = missing.iter().map(|e| Self::entry_text(e)).collect();
let vectors = self.recall_embed(&texts).await?;
for (entry, vector) in missing.iter().zip(vectors) {
doc.vectors.insert(entry.slug.as_str().to_owned(), vector);
}
// Drop vectors of notes that no longer exist, then persist best-effort.
let live: std::collections::HashSet<&str> =
entries.iter().map(|e| e.slug.as_str()).collect();
doc.vectors.retain(|slug, _| live.contains(slug.as_str()));
self.save_doc(root, &doc).await;
}
// Embed the query and rank by cosine similarity (descending).
let query_vec = self
.recall_embed(std::slice::from_ref(&query.text))
.await?
.into_iter()
.next()
.unwrap_or_default();
let mut ranked: Vec<(f32, MemoryIndexEntry)> = entries
.into_iter()
.map(|entry| {
let score = doc
.vectors
.get(entry.slug.as_str())
.map_or(0.0, |v| cosine_similarity(&query_vec, v));
(score, entry)
})
.collect();
// Stable, deterministic order: score desc, then slug asc for ties.
ranked.sort_by(|a, b| {
b.0.partial_cmp(&a.0)
.unwrap_or(std::cmp::Ordering::Equal)
.then_with(|| a.1.slug.cmp(&b.1.slug))
});
let ordered = ranked.into_iter().map(|(_, e)| e);
Ok(truncate_to_budget(ordered, query.token_budget))
}
}
impl VectorMemoryRecall {
/// Embeds `texts`, mapping an [`EmbedderError`] into a [`MemoryError`] only so
/// the `?` plumbing compiles; in practice [`AdaptiveMemoryRecall`] guards this
/// path and falls back to naïve before any such error reaches a use case.
async fn recall_embed(&self, texts: &[String]) -> Result<Vec<Vec<f32>>, MemoryError> {
self.embedder
.embed(texts)
.await
.map_err(map_embedder_error)
}
}
/// Maps an [`EmbedderError`] to a [`MemoryError`] for the (guarded) `?` path.
fn map_embedder_error(e: EmbedderError) -> MemoryError {
match e {
EmbedderError::Io(m) => MemoryError::Io(m),
other => MemoryError::Serialization(other.to_string()),
}
}
/// Cosine similarity of two equal-length vectors; `0.0` when a length mismatches
/// or either vector is zero (defensive — never panics).
fn cosine_similarity(a: &[f32], b: &[f32]) -> f32 {
if a.len() != b.len() {
return 0.0;
}
let dot: f32 = a.iter().zip(b).map(|(x, y)| x * y).sum();
let na: f32 = a.iter().map(|x| x * x).sum::<f32>().sqrt();
let nb: f32 = b.iter().map(|x| x * x).sum::<f32>().sqrt();
if na == 0.0 || nb == 0.0 {
0.0
} else {
dot / (na * nb)
}
}
// ---------------------------------------------------------------------------
// AdaptiveMemoryRecall — the étage-1/étage-2 switch.
// ---------------------------------------------------------------------------
/// **Pure, I/O-free** routing decision for [`AdaptiveMemoryRecall`].
///
/// Returns `true` (use the vector recall) **iff** the strategy is not
/// [`EmbedderStrategy::None`] **and** the memory is larger than the budget
/// (i.e. there is something to rank/prune semantically). Otherwise the naïve
/// recall is sufficient and cheaper. This is the single objective rule, unit
/// testable without any store.
#[must_use]
pub fn should_use_vector(memory_size: usize, budget: usize, strategy: EmbedderStrategy) -> bool {
if strategy == EmbedderStrategy::None {
return false;
}
memory_size > budget
}
/// Adaptive [`MemoryRecall`]: routes between a naïve étage-1 recall and a vector
/// étage-2 recall by [`should_use_vector`], with an automatic **fallback to
/// naïve** whenever the vector path fails (embedder unavailable/unsupported).
///
/// Substitutable for either composed recall (Liskov): same emptiness/budget
/// guarantees, and — by construction — it **never fails hard** on an embedder
/// problem. With strategy [`EmbedderStrategy::None`] it is behaviourally identical
/// to the naïve recall (the default product posture, zero dependency).
#[derive(Clone)]
pub struct AdaptiveMemoryRecall {
naive: Arc<dyn MemoryRecall>,
vector: Arc<dyn MemoryRecall>,
store: Arc<dyn MemoryStore>,
strategy: EmbedderStrategy,
}
impl AdaptiveMemoryRecall {
/// Builds the switch from the two recalls, the index store (for the pure size
/// measure), and the active embedder strategy.
#[must_use]
pub fn new(
naive: Arc<dyn MemoryRecall>,
vector: Arc<dyn MemoryRecall>,
store: Arc<dyn MemoryStore>,
strategy: EmbedderStrategy,
) -> Self {
Self {
naive,
vector,
store,
strategy,
}
}
}
#[async_trait]
impl MemoryRecall for AdaptiveMemoryRecall {
async fn recall(
&self,
root: &ProjectPath,
query: &MemoryQuery,
) -> Result<Vec<MemoryIndexEntry>, MemoryError> {
// Budget-0 short-circuits before any I/O, homogeneously with the naïve and
// vector recalls (a zero budget can hold no entry: nothing to read/measure).
if query.token_budget == 0 {
return Ok(Vec::new());
}
// Pure measure from the index (the only I/O is reading the index once;
// the decision itself is pure via `should_use_vector`).
let memory_size = index_token_size(&self.store.read_index(root).await?);
if !should_use_vector(memory_size, query.token_budget, self.strategy) {
return self.naive.recall(root, query).await;
}
// Vector path with naïve fallback — never fail hard on the embedder.
match self.vector.recall(root, query).await {
Ok(entries) => Ok(entries),
Err(_) => self.naive.recall(root, query).await,
}
}
}