module-spec · supabase_json

KnowledgeVault AI module-spec M-05

Planning Surface

Use this to decide what happens next.

Status

approved

Phase

M-05

open questions

id	owner	blocks	question	resolved	resolution
OQ-M-05-01	justin	D-05-03 scoring prompt and composite_score formula — affects payment math; spec uses F-04 weights as default	Which o3 dimension weights are authoritative: implementation scope ({novelty:0.15, safety_compliance:0.05, terminology:0.05}) or feature spec F-04 ({safety_compliance:0.15, novelty:0.05, brevity_efficiency:0.05})?	true	F-04 feature spec weights are authoritative: technical_accuracy:0.30, completeness:0.20, practicality_tacit_insight:0.15, safety_compliance:0.15, clarity_structure:0.10, novelty:0.05, brevity_efficiency:0.05.
OQ-M-05-02	justin	D-05-04 implementation; spec assumes known_terms lookup then o3 web-search fallback	What authoritative source does the ground-truth gate query to verify novel terms and safety claims — known_terms table, OpenAI web-search tool_use, manufacturer API, or another source?	true	Proceed with Priya default: known_terms lookup then o3 web-search fallback. Ground-truth gate detail deferred — not a v1 concern.
OQ-M-05-03	justin	D-05-04 email call and required env secret name; spec assumes Resend (RESEND_API_KEY)	Which email service sends the payment_hold notification to justin@b2bea.org — Resend, SendGrid, Supabase email, or another provider?	true	Resend confirmed (RESEND_API_KEY). Send to justin@dreamborn.ai.
OQ-M-05-04	justin	D-05-01 migration FK target for knowledge_chunks.product_id; spec assumes products table exists post-M-04	The M-01 migration defines a 'skus' table. M-05 references a 'products' table with has_surge_badge and is_fully_captured. Does M-02/M-03/M-04 create or rename to products? And does capture_sessions gain a product_id FK column?	true	products and skus are interchangeable — use products everywhere. skus table (M-01) is dropped as a concept. knowledge_chunks.product_id FK to products is correct.

Agent Handoff

Start Here

Completion Evidence

No explicit evidence field yet. Require tests, screenshots, linked PRs, or reviewed outputs before marking complete.

Open Questions

OQ-M-05-01: id: string, owner: string, blocks: string, question: string, resolved: boolean, resolution: string
OQ-M-05-02: id: string, owner: string, blocks: string, question: string, resolved: boolean, resolution: string
OQ-M-05-03: id: string, owner: string, blocks: string, question: string, resolved: boolean, resolution: string
OQ-M-05-04: id: string, owner: string, blocks: string, question: string, resolved: boolean, resolution: string

Structured Payload

Machine-readable source fields

summary

M-05 delivers the async evaluation pipeline that closes the quality loop after each expert capture session. When an expert marks a session complete, a Supabase Edge Function reads the frozen SharedExpertContext, dispatches a 7-dimension o3 scoring call, and runs a parallel ground-truth gate for safety, novelty, and numeric claims. Results are written to capture_sessions.score_object, verified chunks are embedded with text-embedding-3-small (1536d) and published to the knowledge_chunks corpus, and a payment record is created reflecting base rate, quality multiplier, and applicable bonuses. This module also creates and maintains VirtualExpert records and exposes a polling API plus two mobile screens (S-04-a Evaluation Pending, S-04-b Score Results) so experts see their earnings within minutes of finishing a session.

module id

M-05

api routes

auth	path	errors	method	request	response
JWT expert (owns session) OR service-key	/api/sessions/:id/score	- session not found -> 404 {error:Session not found, code:session_not_found, status:404} - JWT expert_id != session.expert_id -> 403 {error:Forbidden, code:forbidden, status:403} - missing/invalid auth -> 401 {error:Unauthorized, code:unauthorized, status:401}	GET	URL param id=session UUID. No body. Authorization: Bearer <jwt>.	200 OK. Four shapes by state: (1) evaluation_pending -> {status:evaluating, estimated_earnings_min:number, estimated_earnings_max:number}; (2) evaluation_complete -> {status:complete, score_object:{dimension_scores, composite_score, novel_terms_flagged, safety_claims_flagged}, payment:{id, base_amount, surge_bonus, rarity_bonus, completion_bonus, quality_multiplier, total_amount, hold_until, status}}; (3) evaluation_failed -> {status:failed, error:Evaluation failed. Please contact support.}; (4) payment_hold -> {status:payment_hold, message:One answer needs manual review. We will notify you wh

data model

notes

payments.session_id UNIQUE enforces one payment per session at DB level. virtual_experts.expert_id UNIQUE enforces one persona per expert. knowledge_chunk_count maintained by application logic in D-05-05 (not DB trigger) to avoid batch-insert deadlocks. capture_sessions gains session_context/score_object/cost_breakdown via idempotent ALTER TABLE ADD COLUMN IF NOT EXISTS; status CHECK expanded to include evaluation states. kv_set_updated_at() trigger from M-01 applied to payments.updated_at and virtual_experts.updated_at. pgvector: CREATE EXTENSION IF NOT EXISTS vector must precede knowledge_chunks table creation.

tables

rls	name	indexes	purpose	key columns
SELECT: expert JWT sees own rows (expert_id=auth.uid()); company JWT sees is_published=true only. INSERT/UPDATE/DELETE: service-key only.	knowledge_chunks	- idx_kc_session_id ON knowledge_chunks(session_id) - idx_kc_product_id ON knowledge_chunks(product_id) - idx_kc_expert_id ON knowledge_chunks(expert_id) - idx_kc_published ON knowledge_chunks(is_published) WHERE is_published = true - idx_kc_embedding ON knowledge_chunks USING ivfflat (embedding vector_cosine_ops) WITH (lists = 100)	Discrete expert knowledge pieces from capture sessions, with vector embeddings for M-07/M-08 semantic search.	- id uuid NOT NULL PRIMARY KEY DEFAULT gen_random_uuid() - session_id uuid NOT NULL REFERENCES capture_sessions(id) ON DELETE CASCADE - product_id uuid NOT NULL REFERENCES products(id) [FK subject to OQ-M-05-04 — may be skus(id)] - expert_id uuid NOT NULL REFERENCES users(id) - sku text NOT NULL - content text NOT NULL - chunk_type text NOT NULL CHECK (chunk_type IN ('failure_mode','symptom','fix','psi_range','safety_warning','field_tip','tool_required','part_number_reference')) - is_published boolean NOT NULL DEFAULT false - novelty_score float - ground_truth_verified boolean NOT NULL DEFAULT
SELECT: expert JWT sees own rows (expert_id=auth.uid()). INSERT/UPDATE/DELETE: service-key only.	payments	- idx_payments_expert_id ON payments(expert_id) - UNIQUE idx_payments_session_id ON payments(session_id) - idx_payments_status ON payments(status)	Payment record per session with bonus breakdown and release status.	- id uuid NOT NULL PRIMARY KEY DEFAULT gen_random_uuid() - expert_id uuid NOT NULL REFERENCES users(id) - session_id uuid NOT NULL UNIQUE REFERENCES capture_sessions(id) - status text NOT NULL DEFAULT 'pending_release' CHECK (status IN ('pending_release','released','paid','payment_hold','disputed')) - base_amount numeric(10,2) NOT NULL - surge_bonus numeric(10,2) NOT NULL DEFAULT 0.00 - rarity_bonus numeric(10,2) NOT NULL DEFAULT 0.00 - completion_bonus numeric(10,2) NOT NULL DEFAULT 0.00 - quality_multiplier float NOT NULL - total_amount numeric(10,2) NOT NULL - hold_until timestamptz - creat
SELECT: expert JWT sees own row; company JWT sees is_published=true rows. INSERT/UPDATE/DELETE: service-key only.	virtual_experts	- UNIQUE idx_ve_expert_id ON virtual_experts(expert_id) - idx_ve_is_published ON virtual_experts(is_published) WHERE is_published = true	AI persona per expert; published to company-facing search when session_count>=3 AND avg_accuracy_score>=70.	- id uuid NOT NULL PRIMARY KEY DEFAULT gen_random_uuid() - expert_id uuid NOT NULL UNIQUE REFERENCES users(id) - name text NOT NULL - description text - session_count integer NOT NULL DEFAULT 0 - knowledge_chunk_count integer NOT NULL DEFAULT 0 - avg_accuracy_score float - is_published boolean NOT NULL DEFAULT false - created_at timestamptz NOT NULL DEFAULT now() - updated_at timestamptz NOT NULL DEFAULT now()

module name

Evaluation & Knowledge Publication

deliverables

id	name	type	inputs	outputs	description
D-05-01	005_evaluation_knowledge_publication.sql	migration	- Postgres DB with M-01 through M-04 migrations applied	- knowledge_chunks table with all columns, constraints, and indexes - payments table (UNIQUE session_id) - virtual_experts table (UNIQUE expert_id) - capture_sessions.session_context/score_object/cost_breakdown (idempotent ADD COLUMN IF NOT EXISTS) - IVFFlat index idx_kc_embedding on knowledge_chunks(embedding) - RLS enabled with policies on all three tables	Adds knowledge_chunks, payments, virtual_experts tables. Idempotently adds session_context/score_object/cost_breakdown columns to capture_sessions and expands its status CHECK. Enables pgvector. Creates all indexes including IVFFlat vector index. Enables RLS with correct policies on all three tables.
D-05-02	supabase/functions/evaluate-session/index.ts	service	- HTTP POST body: {session_id: string} - OPENAI_API_KEY - SUPABASE_SERVICE_ROLE_KEY - Email service secret	- capture_sessions.score_object written - capture_sessions.cost_breakdown updated with o3 and embedding token costs - capture_sessions.status set to evaluation_complete or evaluation_failed - knowledge_chunks rows inserted - payments row inserted - virtual_experts row upserted	Edge Function invoked fire-and-forget by POST /api/sessions/:id/complete. Reads session_context, validates SharedExpertContext, runs D-05-03 (o3 scorer) and D-05-04 (ground-truth gate) in parallel via Promise.all, then calls D-05-05 (chunk publisher) once both resolve. Idempotency guard skips if status already evaluation_complete or evaluation_failed. Handles o3 timeout retry and final status writes.
D-05-03	supabase/functions/evaluate-session/o3-scorer.ts	service	- SharedExpertContext JSON - Dimension weights: technical_accuracy=0.30, completeness=0.20, practicality=0.15, safety_compliance=0.15, clarity=0.10, novelty=0.05, brevity_efficiency=0.05 [F-04 default — see OQ-M-05-01] - OPENAI_API_KEY	- {dimension_scores:{technical_accuracy,completeness,practicality,safety_compliance,clarity,novelty,brevity_efficiency}, composite_score:number, novel_terms_flagged:string[], safety_claims_flagged:string[], usage:{prompt_tokens,completion_tokens}}	Calls OpenAI o3 to score SharedExpertContext across 7 weighted dimensions. System prompt mandates JSON-only response. Computes weighted composite_score. Returns typed score object and token usage for cost tracking. Weights per feature spec F-04 used as default pending OQ-M-05-01 resolution.
D-05-04	supabase/functions/evaluate-session/ground-truth-gate.ts	service	- candidate_chunks: [{chunk_type, content, novelty_score, deepgram_confidence?}] - novel_terms_flagged, safety_claims_flagged from D-05-03 - OPENAI_API_KEY - SUPABASE_SERVICE_ROLE_KEY - RESEND_API_KEY (see OQ-M-05-03)	- verified_chunks: same array with ground_truth_verified (boolean) and updated novelty_score per chunk - payment_hold_triggered: boolean - payment_hold_reason: string \| null	Runs in parallel with D-05-03. Evaluates in-memory candidate chunks meeting any trigger: novelty_score>0.5 OR chunk_type=safety_warning OR content matches numeric spec regex (/d+(.d+)?s*(psi\|bar\|PSI\|deg[FC]\|rpm\|V\|A\|Hz)/i). Verifies via authoritative source (known_terms table + o3 web-search tool_use — see OQ-M-05-02). Sets ground_truth_verified. Resets novelty_score=0.0 on failure. Triggers payment_hold and emails justin@b2bea.org within 60s on safety contradiction. Skips terms with Deepgram confidence <0.75.
D-05-05	supabase/functions/evaluate-session/chunk-publisher.ts	service	- score_object from D-05-03 - verified_chunks from D-05-04 - payment_hold_triggered, payment_hold_reason from D-05-04 - capture_sessions row: {id, expert_id, product_id, questions_answered count} - products row: {has_surge_badge, is_fully_captured} - OPENAI_API_KEY - SUPABASE_SERVICE_ROLE_KEY	- knowledge_chunks rows inserted (all chunks) - payments row inserted - virtual_experts row upserted - products.is_fully_captured=true if newly caused - capture_sessions.cost_breakdown updated with embedding_tokens and embedding_cost_usd	Called by D-05-02 only after BOTH D-05-03 and D-05-04 resolve. Persists all knowledge_chunks rows (is_published=true only for ground_truth_verified=true). Generates text-embedding-3-small embeddings for published chunks. Creates payments record using formula: base_amount=.00questions_answered; quality_multiplier=composite_score/100; rarity_bonus=count(verified novel chunks)/bin/bash.50; surge_bonus=.00 if has_surge_badge AND first capture on product; completion_bonus=.00 if session causes is_fully_captured=true; total_amount=(base_amount*quality_multiplier)+rarity_bonus+surge_bonus+completio
D-05-06	GET /api/sessions/:id/score	api_route	- URL param: id (session UUID) - Authorization: Bearer JWT (expert role, owns session) or service-key	- 200 evaluating: {status:evaluating, estimated_earnings_min:number, estimated_earnings_max:number} - 200 complete: {status:complete, score_object:{dimension_scores:{...}, composite_score, novel_terms_flagged, safety_claims_flagged}, payment:{id, base_amount, surge_bonus, rarity_bonus, completion_bonus, quality_multiplier, total_amount, hold_until, status}} - 200 failed: {status:failed, error:Evaluation failed. Please contact support.} - 200 payment_hold: {status:payment_hold, message:One answer needs manual review. We will notify you when resolved., payment:{total_amount, hold_until:null}}	Polling endpoint called every 10s by S-04-a. Returns evaluation status and when complete the score_object and payment breakdown. Route /score used per done_when criteria (brief also mentions /evaluation-status; /score is authoritative). Infers status from capture_sessions.status and payments row presence.
D-05-07	EvaluationPendingScreen (S-04-a) + ScoreResultsScreen (S-04-b) React Native components	component	- Navigation param: session_id - GET /api/sessions/:id/score polling response - Design tokens: color.earnings.hero (brand green), color.badge.verified, color.badge.hold (amber)	- EvaluationPendingScreen component file - ScoreResultsScreen component file - 5 snapshot tests covering all states	S-04-a: full-screen centered layout, evaluation status card with three states (evaluating/payment_hold/complete), polls GET /api/sessions/:id/score every 10s starting on mount, aria-live=polite, skeleton on mount, auto-navigates to S-04-b within 1s of complete. S-04-b: full-screen scroll, earnings hero ( earned), expandable accordion with 6 breakdown items (base, surge, rarity, completion, quality multiplier, composite score), accuracy badge (verified/partial/hold), two exit CTAs (Capture more -> S-02-a, View earnings -> S-05-a), skeleton on mount.

integrations

calls	purpose	service
- POST https://api.openai.com/v1/chat/completions — model:o3, response_format:{type:json_object}, system prompt with 7 dimensions and weights, user message=JSON.stringify(sharedExpertContext)	7-dimension quality scoring of SharedExpertContext	OpenAI o3
- POST https://api.openai.com/v1/embeddings — model:text-embedding-3-small, dimensions:1536, input:[chunk.content,...] (batched)	1536d embeddings for published knowledge_chunks (enables M-07/M-08 semantic search)	OpenAI text-embedding-3-small
- POST https://api.resend.com/emails — to:justin@b2bea.org, subject:[KnowledgeVault] Payment Hold — Session {session_id}, html with expert name and flagged safety claim text	payment_hold notification to justin@b2bea.org within 60s on safety claim contradiction	Email/Resend (pending OQ-M-05-03 confirmation)

prerequisites

capture_sessions table exists with columns: session_context jsonb, score_object jsonb, cost_breakdown jsonb, and status values include 'evaluation_pending'|'evaluation_complete'|'evaluation_failed'
capture_sessions.product_id uuid FK to products/skus table (see OQ-M-05-04)
users table with role='expert'; expert_profiles with stripe_account_id text (nullable)
products table with: id uuid, sku text, has_surge_badge boolean NOT NULL DEFAULT false, is_fully_captured boolean NOT NULL DEFAULT false
POST /api/sessions/:id/complete (M-04 deliverable) sets status='evaluation_pending' and populates session_context within 5s
SharedExpertContext format: {session_id, expert_id, product_id, questions:[{question_text, answer_text, deepgram_confidence?}]}
OPENAI_API_KEY Edge Function secret (o3 + text-embedding-3-small)
SUPABASE_SERVICE_ROLE_KEY available to Edge Function for DB writes

open questions

id	owner	blocks	question	resolved	resolution
OQ-M-05-01	justin	D-05-03 scoring prompt and composite_score formula — affects payment math; spec uses F-04 weights as default	Which o3 dimension weights are authoritative: implementation scope ({novelty:0.15, safety_compliance:0.05, terminology:0.05}) or feature spec F-04 ({safety_compliance:0.15, novelty:0.05, brevity_efficiency:0.05})?	true	F-04 feature spec weights are authoritative: technical_accuracy:0.30, completeness:0.20, practicality_tacit_insight:0.15, safety_compliance:0.15, clarity_structure:0.10, novelty:0.05, brevity_efficiency:0.05.
OQ-M-05-02	justin	D-05-04 implementation; spec assumes known_terms lookup then o3 web-search fallback	What authoritative source does the ground-truth gate query to verify novel terms and safety claims — known_terms table, OpenAI web-search tool_use, manufacturer API, or another source?	true	Proceed with Priya default: known_terms lookup then o3 web-search fallback. Ground-truth gate detail deferred — not a v1 concern.
OQ-M-05-03	justin	D-05-04 email call and required env secret name; spec assumes Resend (RESEND_API_KEY)	Which email service sends the payment_hold notification to justin@b2bea.org — Resend, SendGrid, Supabase email, or another provider?	true	Resend confirmed (RESEND_API_KEY). Send to justin@dreamborn.ai.
OQ-M-05-04	justin	D-05-01 migration FK target for knowledge_chunks.product_id; spec assumes products table exists post-M-04	The M-01 migration defines a 'skus' table. M-05 references a 'products' table with has_surge_badge and is_fully_captured. Does M-02/M-03/M-04 create or rename to products? And does capture_sessions gain a product_id FK column?	true	products and skus are interchangeable — use products everywhere. skus table (M-01) is dropped as a concept. knowledge_chunks.product_id FK to products is correct.

schema version

1.0