AI Gold Trading Intelligence Platform

Exit criteria

• Scope is documented.README.md, DECISIONS.md
• Live trading is explicitly out of scope for early phases.README, .env.example LIVE_TRADING_ENABLED=false
• Unknowns are visible and assigned for decision.DECISIONS.md; tracker Open decisions

Deliverables

• Product requirements summarydocs/PRODUCT_REQUIREMENTS.md
• Glossarydocs/PRODUCT_REQUIREMENTS.md → Glossary (30 terms)
• Assumptions and unresolved decisions listDECISIONS.md 'Remaining integration decisions'
• Phase checklist and acceptance templatedocs/PHASE_ACCEPTANCE.md generated from roadmap.json

Work

• Inspect the repository and environment before changing anything.
• Create README.md, PROJECT_STATUS.md, DECISIONS.md, CHANGELOG.md, and docs/.
• Record product scope, non-goals, user roles, initial XAUUSD-only scope, and research-first release plan.
• Record the initial choice of historical market data + historical news.
• Record browser dashboard, Windows development PC, dedicated VPS, MT5-first/MT4-later, and Telegram public/private channels.
• List unresolved provider, broker, account, and hosting questions; do not invent answers.
• Define terms consistently.

Exit criteria

• Minimal services start locally.node scripts/workspace.mjs start
• Health endpoints work./api/health, /health
• Automated checks run in CI.GitHub Actions run 40287596449: format, typecheck, setup, ruff, pytest, roadmap diff, build, Playwright all success
• No broker credentials or live execution are present.grep: no broker adapter code

Deliverables

• Architecture documentdocs/ARCHITECTURE.md
• Repository scaffoldsrc/, backend/, scripts/, tests/, docs/
• Coding standardsPrettier, tsc, ruff (ruff.toml), pytest, Playwright; docs/TEST_PLAN.md
• Threat modeldocs/THREAT_MODEL.md
• Service/API contract outlinedocs/LOCAL_RESEARCH_CONTRACT.md, FastAPI OpenAPI

Work

• Create repository layout for frontend, Node.js API, Python/FastAPI service, workers, tests, migrations, and docs.
• Define service contracts and ownership.
• Define TypeScript/Python formatting, linting, type checking, and test conventions.
• Define configuration management and environment variables.
• Define logging format, error model, ID strategy, UTC convention, and API versioning.
• Decide single vs multiple repositories.
• Create architecture diagram and threat model.

Exit criteria

• Unauthenticated users cannot access protected pages/APIs.security.py RequestGuard; REQUIRE_OWNER_AUTH
• Privileged endpoints enforce server-side authorization.FastAPI dependency, X-Service-Token + owner session
• No credentials are stored in browser local storage.HttpOnly cookie gold_owner_session
• Security tests pass.9 security tests passed 2026-10-26; 180 backend tests passed

Deliverables

• Login UI/login (setup + sign-in)
• Protected browser routesproxy 401/423 redirect to /login when REQUIRE_OWNER_AUTH=true
• Admin panel shell/settings
• Audit log base/logs, audit_events table
• Security testsbackend/tests/test_security.py (9 tests) + operations auth coverage + proxy host/origin Playwright checks
• Work order Part 4 item 27 (roadmap 5.8): Admin & Settings - users, own sessions and sign-out of others, MFA (TOTP) setup, data-provider status, feature flags (live fixed false), backup status, maintenance mode (migration 022; blocks entries, pauses bot worker and shadow runner; ending needs re-authentication), emergency stopsrc/components/admin-panel.tsx, backend/research/monitoring.py (/admin/status, /admin/maintenance), security.py (/auth/sessions)

Work

• Implement secure login/session handling.
• Add admin role and permission checks.
• Add MFA/TOTP design and implementation before production/live use.
• Add CSRF/XSS protections, rate limits, session expiry, and secure cookies.
• Add audit events for login, logout, configuration changes, and rejected privileged actions.
• Build admin shell and system status page.
• Add secure secret references; do not yet connect a live account.

Exit criteria

• Fresh database can be migrated from zero.every test creates a fresh SQLite file
• Migrations are repeatable and tested.backend/tests/test_api.py
• Tests do not depend on production data.tmp_path databases
• Sensitive secrets are not stored as plain text.scrypt hashes; Fernet-encrypted TOTP secrets

Deliverables

• Database schema and migrationsbackend/research/migrations 001-004
• Entity relationship diagramdocs/DATA_DICTIONARY.md (entity map + every table/column)
• Seed/test fixturessynthetic candles/events; tests use tmp databases
• Backup and restore guidedocs/OPERATIONS.md, npm run backup

Work

• Configure the database and migrations.
• Create initial schemas for users, audit events, providers, instruments, jobs, and configuration.
• Define UTC timestamps, indexes, foreign keys, and retention strategy.
• Add connection pooling and safe transaction handling.
• Add backup/restore instructions for development.
• Use synthetic fixtures for tests.

Exit criteria

• Re-running an ingestion does not create duplicates.tests: idempotent incremental, append-only ticks; real incremental runs 2 and 3: 0 duplicate M1 bars, 0 duplicate tick keys
• Missing intervals and bad candles are flagged.real import: 1 weekday gap (2026-09-07 US Labor Day early close), 1 tick price jump, 42 duplicates dropped; weekends/holidays not flagged
• Data coverage and source are visible.Historical Research card + materialized dataset source: provider, server, symbol, bid, UTC convention, offset model, run id, range, quality
• Backtest refuses or warns on unacceptable data quality.materialized datasets carry parse warnings + tick-volume warning; research results stay INCONCLUSIVE

Deliverables

• Historical data adapterbackend/research/dukascopy_csv.py: Dukascopy JForex tick CSV (research history, owner 2026-10-28), streamed, resumable, SHA-256, day tick files; backend/research/mt5_history.py: read-only MT5 broker reference (Exness primary, XM)
• Ingestion workerfull and incremental modes, append-only ticks with hole filling, M1 replace from the first re-fetched bar, lock file, never starts the terminal; hourly task 'AI Gold MT5 History'
• Candle tables/migrationsmarket store: mt5_bars (M1..D1, UTC, bid, spread, tick volume), mt5_ticks, mt5_history_runs (provenance)
• Data quality reportexisting tick audit + M1 checks into quality_events; Historical Research card: coverage per timeframe, ticks, quality flags, last run, Dukascopy inactive
• Provider setup instructionsdocs/DATA_SOURCES.md section 0 (MT5 source, measured import, Max bars note, commands)
• Work order Part 2: Dukascopy/Exness selector in every research view (Dukascopy default; Live Gold and Overview on Exness), data explorer per year (coverage, gaps, quality), synthetic data only in tests/CIbackend/research/history.py (source, explorer), src/components/dataset-select.tsx, market-coverage.tsx; backend/tests/test_work_order_part1.py

Work

• Select a legal/usable data provider only after coverage, cost, granularity, and licensing are reviewed.
• Implement provider adapter and a normalized candle format.
• Ingest available historical OHLCV data for initial timeframes.
• Store source, retrieval time, instrument, timeframe, and coverage metadata.
• Add duplicate, gap, invalid OHLC, outlier, and timestamp checks.
• Add resumable/idempotent ingestion jobs.
• Add data coverage and quality report in the dashboard.
• Keep provider-specific assumptions explicit.

Exit criteria

• Every event has a source and timestamps where available.every event row has source, scheduled_at UTC, source_time, retrieval times and snapshot ids; COT rows carry release_basis + release_source
• Conflicting/missing data is marked, not fabricated.MT5 ↔ FF conflicts stored and flagged (both values kept); MT5 years without events listed as missing; COT weeks without a CFTC-published release time stored as unknown and never used
• Provider limits and errors are handled.ForexFactory: hourly schedule, local guard ≤ 2 downloads / 5 min, denied/HTML/invalid/HTTP/network errors recorded with exponential backoff; CFTC: daily check, errors recorded. Tests in backend/tests/test_news_sources.py
• Daylight-saving transitions are tested.tests: FF offsets across 2026-11-01; New York rules 2006/2007+; MT5 server offset model (US and EU DST transitions, fixed) and event conversion either side of 2026-11-01; COT release times

Deliverables

• News/event schema and migrationsnews_events (CSV) + migration 005 (feed_snapshots, economic_events, cot_reports) + migration 006 (cot release times, cot_release_dates, mt5_exports, mt5_server_offsets, event_name_map, event_links; rows preserved)
• Source adaptersForexFactory weekly JSON + CFTC COT (backend/research/news_sources.py); MT5 calendar: read-only MQL5 service mt5/AIGoldCalendarExport.mq5 (0 errors/0 warnings) + import backend/research/mt5_calendar.py with detected server offset and per-year DST model; MT5 ↔ ForexFactory merge with name-mapping table and conflict flags
• Historical event importer where availablePOST /news/import (CSV); COT history 2006-06-13 → 2026-10-20 (1 063 weeks) imported from the CFTC API
• News calendar APIGET /news, GET /news/calendar, GET /news/cot, GET /news/sources
• Source freshness dashboardAnalytics → Events 'Calendar and COT sources': FF/COT/MT5 freshness and stale warning, COT unknown-release weeks, MT5 broker/server/demo, detected offset + model check, missing years, links/conflicts, both values side by side
• Work order #3 item 2: a stale export never erases a stored actual (cause of release delays of 4 442 s median / 86 282 s max); delays before and after the fix reported separately with our pipeline timebackend/research/mt5_calendar.py; mt5/AIGoldCalendarExport.mq5 default every minute

Work

• Evaluate official/licensed economic-calendar and news sources.
• Implement adapters with rate limits and terms-compliant access.
• Normalize scheduled events, actual/forecast/previous values, source timestamps, and headlines.
• Keep source publication time separate from retrieval time.
• Support UTC and correct US Eastern/local display conversions.
• Deduplicate stories/events without erasing provenance.
• Store source reliability and verification status.
• Add ingestion monitoring and missing-data reports.

Exit criteria

• All main pages load under authenticated access.Playwright: all 16 routes load with the correct heading and active link (desktop + mobile)
• Empty and failure states are handled.panel empty states, service-failure retry test, pending sections without values
• Mock values are never confused with live account or market data.synthetic data labelled; delayed feed labelled; pending sections show no values
• Browser tests pass.passed: 36 passed, 2 device skips (cold dev server, desktop + mobile Edge)

Deliverables

• Working browser dashboard16 routes: the 15 roadmap 5.2 sections + Roadmap; research sections share one workspace; later-phase sections state their phase and show no values
• Navigation and layout componentssrc/lib/sections.ts (roadmap 5.2 order), dashboard-shell.tsx (scrolling sidebar and mobile drawer), src/app/(research)/layout.tsx, section-pending.tsx
• Reusable panel/chart/table componentsterminal-widgets, terminal-topology, home-dashboard charts
• UI test suitetests/dashboard.spec.ts: every route direct entry/reload/active link, history, keyboard, collapsed sidebar, mobile drawer + overflow on all routes, pending sections, shared research workspace
• Work order 2026-10-29 Part 1: sidebar 240/64 px, one-line labels, vertical scroll only; real kill switch, adapter and broker state everywhere; Exness XAUUSDm price with GC=F as reference; dated, newest-first activity; real counters; report loading/details; News layout and synthetic block removedsrc/components/dashboard-shell.tsx, home-dashboard.tsx, reports-center.tsx, logs-dashboard.tsx, research-events.tsx, settings-dashboard.tsx; backend/research/terminal.py, monitoring.py; tests/work-order.spec.ts
• Work order #3 Parts B-D: readable activity, real Overview state and Exness demo block, live interactive charts (1m-1D, zoom/pan/scale, live overlays), neural shell driven by recorded events at the top of the Overviewsrc/components/home-dashboard.tsx, structure-chart.tsx, brain-shell.tsx; backend/research/terminal.py (/live/chart), shell.py (/brain/activity)
• Work order #4 Part A: Overview redesign - organic neural shell (real events, core glows while the LLM thinks), full-width live chart (~60% viewport, timeframe buttons, wheel/drag/double-click, labels drawer), level ladder, live news & speeches (official embeds, countdown, auto switch), wallet | latency row, cone, paper + demo execution tape, status tiles, latest 5 wire eventssrc/components/home-dashboard.tsx, brain-shell.tsx, live-news.tsx, structure-chart.tsx, video-sources.tsx; backend/research/shell.py; migration 028

Work

• Build responsive dark trading-terminal layout.
• Create navigation for Overview, Gold Chart, News, Research, Backtesting, Strategies, Demo, Accounts, Risk, Telegram, Reports, Admin, and Health.
• Add loading, empty, error, stale-data, and disconnected states.
• Add accessible contrast, keyboard navigation, responsive resizing, and clear status labels.
• Build initial overview with actual data only; use explicit mock-data labels for fixtures.
• Add live updates via authenticated WebSocket/SSE if appropriate.

Exit criteria

• Each label is reproducible from documented rules.every emitted kind has a rule text (test asserts it); hand-built fixtures reproduce each label
• Historical signals do not use future candles unavailable at decision time.prefix tests: events known by bar n never change when bars are appended; HTF alignment never ahead of the lower bar's close
• UI shows method/version and timestamp.chart header and label details: method version, run id, computed time, observed and known-at timestamps; detector_runs stores each run
• Edge-case tests pass.ATR warm-up, gaps reset levels, incomplete HTF windows excluded, invalid higher timeframe rejected

Deliverables

• Feature/structure engineadvanced.technical_analysis advanced-0.4.0: EMA12/20/26/50, ATR14, RSI, MACD, volatility, swings, HH/HL/LH/LL, BOS/CHoCH, FVG, S/R, liquidity, order-block hypotheses, session labels, higher-timeframe alignment
• Detector documentationdocs/TECHNICAL_ANALYSIS_RULES.md (advanced-0.4.0 table) + `rules` text returned with every calculation
• Unit testsbackend/tests/test_advanced.py + test_structure_detectors.py (hand-calculated ATR, S/R, liquidity, order blocks, prefix invariance, HTF alignment)
• Chart overlayssrc/components/structure-chart.tsx on Market Structure and Live Gold: overlays, toggles, confirmation-delay connectors, replay, label details

Work

• Implement EMA, ATR, volatility, swing highs/lows, HH/HL/LH/LL, support/resistance, BOS, CHoCH, FVG, order-block hypotheses, liquidity zones, and session features.
• Define exact rules and invalidation logic for each detector.
• Add multi-timeframe alignment without future-data leakage.
• Add unit tests with known synthetic candles and edge cases.
• Store detector version and parameters.
• Add chart overlays and details explaining why each label exists.
• Make repainting/confirmation delays visible.

Exit criteria

• Reports show sample size and data limitations.counts block (selected / outside bar window / incomplete / overlapping / used / baseline) + limitations list incl. small-sample warning under 10
• Overlapping events are flagged.occurrence.overlaps lists every other high-impact release from the pre-window to the longest horizon (same minute included); include-and-flag or exclude; clean-only statistics alongside
• Results are descriptive/statistical, not presented as guaranteed causal effects.first limitation states association only and not a trading rule; surprise never labelled good/bad for gold
• Reproducible event studies are saved.event_studies table (migration 009): parameters, event ids, bar range, SHA-256 fingerprint; test asserts identical fingerprint and occurrences on rerun

Deliverables

• Event-study enginebackend/research/event_studies.py (event-study-1): MT5 calendar releases joined to MT5 M1 bid bars; pre-move, returns per horizon, range, max up/down, volatility; surprise vs MT5 forecast, session, prior 24 h regime; baseline windows. Per-dataset engine.event_study kept for imported journals
• News impact reportPOST /events/studies and /events/cot-studies, GET /events/studies/{id}: counts, overall and grouped statistics (n, mean, median, 95% interval, share up), baseline comparison, analogues, every occurrence, limitations
• Dashboard visualizationsNews & Economic Calendar → Calendar event studies panel: horizon table vs quiet days, surprise/session/regime groups, historical analogues, every release with overlap flags, saved studies list
• Event-study testsbackend/tests/test_event_studies.py (4) + test_engine event_study tests + Playwright panel/endpoint check

Work

• Join event timestamps with quality-checked gold prices.
• Calculate configurable pre/post-event returns, volatility, range, and adverse/favourable movement where granularity permits.
• Compare event outcomes by surprise size, session, and prior market regime where data supports it.
• Track sample size, uncertainty, event overlap, and missing data.
• Compare with baseline windows.
• Create a news-event report and historical analogue view.
• Do not encode a universal rule such as 'strong USD always means gold falls.'

Exit criteria

• Every experiment is reproducible from stored configuration and data references.data_ref must resolve to a stored dataset (sha256 pinned) or a market range; result_ref must exist; backtests keep parameters + dataset sha256
• Approved versions are immutable.rules never updated; new versions are new rows in the lineage with rules_sha256; AI actor refused (403) on APPROVED/LIVE_LIMITED/PAUSED and on approval
• State transitions are validated and audited.roadmap 9.1 states, one gate at a time, evidence gates (backtest, walk-forward), closed gates for demo/shadow/live; strategy_transitions rows + audit events, refusals audited

Deliverables

• Strategy registry API/UIbackend/research/registry.py: /strategies (search by text, state, lineage), /strategies/{id}/versions, /transition, /transitions; Strategies tab with next-state moves, reasons and new versions
• Experiment databaseexperiments table (migration 010): hypothesis, data_ref with pinned dataset sha256 or market range, feature set, configuration, result_ref, one-time outcome; /experiments search + Experiment registry panel
• Research memory recordsevidence_snapshots (hashed, insert-only), research_notes kind note/failure_review with what_failed / suspected_cause / lesson
• State-transition testsbackend/tests/test_registry.py (5) + Playwright registry test
• Work order Part 3 items 20, 21, 26: bot strategies S1 SMC, S2 volume profile (demo priority), S3 session breakout, S4 news pulse (research only) as versioned engine-'bot' registry entries; buy and sell; reward:risk 1:3 editable as a new version; S1/S2 only in London/New York; normal gates; Case B offbackend/research/bot.py, migration 021, docs/TECHNICAL_ANALYSIS_RULES.md (Trading bot strategies), docs/STRATEGY_LIFECYCLE.md

Work

• Create strategy and strategy-version entities.
• Store immutable rule snapshots and configuration.
• Add experiment registry, hypothesis, data range, feature set, and outcome.
• Store decision-time evidence snapshots for each signal.
• Add failure-review records and research notes.
• Create lifecycle states and allowed transitions.
• Add search/filter UI for experiments and versions.
• Prevent an AI model from overwriting an approved strategy version.

Exit criteria

• Re-running a fixed experiment produces consistent results within documented tolerances.deterministic engine; dataset sha256, parameters, cost profile version and code commit stored
• Cost assumptions are visible.per-unit costs, measurement basis (ticks/days, spec capture), slippage multiple and swap in every result and the UI; sensitivity table
• Data leakage tests pass.next-open execution; stop/target from the signal bar's ATR; out-of-sample split by bar index; existing leakage tests
• Inadequate sample size returns INCONCLUSIVE, not PASS.fewer than 200 trades, no profit factor, < 4 walk-forward windows, no demo/shadow records, synthetic data -> INCONCLUSIVE (test_promotion)

Deliverables

• Backtesting serviceengine.run_backtest 0.3.0: next-open fills, ATR stop/target (stop first on a same-bar touch, gap fills at the open), timeout, swap per weekday rollover with triple day, cost sensitivity; POST /backtests
• Report/metrics schemabacktests result: metrics + out_of_sample block + buy-and-hold benchmark + sensitivity + promotion gates + code_version (engine + git commit) + cost_profile_version
• Backtest UIBacktesting Lab: owner cost profile button, exits & financing inputs, PASS/FAIL/INCONCLUSIVE with gates, out-of-sample, benchmark, sensitivity; cost profile & threshold editor
• Regression datasets and teststest_engine (stop/target same bar, gap, timeout, swap nights, out-of-sample, benchmark, sensitivity), test_promotion, test_registry gates
• Work order Part 3 items 23-25: tick-level backtester (bid/ask fills, slippage, swap, SL/TP/breakeven on ticks, pre-news Case C/A, weekend close, 1% lots on the Exness spec, all Risk Center limits), results by year/session/side/news/regime, out-of-sample and walk-forward against the owner thresholds; continuous background worker over 2003-2026; daily preparation before London; live signal path that never waits for a backtestbackend/research/bot.py; scheduled task 'AI Gold Bot Research' (10 min); Backtesting Lab and Strategy Registry 'Trading bot · S1-S4'
• Work order #2 Part A: S1-S4 diagnostics - gross R before costs, net R, costs as % of risk, win rate, average win/loss, MFE/MAE quartiles, 2015-2026 separately, best reward:riskbackend/research/bot.py diagnose()/excursion(); Backtesting Lab -> Trading bot -> Details -> Diagnosis
• Work order #3 item 3: per-setup statistics of ALL setups for S1-S4 and discovered strategies on the Backtesting Lab and Strategy Registry/bot/diagnostics, discovery.rule_diagnostics, src/components/trading-bot.tsx

Work

• Build a deterministic bar-based backtest engine.
• Support entries, exits, SL/TP, partial exits if defined, timeouts, and position state.
• Include spread, commission, slippage, swap/financing and broker symbol rules where data is available.
• Handle bars where SL and TP could both be touched using conservative assumptions or finer data.
• Prevent look-ahead and leakage.
• Store data snapshot, strategy version, configuration, code version, metrics, warnings, and logs.
• Add backtest job queue and resource limits.
• Add benchmark, sensitivity, and walk-forward support.

Exit criteria

• Models are evaluated on unseen periods.walk-forward test blocks after train/validation with embargo; the stored artifact never saw its last block; drift checks only on samples after calibration
• Baseline comparison is documented.rule-based baselines (up-frequency, last-bar persistence) in every block and pooled; docs/RESEARCH_GUIDE.md Phase 12
• No model can directly bypass strategy promotion or risk controls.only app/evaluations/ml read the registry (test); approval changes no strategy (test)
• Models with weak/insufficient evidence remain research-only.approve returns 409 unless evidence PASS; synthetic, fewer windows, non-significant after Bonferroni = not PASS (tests)

Deliverables

• ML experiment frameworkml.evaluate (ml-walk-forward-1) on the durable evaluation queue, kind ml_walk_forward; Strategy Registry panel
• Model registrymigration 013 ml_models + ml_drift_checks; /models register, approve (PASS + owner re-auth), retire, drift
• Evaluation reportsper-block model/baseline metrics, pooled significance, calibration table, regimes, PSI drift; stored with SHA-256 and exportable
• Leakage and reproducibility teststest_ml.py: causal features, chronological splits with leakage guard, identical reports for identical inputs
• Work order Part 3 item 22: per-strategy meta-label filter (TAKE/SKIP) on session, news distance, COT bias, volume, trend strength, volatility, spread; chronological 50/20/30 split, walk-forward, Bonferroni; filter only, never a gatebackend/research/bot.py meta_label; backend/tests/test_bot.py::test_meta_label_only_filters; test_ml.py guard narrowed to ml.fit/ml.predict
• Work order #2 Part B: strategy discovery engine - building blocks (structure, OB/FVG, liquidity sweep, volume profile, sessions, news timing, COT, volatility, candle patterns, H4->M5), stops from structure/ATR, targets at R multiples or the setup's level, tick outcomes, train/validation/test with Bonferroni, walk-forward, account gates; survivors versioned in the registry; runs every 15 minutesbackend/research/discovery.py, migration 023, task 'AI Gold Strategy Discovery', Backtesting Lab -> Strategy discovery, docs/AI_TRADER.md
• Work order #2 Part C: AI brain - Claude (official SDK), Gemini, OpenAI-compatible (Grok, OpenRouter, Groq, local); providers/models editable in the dashboard; encrypted key manager with priority, automatic failover, statuses, Test key and alerts; versioned prompt files with input contract and output schema; schema validation with fallback; call journal; no LLM in backtestsbackend/research/brain.py, backend/research/prompts/, migration 024, /ai page, monitoring check ai_keys
• Work order #3 Part E: LLM cost control - cost per call, daily budget per key with failover and alert, credit and days left, prompt caching of the static part, model routing per purpose, explain and ideas endpointsbackend/research/brain.py, migration 027, /ai page
• Work order #4 Part D: LLM only when needed (news window asked once per phase), routed key answers first and shows active, Groq /models picker, duplicate key warningbackend/research/brain.py (_order, list_models, duplicate_of), autopilot.llm_reason

Work

• Establish rule-based baseline before ML.
• Define feature generation and target labels without leakage.
• Split data chronologically into training, validation, and test sets.
• Add walk-forward evaluation, calibration, robustness checks, and drift monitoring.
• Store model artifacts and metadata securely.
• Prevent arbitrary online learning from directly changing live strategy behaviour.
• Compare ML with simple baselines and account for multiple-testing/overfitting risks.
• Require reproducible experiments and model approval.

Exit criteria

• Demo orders never reach a live account.no order function exists (test); broker read refuses non-demo accounts (test); LIVE/DEMO flags false
• State reconciliation works.paper journal replay MATCH, tampering MISMATCH (test); Exness demo terminal MATCH (real)
• All trades and rejected orders are recorded.paper_orders FILLED/REJECTED with reasons, blocked previews, reconciliations table
• Environment label is visible everywhere.PAPER header/badge and Mode column on /paper and /positions; mode field in every paper/demo API result

Deliverables

• Demo/paper engineoperations.py previews/fills/rejections/closes + demo.py journal, reconciliation
• Demo account UI/paper and /positions (PAPER), broker demo check (read-only)
• Trade journalGET /paper/accounts/{id}/journal: signal snapshot, fills, outcomes, rejections, P&L, drawdown, execution costs
• Environment-isolation teststest_demo.py: non-demo accounts refused unread; no order function in backend or MQL5

Work

• Create explicit paper mode and, when available, connect to a broker demo account.
• Simulate order lifecycle, spread/slippage assumptions, rejected orders, and fills.
• Reconcile demo account positions with the internal state.
• Record signal-time snapshots and trade outcomes.
• Add demo P&L, drawdown, and execution-quality views.
• Label every screen and message with DEMO/PAPER.
• Add tests proving demo configuration cannot route to live credentials.

Exit criteria

• Invalid/stale inputs reject an order.stale quote/equity, missing fields, unset thresholds, bad prices and specs block (tests)
• Hard limits cannot be bypassed by a strategy or model.settings written only by the re-authenticated endpoint (test); no strategy/model code calls it
• Risk preview matches test fixtures.1040 USD per lot -> 0.09 lots; USC 104000 -> 0.09; EUR at 0.5 -> 0.19
• No live orders are enabled yet.mode live always blocks; no order function exists

Deliverables

• Risk enginerisk.assess, pre_news_decision (Case A, B, C), news_state, breakeven_stop; Case B off by default (owner)
• Lot-sizing previewPOST /risk/check with explanation; Risk Center preview from the terminal spec
• Risk configuration UI/risk: owner settings with re-authentication, kill switch, resume
• Risk audit trailrisk_decisions with input snapshots; audit events for every check, change, halt and resume
• Extensive unit teststest_risk.py: fixtures, USC, conversion, rounding, stale/missing, limits, news, Case A/C
• Work order #4 Part C: news window currencies and medium/low impact options in the Risk Center (default high-impact USD), re-authentication + auditbackend/research/risk.py news_filter, src/components/risk-center.tsx

Work

• Implement lot-size calculation using actual symbol specifications, stop distance, account currency, tick/contract value, cost assumptions, broker volume step and margin constraints.
• Round volume down and reject if minimum permitted volume exceeds risk policy.
• Implement per-trade/open-risk/daily/weekly/drawdown/exposure/margin/news/spread/stale-data limits.
• Add preview API and risk explanation.
• Store each decision and input snapshot.
• Add kill switch and pause state.
• Add tests for missing data, stale equity, currency conversion, unusual contract specs, and rounding boundaries.
• Ensure AI cannot alter hard limits.

Exit criteria

• Read-only sync is reliable.real Exness demo sync ok in 0.44 s; failures become unavailable/refused, never exceptions (tests)
• Demo order lifecycle is tested with explicit confirmation.Owner authorized demo orders in writing (work order #2 item 11). Previews a02d3ca1 and b350de17 BLOCKED by the risk engine (news blackout; 0.01 lot over 1% of 100 USD). First demo order through the demo AI path: ticket 2659776381 buy 0.01 XAUUSDm @ 4024.782, SL 4024.222, TP 4027.222, retcode 10009 FILLED; closed at the stop -0.56 USD (-1.0 R), synced from the broker deals.
• No secret appears in logs.password encrypted, never returned; audit and snapshot checked (test)
• Live execution remains disabled.real accounts refused before sending (test); LIVE_TRADING_ENABLED=false

Deliverables

• MT5 adapterbrokers.Mt5Adapter behind the common interface; read-only sync, health, reconciliation, demo-only orders
• Demo integration teststest_brokers.py with a mocked terminal; real read-only sync on Exness demo; first real demo order awaits the owner
• Broker setup guidedocs/BROKER_ADAPTERS.md
• Connection/health UI/accounts
• Work order #2 Part D: demo AI mode - demo-only guard, kill switch and maintenance stop it, live snapshot, LLM decision checked before execution, open/move SL-TP/breakeven/trail/partial/close with a reason each, broker_orders + trade journal + decision journal, R from deals, demo report pagebackend/research/autopilot.py, migration 025, task 'AI Gold Demo AI' (1 min), /demo-ai page
• Work order #3 Part A/E: rule scans on every closed M5 bar, LLM called only when needed (setup, open trade, news window, 30-minute London/New York review), journal states the actual fill and riskbackend/research/autopilot.py (llm_reason, strategy_scans, migration 026)
• Work order #4 Part B: demo evidence mode - S1-S4 and all discovered strategies on demo, prompt trader-v2 (negative evidence is no reason to refuse; AI chooses SL/TP/risk and may exceed owner settings on demo)backend/research/autopilot.py, prompts/trader/v2

Work

• Confirm exact broker, MT5 terminal environment, account type, server, and symbol naming.
• Implement a separate adapter behind the common broker interface.
• Connect read-only first; retrieve account metadata, symbol specifications, quotes, positions, and orders.
• Add connection health, timeouts, reconnection, and reconciliation.
• Test demo order placement only after read-only checks and risk preview pass.
• Handle partial fills, rejections, disconnects, and ambiguous order acknowledgements.
• Keep live order path disabled.
• Document installation and terminal dependencies on Windows VPS.

Exit criteria

• Supported MT4 setup passes demo tests.everything built and tested with mocks (9 tests); the last test needs the owner's MT4 demo terminal with the EA attached (docs/BROKER_ADAPTERS.md, 9 steps)
• Unsupported configurations fail safely and explain why.missing/stale/unreadable bridge, wrong symbol, no secret, real account, foreign ticket, bad signature, expired command: refused with the reason (tests)
• MT4 and MT5 credentials/adapters remain isolated.separate adapters, connection rows and secrets (bridge secret encrypted per connection)

Deliverables

• MT4 adapter or a documented compatibility blockermt4_bridge.Mt4Adapter + AIGoldBridge.mq4 (read-only); blocker: no MT4 terminal to validate
• Bridge security designdocs/BROKER_COMPATIBILITY.md
• Compatibility test suitetest_mt4_bridge.py on fixture files
• Broker compatibility matrixdocs/BROKER_COMPATIBILITY.md
• Work order #5: signed command bridge - EA v2 executes HMAC-SHA256-signed commands (open with SL/TP, close, modify, cancel), demo-only on both sides, 30 s command lifetime, at-most-once ids, magic-number guard, stop always set; adapter submit/close/modify/cancel, deals from closed orders, UNKNOWN on a lost answermt4/AIGoldBridge.mq4 v2, backend/research/mt4_bridge.py, brokers.py (MT4 previews), Accounts form (bridge secret)

Work

• Evaluate the selected broker's MT4 interface and terminal requirements.
• Implement a separate MT4 bridge/EA/API adapter as technically supported.
• Define message schema, authentication, heartbeat, retries, idempotency, and reconciliation.
• Test account information, symbol specs, quotes, orders, fills, stops, and error cases.
• Add a compatibility matrix by broker, terminal version, and account type.

Blockers / decisions needed

• Owner: open an Exness MT4 demo account, install MT4, compile and attach mt4/AIGoldBridge.mq4 with a bridge secret (docs/BROKER_ADAPTERS.md 'MT4 demo terminal and bridge EA'), add the MT4 connection with the same secret, Sync now, one demo order preview -> confirm.

Exit criteria

• Public and private destinations cannot be confused.explicit visibility; public only when approved and sanitised (test)
• Retries do not create duplicate messages.UNIQUE(event_id, destination_id); a SENT row is never resent (test)
• Messages distinguish signal, simulated trade, demo execution, and live execution.mode line SIGNAL/PAPER/SHADOW/DEMO; live refused (test)
• No secret or private account identifier is published.tokens encrypted and masked; public text without balance/account/ticket (test)

Deliverables

• Telegram settings UI/telegram
• Outbox/workertelegram_outbox + telegram.deliver (python -m backend.research.telegram)
• Public/private message templatestelegram.render for all roadmap 14 kinds, public sanitised
• Delivery history and testsoutbox table and UI; test_telegram.py with a mocked Bot API
• Work order #5: send test message per destination, bot enable/disable, destination removal (history kept), demo AI trade events (opened with chart screenshot, SL/TP, partial, TP hit/closed with R) from broker confirmations, LIVE messages only while the live switch is on, confidence + live state in the daily reportbackend/research/telegram.py (send_test, notify), autopilot.py, monitoring.py, telegram-center.tsx
• Telegram test message SENT on the Contabo VPS (bot 'MD Gold Trading', private destination, purposes signals/alerts/system)owner, 2026-11-12, Telegram Center 'Send test message' on https://gold.mdclothstore.com (release 0.34.2); reports bot still open
• 0.34.3 (work order #14 D): a network failure shows its underlying reason (for example 'network: URLError: ... CERTIFICATE_VERIFY_FAILED ...') in the outbox and the test result, the token maskedtelegram.deliver / send_test; test_network_errors_keep_the_reason_but_never_the_token

Work

• Create Telegram bot configuration using a server-side secret.
• Configure separate private and public destinations.
• Implement persistent outbox, retries, deduplication, delivery receipts/status, and message IDs.
• Build templates for every lifecycle event and the daily report.
• Public messages must be sanitized and explicitly approved for publication.
• Only report actual fills and SL/TP changes after broker confirmation.
• Prevent Telegram outages from blocking risk decisions or order safety.

Blockers / decisions needed

• Owner: the reports bot (token and its destination) - add it in Telegram Center, then press 'Send test message'. The signals bot 'MD Gold Trading' is set up and its test message was sent on the VPS (2026-11-12).

Exit criteria

• Shadow mode demonstrably sends no broker orders.runner calls only bars and quote; fake terminal records no submit (test); isolation test
• Promotion requires all configured gates.approve refused unless every gate PASS (test)
• Approval/rejection is auditable.promotion_decisions with evidence + audit events (test)
• Insufficient evidence cannot pass automatically.INCONCLUSIVE below thresholds or without shadow trades; registry cannot approve (tests)

Deliverables

• Shadow runnershadow.run_all / run_strategy, task every 5 min, /shadow/run, /shadow/status
• Promotion gate serviceshadow.evidence with demo/shadow gates from real records
• Evidence reportGET /promotion/evidence and the Strategy Registry panel
• Approval and rollback workflowPOST /promotion/approve, reject, rollback (re-authentication, audit, stored evidence)
• Work order #2 Part E: shadow runs for bot (S1-S4) and discovered strategies in DEMO/SHADOW on the live signal path (hypothetical fills, stop/target on M5 bars, intraday close)backend/research/shadow.py run_rule_strategies; backend/tests/test_autopilot.py

Work

• Run approved research versions against live market/news data in shadow mode.
• Save signal-time snapshots, hypothetical entries/exits, estimated costs, and later outcomes.
• Measure data delays, spread, missed signals, and simulated slippage.
• Implement promotion checklist and evidence report.
• Require owner approval and re-authentication for promotion.
• Add pause/revoke/rollback to previous strategy version.
• Define minimum observation and evidence criteria before the first live pilot.

Exit criteria

• All live preconditions are explicitly checked.pilot.preconditions + tests (test_pilot.py)
• No critical security, data, risk, or reconciliation defects remain.11 live/confidence tests; demo drills passed; live refused without MFA/confirmation/preconditions; owner limits enforced on live (tests)
• Owner approval is recorded.by design: only the owner switches live on (Live Pilot) after 2-3 months of demo results; the approval is then recorded in live_audit and live_settings
• Kill-switch and recovery drills pass in a non-live environment first.2026-11-02 on Exness-MT5Trial15 demo: kill switch PASSED (position opened, risk engine blocked, flatten closed it, switch restored), cancel pending PASSED (buy limit placed, seen, cancelled), safe restart PASSED (order sent, row left SENDING, recovery -> FILLED, no re-send, idempotency key, live stays off)

Deliverables

• Live feature flag and approval workflowlive_settings (OFF), POST /live/enable: owner password + MFA code (MFA must be enrolled), exact typed confirmation, REAL broker account + ONE approved version, every precondition; Live Pilot page
• Live execution audit traillive_audit (append-only, no update/delete path): enable/refusal, send, order, block, error, pause, recover, review, emergency, drills
• Emergency controlskill switch; POST /live/emergency (kill switch + live off + close system positions + cancel pending); disable without password
• Runbook and incident checklistdocs/RUNBOOK.md, docs/INCIDENT_RESPONSE.md (work order 2026-10-29 item 29)
• Preconditions checklistpilot.preconditions: MFA, REAL account synced <15 min, reconciliation MATCH, one approved version, no other live version, overall confidence >= threshold, 3 demo drills <30 days, halt clear, no critical alert, backup <26 h
• Live cycle: idempotency, reconciliation, automatic blocks and pauses, owner limitspilot.cycle: client_key UNIQUE per signal, recover SENDING/UNKNOWN from the broker, block on stale data / disconnection / spread / risk engine, pause on state mismatch / 3 errors in 60 min / drawdown / daily loss / confidence below threshold, risk.assess(live_enabled) with the Risk Center limits; unverified deployment switches live off
• Daily reviewpilot.daily_review in the daily report task (orders, blocks, errors, equity change, confidence, drills; daily-loss pause)

Work

• Keep live disabled by default until owner explicitly authorizes it.
• Require MFA, re-authentication, confirmation, and documented broker/account selection.
• Enable only one approved strategy version at first.
• Use strict, pre-agreed risk limits and minimal permitted exposure.
• Require risk engine approval for every order.
• Add order idempotency and broker reconciliation.
• Block new orders on stale data, disconnection, state mismatch, excess spread, repeated errors, or risk breach.
• Test kill switch, cancel-pending flow, and safe restart in demo/staging first.
• Add daily review and automatic pause conditions.

Blockers / decisions needed

• Owner: switching live on is the owner's own action in Live Pilot after the demo results (overall confidence >= 80 % and every precondition).

Exit criteria

• Backups can be restored.restore-test passed (local); off-site copy needs the owner's destination
• Alerts cover critical failures.database, disk, data freshness, jobs, broker, Telegram, kill switch (tests)
• A clean deployment can be reproduced from documentation.reproduced by the owner on the Contabo Windows VPS from docs/DEPLOYMENT.md on 2026-11-12 (C:\AiGold, user aigold, release 0.34.2, https://gold.mdclothstore.com, MFA on, demo AI running); the four problems found there are fixed in 0.34.3 (work order #14)
• Live mode starts disabled after recovery or unverified deployment.no live path; restore-test reports live disabled

Deliverables

• Production runbookdocs/RUNBOOK.md, DEPLOYMENT.md, OPERATIONS.md
• Backup/restore evidencescripts/backup.py restore-test passed 2026-10-28 on the newest backup (migrations applied, counts read)
• Monitoring dashboards and alertsmonitoring.py checks/alerts, /reports health panel, private Telegram alerts
• Release checklistdocs/RUNBOOK.md
• Maintenance plandocs/RUNBOOK.md
• Dashboard no longer waits for the history explorer: coverage keyed by store file, refreshed in the background (2 s wait, else the last report); explorer has its own lockbackend/research/history.py; /dashboard 60 s timeout -> 0.06 s
• Work order #5 Part F: single Windows VPS deployment (no Docker) - install.ps1 (machine env, build, firewall, auto-start tasks), release.ps1 (backup, update, build, restart, health check, automatic return; -Ref rollback), Caddyfile with log rolling, server.env templatedeploy/windows/, docs/DEPLOYMENT.md (15 steps, memory and disk budget, release checklist, rollback)
• Worker resource limits and log retentionscripts/run_task.py: Windows Job Object hard memory + CPU caps for research workers (verified: 300 MB cap -> MemoryError), logs/<task>/YYYY-MM-DD.log with retention
• Off-site backups (configurable destination), market-store backup, restore commandscripts/backup.py --offsite / BACKUP_OFFSITE_DIR, --market, restore FILE (live OFF); monitoring 'Backups' check
• Load and failure test scriptsscripts/load_test.py, scripts/failure_test.py
• 0.34.3 (work order #14, found on the VPS): A periodic tasks run on their schedule right after install.ps1 / -TasksOnly / release.ps1 without a logon (time trigger with the repetition beside AtLogOn); B install.ps1 resets the ACLs under the project folder and gives the -User account Modify (copied files had no write access for aigold); C install.ps1 updates the machine root certificates from Windows Update (Python CERTIFICATE_VERIFY_FAILED to api.telegram.org); D Telegram network errors keep their reason; DEPLOYMENT.md steps 3, 7 (copy + size/SHA256 check) and 9deploy/windows/install.ps1, docs/DEPLOYMENT.md; on this PC: temporary task 'ZZ WO14 Repeat Test' ran every minute without a logon (02:22:54, 02:23:54; re-registered: 02:25:55, 02:26:55; result 0), then deleted; ACL reset + grant on a scratch copy of the VPS's broken ACL (workspace.sqlite3 and service-token writable again); certutil -generateSSTFromWU produced 555 roots incl. Go Daddy
• 0.34.4 (work order #15): release.ps1 and install.ps1 -TasksOnly run as the non-admin aigold - an elevated install.ps1 (or install.ps1 -TasksOnly once on an installed server) gives the -User account full control of each AI Gold task it registers (no other user, no other task); the elevated install also makes -User the owner of the project files (git refused the repository owned by Administrators); DEPLOYMENT.md 'Administrator or aigold' with the one-time steps for a server installed before 0.34.4deploy/windows/install.ps1 Grant-TaskControl, icacls /setowner; on this PC (elevated, temporary non-admin user zzwo15test and ZZ WO15 tasks, all deleted): a task registered the old way -> disable, enable, re-register 'Access is denied' as the user (the VPS error); with the grant -> disable, stop, enable, start, re-register (periodic and monthly) all OK and the grant kept; git status as the user: 'dubious ownership' after the 0.34.3 sequence, OK after /setowner; owner's AI Gold tasks unchanged

Work

• Add production monitoring, alerting, log retention, disk/database growth alarms, and worker resource limits.
• Configure encrypted off-VPS backups and test restore.
• Run load and failure tests.
• Review security, dependency updates, permissions, and secrets rotation.
• Add daily/weekly/monthly reports.
• Add release/rollback process and change approval.
• Document incident response, VPS replacement, broker outage, data provider outage, and database recovery.
• Review performance and add time-series/object storage only when measured need justifies it.

Blockers / decisions needed

• Owner: choose the off-site backup destination (BACKUP_OFFSITE_DIR) and run the off-site backup on the VPS.
• Owner: restore test and load/failure tests on the VPS (docs/DEPLOYMENT.md steps 14-15).

Exit criteria

• Every skill passes the causal test before it can be registered; outputs carry method/version, as_of and lag.registry register: 3/3 passed (catalogue with 9 exclusions)
• No skill reaches the AI unless the owner switched it to ai_input (re-authentication, audit).all skills research_only after registration; tests
• LIVE_TRADING_ENABLED stays false; S1-S4, discovery, risk engine, live pilot, demo AI behaviour and Telegram unchanged.git diff: autopilot.py adds only the optional skills block; full backend suite
• Parts F-G built.Parts B-G built (0.29.0-0.34.1); x2strategy dropped by owner decision (0.34.1); phase marked complete by the owner

Deliverables

• Part A1: Phase 21 in roadmap.json with parts A-G (only A has deliverables); tracker and acceptance record regeneratedroadmap.json, ROADMAP_TRACKER.html, docs/PHASE_ACCEPTANCE.md (npm.cmd run roadmap)
• Part A2: skill contract (id, name, category, version, source repo + exact package version + licence, run() on closed bars only -> one JSON result with skill id/version, symbol, timeframe, as_of, computed_at, input fingerprint, status ok/stale/error, confirmation_lag_bars per output, rule text, outputs)backend/research/skills/contract.py; test_result_carries_the_contract_fields_and_uses_closed_bars_only
• Part A2: migration 030 - skills (mode research_only / ai_input / disabled, default research_only) and skill_runs; causal_test column keeps the last test report (DECISIONS 2026-11-09)backend/research/migrations/030_skills.sql; test_migration_030_creates_the_registry_tables
• Part A2: generic causal harness for all skills (bars[:n] vs bars[:n+k], 6 cuts x k 1/3/25/150 = 24 cases, exact equality of every value known at bar n after its lag); a failing skill cannot be registered (a failing new version is disabled)contract.causal_test, registry.register; tests catch a centred mean, whole-range scaling, a lag-0 swing that needs lag 2 and a skill reading the next close
• Part A2: mode changed only by the owner from the dashboard with re-authentication; every change audited (category Skills)PUT /skills/{id}/mode (security.reauthenticate); test_the_skills_page_lists_every_skill_and_only_the_owner_changes_a_mode
• Part A3: main .venv gets only TA-Lib 0.8.3 (bundled C library wheel) and talipp 2.7.0, pinned in requirements.txt and requirements-lock.txt; research .venv-research with requirements-research.txt (pandas-ta-classic 0.9.0, pandas 3.1.1, numpy 2.5.4, python-dateutil 4.0.0, tzdata 2026.6)backend/requirements*.txt; node scripts/workspace.mjs setup creates and updates both
• Part A3: the main backend calls the research environment only as a worker subprocess through scripts/run_task.py --capped (JSON in / JSON out files, CPU/RAM caps)skills/catalogue.py call(); logs/skills-research: 'caps: memory 3072 MB, CPU 35 %, below-normal priority (applied)'
• Part A3: install.ps1, release.ps1 and docs/DEPLOYMENT.md create/update both environments, register the skills and add the task 'AI Gold Skills Live'; server.env.example unchanged (no new setting needed); Start Workspace.cmd and npm.cmd run workspace keep workingdeploy/windows/install.ps1, release.ps1 (PowerShell parser: 0 errors), docs/DEPLOYMENT.md steps 9 and Release; workspace setup run on this PC
• Part A4: indicators-talib - ATR, ADX, +DI/-DI, RSI, MACD, Bollinger Bands, EMA 12/20/26/50/200, Stochastic (fast), CCI, OBV on tick volume, all 61 TA-Lib candlestick patterns (data only, never signals), any timeframe M1-D1, Dukascopy or Exness XAUUSDmskills/indicators.py; registry run indicators-talib --source exness --timeframe M15 (as_of 2026-11-03T08:00:00Z, ok)
• Part A4: cross-check against advanced.technical_analysis (advanced-0.4.0, unchanged): EMA20/EMA50/RSI equal (max 4.5e-12); ATR max 1.4e-3 and MACD/signal max 0.18/0.38 from different seeds (TA-Lib skips the first bar's true range; aligns the MACD fast EMA with the slow one), below 1e-11 after 500 bars - documentedtest_talib_against_advanced_py_on_the_same_bars; docs/SKILLS_LAYER.md
• Part A5: indicators-live - talipp incremental update on every CLOSED M1 and M5 Exness XAUUSDm bar (never the forming bar), no recomputation; warm-up from stored history on start and after a reconnectskills/live.py; test_live_worker_warms_up_adds_each_closed_bar_and_rewarms_after_a_reconnect; 75 s run on the open MT5 terminal: warmed up M1 2012 / M5 2001 bars, new M1 bar 09:18 stored at 09:19:02 (scratch database)
• Part A5: incremental talipp values equal the TA-Lib batch values on the same bars after the 500-bar warm-up: owner's 7,658 stored Exness M15 bars EMA50 max difference 7.3e-12, every indicator <= 1e-9 (CCI 9.9e-10)test_incremental_talipp_values_equal_talib_batch_values, test_talipp_equals_talib_on_the_owner_7658_stored_m15_bars
• Part A6: features-catalogue in the research environment - 403 pandas-ta-classic columns incl. 62 candlestick patterns (7,658 M15 bars: 10.8 s under the 35 % CPU cap); causal harness on every column, 9 excluded with the reason (DPO_20, ICS_26, 7 x TOS_STDEVALL), whitelist 394 stored with its version; not connected to the discovery engineskills/catalogue.py, catalogue_worker.py; registry register (25.5 s) and run features-catalogue; test_features_catalogue_runs_in_the_research_environment_and_excludes_non_causal_columns
• Part A7: optional skills block in the trader input (input_contract trader-v4) with only ai_input skills; empty -> the trader receives exactly today's input; prompt trader-v4 explains as_of, confirmation lag and status; the active prompt stays v3autopilot.py (trader_prompt), skills/registry.ai_block, prompts/trader/v4/*; test_the_trader_input_is_unchanged_while_no_skill_is_in_ai_input_mode, test_an_ai_input_skill_adds_the_skills_block_and_prompt_v4
• Part A8: sidebar page Skills - per skill name, category, mode, version, source repo link, package version, licence, last run, as_of, duration, status, causal test result, expandable latest output; owner-only mode switch with re-authentication; Indicators panel (Exness XAUUSDm from indicators-live with as_of)src/app/skills/page.tsx, src/components/skills-center.tsx; tests/work-order6.spec.ts at 1920, 1366 and phone width
• Part A9: docs/SKILLS_LAYER.md, docs/THIRD_PARTY.md, DECISIONS.md (private personal use), CHANGELOG 0.28.0, PROJECT_STATUS.md, README handofffiles in this release
• Part B0: DECISIONS entry for the three Part A answers (causal_test column approved; no schedule for indicators-talib / features-catalogue, Part F decides; indicators-live without candlestick patterns)DECISIONS.md 2026-11-09
• Part B0b: indicators-live warm-up reads >= 500 closed bars per timeframe (2,000 stored; unchanged) on start and after a reconnect; results published before 500 bars have status stale with null valuesskills/live.py store(); test_live_warm_up_needs_500_closed_bars_and_earlier_values_are_stale
• Part B1: skill smc (smartmoneyconcepts 0.0.27, MIT, SMC_CREDIT=0) in .venv-research: swings, BOS, CHoCH, FVG + mitigation event, order blocks with strength, liquidity and sweeps, previous day/week/month high-low, sessions and ICT kill zones with DST, retracement %; causal adapter with lags swing 5 / FVG 1 / events 0skills/smc.py; raw library FAILS the harness (swing, bos, choch, ob, liquidity, fvg mitigation; 116 swing labels changed on 1,200 synthetic bars), adapter passes 24/24 (test_raw_smc_output_fails_the_causal_harness_and_the_adapter_passes); DST tests
• Part B1: agreement report smc vs advanced.technical_analysis (advanced-0.4.0, unchanged) on Dukascopy and Exness M15/H1, stored and shown on the Skills pageregistry report smc_agreement (5,000 bars each): e.g. Exness M15 swings 575 smc / 552 advanced / 485 within 1 bar, FVG 1051/1022/1022, BOS 171/107/75, CHoCH 98/69/34, OB 233/176/60, liquidity 88/9/6
• Part B2: py-market-profile licence confirmed from its LICENSE file (BSD 2-clause) - used as PyPI MarketProfile 0.2.0; skill auction: per completed session (London, New York, day) POC/VAH/VAL 70 %, IB, OR, HVN/LVN on tick volume; developing session separate with as_of; completed sessions never changeskills/auction.py; test_completed_sessions_never_change_and_the_developing_session_is_separate
• Part B2: OrderFlow-Analysis-Pro (MIT) ported in our own code - shape bias P/b/D/double, qualified levels, merged multi-day value area, value accepted higher/lower, VAH rejection, failed-auction hooks; no delta/footprint/orderbook/Bybit/MT5 feed codeskills/auction_port.py; test_profile_shapes_on_hand_built_histograms, test_failed_auction_hooks_value_acceptance_and_merged_value_area; attribution docs/THIRD_PARTY.md
• Part B2: cross-check auction vs S2 levels on the same days (S2 unchanged)registry report auction_vs_s2 (20 days): Dukascopy 42 sessions POC mean |diff| 2.90 (52 % within 2 rows), Exness 40 sessions POC 4.42 (50 %); causes in the report
• Part B3: skill levels-trendlines (trendln 0.1.18) on the trailing 125-bar window (configurable), confirmed extrema only (METHOD_NAIVE, no edge), slope/intercept/touches/errors, projected price at the current barskills/trendlines.py; test_trend_lines_use_only_the_trailing_window; causal harness passed
• Part B4: stolgo 0.2.0 in .venv-research with --no-deps (owner decision; bandl not installed); S3 implemented once in stolgo with next-open fills, our spread and commission, compared trade by trade with our engine (engine unchanged)skills/stolgo_rules.py, registry report stolgo_vs_engine --year 2025: 192 our setups / 192 stolgo trades, 192 same signal bar and side, 167 same exit kind, R difference mean 0.108 / mean abs 0.533 / max abs 3.875; 20 of our exits were the pre-news rule (not in the port)
• Part B4: stolgo level/relation/pattern grammar and presets consolidation_breakout, breakout_intraday, failed_break_intraday callable as the research-only rule set rules-stolgo (not connected to discovery)worker action rules_eval; test_stolgo_grammar_evaluates_relations_combinators_and_presets; causal harness passed
• Part B5: references only - KVignesh122/MT5-SMC-trading-bot filters vs S1 (ATR displacement no, ADX/ER regime no, session gate yes, spread cap only in the risk engine); EarnForex/MarketProfile MT5 install steps for a visual comparisondocs/SKILLS_LAYER.md 'References only (B5)'
• Part B6: every Part B package installs beside the pinned .venv-research packages (stolgo only with --no-deps, owner decision); full research lock installed with --no-deps; long-running capped worker 'AI Gold Skills Structure' (smc + trendlines on M5/M15/H1/H4, auction M5, every closed M5 bar); install.ps1 registers it, release.ps1 re-registers tasks (install.ps1 -TasksOnly); Dukascopy on demand onlybackend/requirements-research.txt, structure_worker.py, feed.py; fresh venv from the lock imported every package; 3-minute run on the open MT5 terminal stored 9 runs per closed M5 bar (scratch database)
• Part B7: the four new skills on the Skills page with the Part A fields plus the SMC agreement, auction vs S2 and stolgo vs engine reports; no new page, no chart overlaysrc/components/skills-center.tsx; tests/work-order7.spec.ts at 1920, 1366 and phone width
• Part B8: docs/SKILLS_LAYER.md, docs/THIRD_PARTY.md, DECISIONS.md, CHANGELOG 0.29.0, PROJECT_STATUS.md, this recordfiles in this release
• Part C0: causal rule - published_at (source) and known_at (our fetch, whole seconds rounded up) on every news item and Fed text; research reads by known_at only; macro = ALFRED vintages known from the end of the vintage day in New Yorkskills/macro_news.py (headlines_known, fed_known, macro_as_of, Vintages); tests test_research_reads_news_by_known_at_never_by_published_at, test_alfred_values_are_never_used_before_their_vintage_was_published, test_a_skill_reading_items_by_published_at_would_see_the_future
• Part C1: skill macro - FRED/ALFRED DFII10, T10YIE, DGS10, DFF, DTWEXBGS, CPIAUCSL (fredapi 0.5.2, all vintages, owner key encrypted) and Yahoo daily DX-Y.NYB, ^TNX, GC=F (yfinance 1.8.0, labelled 'Yahoo, delayed, personal research use'); snapshot with value, as_of, known_at, 1- and 5-observation change; task AI Gold Skills Macro daily 22:30 UTC + Refresh nowskills/macro_worker.py, providers.py (macro CLI, POST /skills/macro/refresh), Skills page Macro snapshot; Yahoo terms in docs/THIRD_PARTY.md
• Part C2: OpenBB evaluated in a throwaway environment (openbb 5.0.0): providers for gold spot, DXY, US yields, real yields, economic calendar, keys vs free; not added to the botdocs/SKILLS_LAYER.md 'OpenBB evaluation'
• Part C3: existing COT import checked - Disaggregated managed money present, history only from 2006-06-13; cot_reports 0.1.3 fills exactly the gap: Legacy gold 1986-01-15 -> 2006-06-06 (876 weeks, table cot_legacy), same release rule (no CFTC-published dates -> never used); skill cotmigration 032 cot_legacy, macro_worker cot-legacy, news_sources.release_for(backfill=False), news_sources cot-legacy; tests test_legacy_cot_fills_only_the_gap_before_the_disaggregated_history, test_legacy_weeks_without_a_cftc_published_date_are_never_used
• Part C4: skill news - owner feed list (seeded with verified Fed press, Fed speeches, ECB press, BoE news; feedparser every minute, ETag/Last-Modified, backoff), Google News (newspaper4k GoogleNewsSource) and GDELT DOC (gdeltdoc: articles, raw volume and tone) for the owner keyword list, trafilatura full text once per item (robots.txt; failure keeps the headline), dedupe with every source listed, spike flag owner setting OFF without a thresholdskills/news_worker.py, migration 032 (news_feeds, news_keywords, news_settings, news_items, news_item_sources, news_polls, gdelt_timeline); tests on saved fixtures in backend/tests/fixtures/part_c
• Part C5: skill fed - FOMC-RoBERTa (CC BY-NC 4.0) on new Fed statements, minutes and speeches from the Fed feeds, sentence labels stored, Fed index = (hawkish - dovish) / sentences; FOMC statements and minutes history 2003+ from federalreserve.gov (343 documents); FOMC event study with the Phase 9 moduleskills/fed_worker.py, fed_documents, reports.fomc_event_study; test test_fed_index_formula_on_a_hand_labelled_statement. Scoring and the event study wait for the owner's access to the gated model
• Part C6: finBERT model card read - licence cc-by-nc-sa-4.0 (personal use allowed); every headline scored on CPU (label + probabilities), data onlynews_worker.finbert (revision 0bc2ac1), news_items.sentiment; docs/THIRD_PARTY.md
• Part C7: tasks AI Gold Skills News (every minute, research environment, capped; Fed scoring of new Fed texts in the same cycle) and AI Gold Skills Macro (daily); install.ps1, release.ps1, DEPLOYMENT.md step 9a; not left running on this PCdeploy/windows/install.ps1, release.ps1, docs/DEPLOYMENT.md
• Part C8: Skills page (macro, news, fed, cot cards, macro snapshot, Fed index history, FOMC event study), News page Headlines section with GDELT volume/tone per keyword, Settings -> Data providers (FRED key, feed list, keyword list, GDELT spike setting OFF)src/components/skills-part-c.tsx, news-headlines.tsx, data-providers.tsx; tests/work-order8.spec.ts
• Part C9: docs/SKILLS_LAYER.md, docs/THIRD_PARTY.md, DECISIONS.md, CHANGELOG 0.30.0, PROJECT_STATUS.md, this recordfiles in this release
• Part D1: decision context check - multi-timeframe data and SL/TP of open trades were already in the snapshot; exposure, drawdown, recent P&L and consecutive losses added in contract trader-v5trader_flow.exposure / performance, prompts/trader/v5/input_contract.json, docs/AI_TRADER.md D1 table
• Part D2: trader-v5 output schema with falsifier {condition, price} and horizon_utc; an OPEN without both is rejected by the validator and becomes WAIT; WAIT is a first-class answertrader_flow.open_problems; test_open_without_falsifier_or_horizon_becomes_wait
• Part D3: Bull and Bear analysts (routine tier, route debate) only before a new OPEN, 1-3 rounds, then the trader's final decision (decision tier)trader_flow.debate, prompts bull/v1, bear/v1; test_a_full_open_flow_debates_challenges_and_executes
• Part D4: Challenger after the final OPEN on another provider, else another model of the same provider (labelled), else no trade; reject = WAITtrader_flow.challenger_target, prompt challenger/v1; tests for reject, fallback and no second model
• Part D5: owner settings on the AI Brain page (debate on/off, rounds 1-3, Challenger on/off, prompt v3/v4/v5) and the budget check before the flow with the existing budget alertmigration 033, PUT /autopilot/trader-settings, trader-settings.tsx; test_the_budget_is_checked_before_the_flow
• Part D6: decision log (decision_flows) with snapshot reference, debate, falsifier, horizon, Challenger, odds, calls, costs, prompt versions, advisory checks; automatic scoring after the horizon (real R, counterfactual on M1, falsifier hit); shown on the Demo AI page, not used elsewheretrader_flow.log / score_due / scores, GET /autopilot/flows, decision-flows.tsx; test_counterfactual_score_on_hand_built_m1_bars
• Part D7: skill odds-timemachine (10,000 same-time-of-day replays in ATR units, seed 19, plus the same setup on every past pool day), Dukascopy history, Exness live as_ofskills/timemachine.py; test_time_machine_is_reproducible_with_its_seed_and_uses_only_earlier_days
• Part D8: licence confirmed (Apache-2.0 covers the OpenMobius cards); skill knowledge: 726 concept cards indexed in ChromaDB with its local model, our smc/auction terms matched to definitions and rulesskills/knowledge.py, knowledge_worker.py, migration 034 knowledge_terms, backend/research/data/openmobius (LICENSE, NOTICE)
• Part D9: cot-contrarian, news-reaction-failure, macro-regime, drawdown-breaker and discipline-gate (advisory) ported with our data; risk-engine overlap listed; post-trade reviews (postmortem, coach, weekly digest) on the Reports page; the other 65 skills of the repository listedskills/trading.py, reviews.py, docs/SKILLS_LAYER.md Part D; tests in test_trader_skills.py
• Part D10: prompts trader v5, bull v1, bear v1, challenger v1, postmortem v1, coach v1, digest v1 with JSON schemas; v3 and v4 selectable for rollback; v5 active on the demo AI after all tests passedbackend/research/prompts; autopilot_settings.trader_prompt = v5 on the owner's workspace (audit log, 2026-11-09T21:5x UTC)
• Part D11: Demo AI decision flow panel, AI Brain trader settings, Skills page (7 new skills), Reports trade reviewsdecision-flows.tsx, trader-settings.tsx, trade-reviews.tsx; tests/work-order9.spec.ts
• Part D12: docs/AI_TRADER.md, docs/SKILLS_LAYER.md, docs/THIRD_PARTY.md, DECISIONS.md, CHANGELOG 0.31.0, PROJECT_STATUS.md, DEPLOYMENT.md, this recordfiles in this release
• 0.31.1 (owner decisions after Part D) 1: demo AI may OPEN inside the USD high-impact news window only within the Risk Center limits news_window_max_sl_atr / news_window_max_tp_atr (M5 ATR14 multiples, empty by default = WAIT 'news SL/TP limits not set'), enforced in code for trader v3, v4 and v5; the live pilot keeps the hard blockrisk.RiskSettings, autopilot.news_limits / news_issue, trader_flow; prompts trader v3/v4/v5 News policy; risk-center.tsx; test_owner_decisions_part_d.py
• 0.31.1 2: weekly AI self-review (decision tier, prompt self_review-v1) after at least self_review_min_scored scored decisions (default 30, AI Brain page) may change only debate on/off, rounds 1-3 and Challenger on/off, with reasons on the AI Brain page and in the audit logreviews.py, migration 035 (applied on this PC's workspace 2026-11-10T05:44:49Z by the scheduled tasks; schema checked against the file), prompts/self_review/v1, trader-settings.tsx
• 0.31.1 3-4: Challenger skips a provider whose keys are all failing (then another model of the trader's provider, labelled); knowledge sends only exact-name card matches to the trader, nearest matches listed on the Skills page for reviewtrader_flow.challenger_target, skills/knowledge.py, skills-center.tsx; test_owner_decisions_part_d.py
• 0.31.2 A: Risk Center values for the demo AI by the owner's written instruction of 2026-11-10 (risk per trade 10 %, reward:risk 2, 4 trades per day, 1 open trade, daily loss 10 %, weekly loss 20 %, every other value unchanged) saved through the Risk Center write path as settings version 3 with the audit note; the same values on a fresh install; the live pilot and the research account simulations keep the unchanged six limits; live pilot OFFrisk.save / apply_owner_instruction / account_settings, migrations 036-037, pilot.risk_settings, bot.py, discovery.py, news_backtest.py; workspace audit entry 2026-11-10T10:40:34Z; test_release_0_31_2.py
• 0.31.2 B1: a Challenger model with no known price is not available (next provider, then a priced model of the trader's provider); a missing price never blocks a trade by itselftrader_flow.challenger_target / flow_estimate; on the owner's workspace the Challenger is now claude-sonnet-5-5 (the Groq and Gemini models have no price)
• 0.31.2 B2: the path-test trade (ticket 2659776381) is marked 'test' in the journal and left out of demo statistics, confidence inputs, the trader's performance block, the advisory skills and the reviewsdatabase.TEST_STRATEGY / NOT_TEST, autopilot.report, confidence.trades, trader_flow.performance, reviews.run, trading.load_account, demo-ai.tsx, home-dashboard.tsx; workspace: 7 trades in the statistics, 1 marked test
• 0.31.2 B3: automatic read-only MT5 sync of the saved connections in the monitoring task (every 5 minutes) and a correct Overview adapter status (connected / LAST SYNC OLD / SYNC ERROR only for a failed sync; unconfigured adapters not counted)brokers.sync_connection / auto_sync, monitoring.main, operations.integrations, home-dashboard.tsx; on this PC the sync reports 'unavailable' while the MT5 terminal is closed
• 0.31.2 B4: drawdown-breaker and discipline-gate 1.1.0 use the owner's Risk Center daily loss, weekly loss and losses-in-a-row values known at the bar (monthly rule not evaluated, 24 h cooldown kept); advisoryskills/trading.py breaker / limits_at / load_account; re-registered on this PC (causal test 24/24 each); live run: limits source 'owner Risk Center' 10 / 20 / 3
• Part E1: skill volatility - GARCH(1,1) with Student-t errors on closed M5, M15 and H1 bars, fitted once per UTC day on the trailing 2,000 bars closed before the day and only filtered forward; forecast for 1, 3 and 12 bars in $ and ATR units with the ATR baseline; reports onlyskills/volatility.py (arch 8.0.0), risk_ml.VolatilitySkill; test_garch_is_fitted_on_a_trailing_window_before_the_day_and_only_filtered_forward; causal harness 24/24
• Part E1 evaluation (walk-forward, Dukascopy 2003-10-28 to 2026-10-27; 1,643,944 M5, 552,497 M15, 138,539 H1 bars): GARCH beats the 24-bar std on correlation and error (owner check confirmed: H1 next bar 0.350 against 0.324, mean of yearly correlations 0.256 against 0.200) but not the ATR14 baseline on error: FAIL in all 9 timeframe x horizon cellsskill_reports volatility_walk_forward 2026-11-10T12:51:33Z (777 s); docs/SKILLS_LAYER.md Part E table
• Part E2: LightGBM (4.7.0, library defaults, research environment) as a second model type in ml.py; meta-labeling of S1-S4 and discovered-strategy setups (label = take-profit before stop-loss on the engine's tick outcomes), 427 causal features (setup features, return features, 394 features-catalogue columns, volatility forecasts), the same walk-forward split with embargo and outcome purging, Platt calibration, Bonferroni verdict and model registry gates (migration 038)ml.meta_setups / meta_matrix / meta_evaluate / run_meta, POST /models with meta_evaluation_id, GET /models/meta; tests for labels on hand-built ticks, the purged walk-forward and the registry gates
• Part E2 results (7 strategies x 2 model types, stored in meta_evaluations): S2 PASS for logistic and LightGBM, S3 PASS for LightGBM and INCONCLUSIVE for logistic, S4 INCONCLUSIVE for LightGBM and FAIL for logistic, S1 and the three discovered versions FAIL; no model improves the net R of taking every setup; no model registered or approvedpython -m backend.research.ml meta (2026-11-10); docs/RESEARCH_GUIDE.md Meta-labeling table (precision, recall, calibration, net R)
• Part E3: skill regime - Gaussian HMM with 2 and with 3 states on H1 log returns and the 24-bar realised volatility, fitted once per ISO week on the trailing 3,000 bars; only FILTERED (forward) probabilities, never Viterbi or smoothed; plain description of each stateskills/regime.py (hmmlearn 0.3.3), risk_ml.RegimeSkill; test_hmm_filtered_probabilities_never_change_when_future_bars_are_added; causal harness 24/24
• Part E3 evaluation (138,539 H1 bars): the states separate the next hour's absolute move (2 states: 0.1214 % against 0.1934 %, 24 of 24 years, PASS; 3 states PASS) and S2's and the pooled S1-S4 results (PASS), not S1, S3 or S4 (FAIL): INCONCLUSIVE overall; the owner's Jul-Oct 2026 check: 0.190 % (1,687 bars) against 0.236 % (261 bars)skill_reports regime_walk_forward 2026-11-10T12:45:04Z (386 s)
• Part E4: skill forecast - zero-shot 10 / 50 / 90 % quantiles of the close and of the high-low range for 1, 4 and 12 bars on closed H1 and H4 bars from Chronos-Bolt small, Chronos-2 small (autogluon/chronos-2-small), Kronos mini and Kronos small (vendored MIT code), CPU only, weights pinned to one revision and cached, one model in memory at a time, model version and CPU time in the outputskills/forecast.py, backend/research/vendor/kronos (LICENSE, NOTICE.md), risk_ml.ForecastSkill; stub-model test (ordered quantiles, causal); causal harness with the real models 24/24, 72 outputs
• Part E4: TimesFM 2.5 - built and measured (crash while loading under the 3072 MB worker cap, works under 4096 MB), then removed by the owner's decision of 2026-11-10 (redundant with Chronos and Kronos; no memory cap raised): model, package pin and weights download removedDECISIONS.md 2026-11-10 point 7; forecast.MODELS has four models; test asserts no TimesFM
• Part E4 evaluation: INCOMPLETE by owner decision (the run stopped when the PC ran low on memory) - H1 done (400 points Chronos-Bolt small and Chronos-2 small, 300 Kronos mini, 200 Kronos small), H4 not done; checkpoint kept, the run continues from itskill_reports forecast_walk_forward 2026-11-10 with incomplete = 'H1: chronos-bolt-small, chronos-2-small, kronos-mini, kronos-small done; H4: not done'; data/skills/forecast/evaluation-H1.json. H1: every close forecast FAIL against 'no change'; range PASS for Chronos-2 small at 1 and 4 bars and Chronos-Bolt small at 1 bar
• Part E5: skill drift - ADWIN and Page-Hinkley (river defaults) on the R of closed demo AI trades overall and per strategy (test trades excluded) and on spread, ATR14 and realised volatility of closed M5 bars; a detection is a warning on the existing monitoring alert path and is shown on the Skills page with each detector's sample size; it never stops tradingskills/drift.py (river 0.26.1), risk_ml.DriftSkill / drift_alert, monitoring.checks key drift; tests on synthetic series with a known change (stub and real detectors)
• Part E6: running - tasks AI Gold Skills Risk and AI Gold Skills Forecast (long-running research workers under run_task.py --capped with their own read-only feed) and AI Gold Skills Drift (15 minutes); install.ps1 / release.ps1 download the weights and register the tasks; evaluations on demand (CLI, Skills page button); nothing left running on the PCskills/risk_ml_worker.py serve risk|forecast|download, skills/risk_ml.py drift|evaluate, POST /skills/{id}/evaluate, deploy/windows/install.ps1 and release.ps1 (0 parse errors), docs/DEPLOYMENT.md 9b
• Part E7: Skills page - the four new skills with the same fields, each evaluation report (metrics against baselines, PASS / FAIL / INCONCLUSIVE, sample sizes, CPU time, incomplete note) with a run button, and the drift detectors; ML page - LightGBM vs logistic regression vs take every setup per strategy with calibration and a register button; no new pageskills-part-e.tsx, skills-center.tsx, research-meta.tsx, research-models.tsx; tests/work-order10.spec.ts
• Part E8: docs/SKILLS_LAYER.md, docs/THIRD_PARTY.md (versions, licences, model cards), docs/RESEARCH_GUIDE.md (Meta-labeling), docs/DEPLOYMENT.md, DECISIONS.md, CHANGELOG 0.32.0, PROJECT_STATUS.md, this record; the answered 'Owner to confirm (0.31.2)' blocker removedfiles in this release
• Part F0: the five research packages (vectorbt 1.2.0, optuna 5.0.0, tsfresh 0.21.2, shap 0.54.1, ruptures 1.1.10) install into .venv-research alongside the existing pins (resolved with the pinned file as constraints, --no-deps, 36 new exact pins, no existing pin changed, no new environment); licences read from each LICENSE filebackend/requirements-research.txt; docs/THIRD_PARTY.md (Part F); DECISIONS.md 2026-11-11 (vectorbt Commons Clause: private, personal use)
• Part F0: free-memory check - every lab job, the Part E evaluations and the lab-enabled discovery search ask for free RAM first; below the job's measured need (peak of its last finished run at that size, else the worker cap) it does not start and the reason is stored (lab_runs 'refused'), audited and shown; worker caps unchangedlab.refusal / lab.recorded / lab.need_mb, skills/risk_ml.py evaluate, discovery.work; migration 039; test_a_heavy_job_does_not_start_without_enough_free_memory_and_says_why
• Part F1 + F7: vectorbt stage at the start of the discovery engine - the stolgo presets consolidation_breakout, breakout_intraday and failed_break_intraday (five families with sides) swept on training-period M5 bars over 4 stops x 5 take profits with costs (spread per hour of day from the Dukascopy bid / ask bars, commission as in our engine); the survivors become triggers whose tick events are built by our own engine and go through the same search and gateslab.discovery_stage / screen / keep / build_events, skills/lab_worker.py screen / sweep / signals, discovery.work and load_tables(lab=True); test_vectorbt_survivors_go_to_our_tick_engine_and_every_screened_candidate_is_counted, test_stolgo_candidates_pass_the_same_search_and_registry_gates
• Part F1: every run records candidates screened, survivors, time, peak memory and the screening rule (the search's own training rule: at least 150 trades and mean R after costs above 0, training period only); every screened candidate is added to the Bonferroni count of the test-period gatelab_reports kind 'screen', lab_runs, discovery_runs.result.lab and multiple_testing; discovery.search(extra_tests)
• Part F1 cross-check: S3 of one year of Dukascopy M5 in vectorbt against our engine with the differences and their causes (as with stolgo in Part B)lab.vectorbt_s3_job, lab_reports kind 'vectorbt_s3'; smoke result in the tests table; test_real_vectorbt_runs_the_engines_s3_setups_on_bars
• Part F2: Optuna (TPE; pruning for LightGBM) instead of a grid - (a) the parameters of the stolgo presets in the screening, on training bars only (the worker refuses a bar at or after the training end); (b) LightGBM hyperparameters of the meta-labeling (third model type lightgbm_optuna) on the first fold's training rows only; studies in the separate file data/optuna.sqlite3; every trial counts in the Bonferroni count and the trial counts are shown next to every resultskills/lab_worker.py screen / lightgbm_tune, lab.tune, ml.meta_evaluate (tune) and run_meta (tests count); test_optuna_sees_training_rows_only_and_its_trials_are_counted, test_real_optuna_and_vectorbt_screen_training_bars_only_and_resume
• Part F3: Hansen's SPA and StepM over all candidates tested in a discovery run and over the model types of an ML evaluation, stationary-bootstrap confidence intervals of expectancy (R) and Sharpe; shown next to the Bonferroni result with 'what would change'; the promotion gates are not changedskills/lab_worker.py stats / bootstrap_interval (arch 8.0.0), lab.discovery_stats / loss_stats / stats_job, ml.evaluate(stats) and run_meta; test_spa_stepm_and_the_bootstrap_on_cases_with_a_known_answer, test_the_promotion_gates_are_unchanged
• Part F4: skill features-tsfresh (research_only) - tsfresh features on rolling windows of 64 closed M5, M15 and H1 bars; the columns pass the causal harness and the whitelist is stored with its version; relevance filtering fitted on training folds onlyskills/catalogue.py FeaturesTsfresh, skills/lab_worker.py tsfresh_causal / tsfresh_rows / relevance, lab.tsfresh_rows / relevance; test_the_tsfresh_whitelist_passes_the_causal_harness_and_the_relevance_filter_finds_the_signal, test_the_tsfresh_relevance_filter_is_fitted_on_each_folds_training_rows_only
• Part F4: the features-catalogue whitelist and the tsfresh whitelist are candidate features of the discovery search (one more filter cut at the training median; a rule with a feature filter gives no live signal) and of the ML research; scheduling of features-catalogue decided: computed inside discovery and ML runs, on demand, no schedulelab.feature_matrix / catalogue_rows / event_features / persist_features, discovery.feature_candidates and Table.mask, ml.meta_frames (meta-features-2); test_feature_candidates_use_a_training_median_and_keep_the_rule_reproducible, test_tsfresh_rows_are_cached_per_year_and_asked_once
• Part F5: SHAP for every LightGBM and logistic model in the registry - global importance and per-prediction explanations of the meta-label probabilities; leakage flags for a dominant feature, a time / date / row identifier name, or values strictly rising or falling with time; shown on the ML pageml.shap_job, skills/lab_worker.py shap_report / leakage_flags (shap 0.54.1), lab_reports kind 'shap'; test_shap_flags_a_planted_leak_and_a_row_identifier_for_both_model_types
• Part F6: offline PELT regimes (ruptures) of the Dukascopy H1 volatility and trend; results of S1-S4 and the discovered strategies per regime as a research report only - never live and never an AI-trader inputlab.regimes_job, skills/lab_worker.py regimes (ruptures 1.1.10), lab_reports kind 'regimes'; test_ruptures_regimes_are_a_research_report_and_never_reach_the_ai_trader
• Part F8: running - no new task: with AIGOLD_LAB_SIZE=full the existing AI Gold Strategy Discovery task (scripts/run_task.py --capped) runs the lab stage inside its schedule and budget, resumable; ML evaluations, SPA / StepM, tsfresh and ruptures reports on demand from the dashboard as capped background tasks; install.ps1, release.ps1, server.env.example and docs/DEPLOYMENT.md 9c updated; nothing left running on the PClab.start / run_job / POST /lab/jobs/{job}, python -m backend.research.lab run JOB, python -m backend.research.ml meta|shap; deploy/windows/*; test_the_scheduled_discovery_is_the_old_engine_unless_the_machine_is_set_to_the_full_size, test_the_lab_api_lists_jobs_and_starts_one_as_a_background_task, test_a_lock_left_by_a_killed_job_is_cleared_and_the_run_marked_failed, test_a_cache_write_that_is_stopped_leaves_the_readable_file_in_place
• Part F9: dashboard, no new page - Backtesting Lab (research lab panel: screening counts, Optuna trial counts, survivors, the S3 cross-check, job buttons with the free-memory state; SPA / StepM and bootstrap intervals next to each discovery result), Strategy Registry (results per regime; ML page: Optuna-tuned LightGBM vs the Part E fixed version vs logistic vs take every setup, trial counts, SHAP views and leakage flags), Skills page (tsfresh whitelist and the features-catalogue connection)src/components/research-lab.tsx, discovery-panel.tsx, research-meta.tsx, skills-center.tsx, analytics-dashboard.tsx; GET /lab, POST /lab/jobs/{job}; tests/work-order11.spec.ts
• Part F10: docs/SKILLS_LAYER.md (Part F), docs/THIRD_PARTY.md, docs/RESEARCH_GUIDE.md, docs/BACKTESTING_RULES.md, docs/DEPLOYMENT.md 9c, DECISIONS.md, CHANGELOG 0.33.0, PROJECT_STATUS.md, this record; the answered 'Owner questions of 0.32.0' blocker removedfiles in this release
• Part F full-size runs on the VPS (the screening, the lab triggers' tick events, the feature caches, then the on-demand jobs): not run on the PC by the owner's rule (smoke sizes only)owner: docs/DEPLOYMENT.md 9c; the smoke results are in the tests table
• 0.33.1 (owner answers to the Part F questions): Bonferroni gate unchanged; one count for lab and engine candidates with the StepM decision shown beside the Bonferroni decision on every discovery result; M5, M15 and H1 caches kept; fix - the lab stage checks its time budget before every chunk inside a yearDECISIONS.md 2026-11-11 (answers); lab.tsfresh_rows (FEATURE_CHUNK, OutOfTime), lab.discovery_stage; discovery-panel.tsx; test_the_time_budget_is_checked_before_every_chunk_inside_a_year_and_the_next_run_continues; tests/work-order11.spec.ts
• 0.33.2 (owner answers after 0.33.1): StepM over the full family of the Bonferroni count - every screened candidate run by vectorbt on the test period for this comparison only, inside the research worker after the search is finished; each discovery result shows StepM over the full family, StepM over finalists only and Bonferroni (count N); gates unchanged; the approved Playwright teardown fix also in the work-order7 and work-order8 specsDECISIONS.md 2026-11-11 (0.33.2); skills/lab_worker.py family_losses / stepm_family / sweep_trades, lab.stepm_family_payload / discovery_stats; discovery-panel.tsx; test_the_full_family_stepm_uses_every_screened_candidate_and_its_test_returns_never_reach_selection, test_real_full_family_stepm_runs_every_screened_candidate_on_the_test_period_and_returns_only_the_result; tests/work-order11.spec.ts
• G1: one shared chart component on TradingView lightweight-charts 5.3.0 used on Overview, Live Gold and Market Structure, with every feature of the old hand-drawn chart (23-row parity list), demo AI entries / SL / TP / exits and AI decisions as markers and price lines, the TradingView attribution logo; the old SVG chart code removed after the parity tests passedsrc/components/gold-chart.tsx, structure-chart.tsx; docs/SKILLS_LAYER.md Part G (parity list with the test of each row); tests/work-order12.spec.ts (7 chart tests, desktop and phone), tests/work-order.spec.ts, tests/research.spec.ts
• G1: the two official lightweight-charts agent skills copied into .claude/skills.claude/skills/lightweight-charts, lightweight-charts-plugin-authoring (tradingview/lightweight-charts b4b036e, Apache-2.0); SOURCES.md
• G2: structure overlays from our skills drawn only with the renderer primitives of lightweight-charts-indicators 0.20.3 (boxes, lines, labels, background zones): smc order blocks and FVG, sessions and kill zones, auction POC/VAH/VAL/HVN/LVN, trend lines, previous day/week high-low; each a toggle with its skill, version and as_of; nothing computed in the browserGET /skills/overlays (skills/registry.chart_overlays); smc.windows (session_windows, kill_zone_windows, left out of the AI input); test_chart_overlays_return_the_latest_stored_result_per_skill_and_null_without_one, test_chart_only_outputs_never_reach_the_ai_trader_input, test_smc_reports_session_and_kill_zone_windows_with_the_range_of_their_bars; work-order12.spec.ts G2 tests
• G3: Telegram trade images for a trade open and close, rendered from the G1 chart (headless, chart only) with the mplfinance / matplotlib fallback; made without a bot token, shown in the Telegram Center preview and the outbox, attached when a destination existsmigration 040 telegram_images; telegram.request_image / render_images / fallback_chart; scripts/chart-shot.mjs; trade-images.tsx; test_trade_images_are_requested_without_any_bot_and_rendered_by_the_worker, test_the_fallback_image_needs_the_three_levels_and_bars, test_a_stored_image_is_sent_and_a_slow_chart_falls_back_at_delivery; work-order12.spec.ts G3
• G4: weekly and monthly QuantStats tear sheets of the demo AI (test trades excluded), overall and per strategy: Sharpe, Sortino, drawdowns, monthly table, streaks, Monte Carlo; on Reports, in the weekly / monthly report and in the Telegram outbox; shown only, not in the confidence scorebackend/research/tearsheets.py, skills/tearsheet_worker.py (quantstats 0.0.86, research environment), migration 040 performance_reports, performance-reports.tsx; test_tear_sheets_exclude_test_trades_are_stored_and_reported, test_weekly_report_carries_the_tear_sheet_status, test_quantstats_worker_computes_the_statistics_and_the_html_sheet; work-order12.spec.ts G4
• G5: aiomql 4.1.2 async wrapper with automatic reconnection for the READ side of the MT5 adapter (bars, quote, symbol info, account, positions, pending orders, deals, history import, read-only sync); orders keep the existing code path; bridge state in Accountsbackend/research/mt5_bridge.py; brokers.Mt5Adapter.read, terminal._module, mt5_history.Terminal; test_adapter_reads_give_the_same_answers_through_the_read_bridge, test_reads_reconnect_after_the_terminal_was_closed_and_reopened, test_a_lost_connection_is_restored_inside_one_read, test_a_closed_terminal_is_never_started_by_a_reconnect, test_the_order_path_does_not_use_the_read_bridge; the unchanged broker tests; work-order12.spec.ts G5
• G6: read-only MT5 MCP server for research chats (mcp-metatrader5-server 0.1.8) in its own environment, stdio only, order tools not exposedbackend/research/mt5_mcp.py (allowlist of 21 read tools), backend/requirements-mcp.txt, docs/MT5_MCP.md, backend/tests/test_mt5_mcp.py
• G7: lot-size cross-check with EarnForex/PositionSizer's formula on 13 cases; our result correct on all; no code changedocs/LOT_SIZE_CHECK.md, backend/tests/test_lot_size_check.py
• G8: coding-agent skills from anthropics/skills (skill-creator, mcp-builder, claude-api, webapp-testing) and tradermonty/claude-trading-skills (mt5-robot-tester, skill-designer, skill-integration-tester, dual-axis-skill-reviewer); every skill's source, commit and licence recorded; vectorbt-backtesting-skills listed as a reading reference only.claude/skills/SOURCES.md, docs/THIRD_PARTY.md (work order #12)
• G8: ALAGENT-HKU/x2strategy as an optional skill - dropped by owner decisionOwner answer 2026-11-11 (DECISIONS.md, 0.34.1): dropped; the repository has no LICENSE file and the skill was optional
• G9: documentationdocs/SKILLS_LAYER.md Part G, docs/THIRD_PARTY.md, docs/MT5_MCP.md, docs/LOT_SIZE_CHECK.md, docs/DEPLOYMENT.md 9d, DECISIONS.md 2026-11-11 (work order #12), CHANGELOG 0.34.0, PROJECT_STATUS.md
• 0.34.1 (owner answers to the Part G questions): Monte Carlo ruin level = Risk Center max drawdown pause, shown with each report; scheduled task AI Gold Monthly Report with the monthly tear sheets; smc 1.1.0 (research_only); MCP server approved with its read-only extra tools; close-trade notify change approved; x2strategy droppedDECISIONS.md 2026-11-11 (0.34.1); tearsheets.ruin_level; deploy/windows/install.ps1 (MonthlyAt, AI Gold Monthly Report); skills/structure.py SmcSkill 1.1.0; test_the_ruin_level_follows_the_risk_center_max_drawdown_pause, test_smc_1_1_0_registers_over_1_0_0_and_keeps_research_only; tests/work-order12.spec.ts
• 0.34.2 (owner work order #13): AI Gold Monthly Report registers on Windows - the monthly trigger (last day of every month 23:59) is set through the task XML instead of MSFT_TaskMonthlyTrigger, which Task Scheduler rejected (HRESULT 0x80070057) and which stopped install.ps1 before AI Gold Daily Backup and AI Gold Weekly Market Backupdeploy/windows/install.ps1 Set-LastDayOfMonthTrigger; verified on this PC with a temporary test task (Export-ScheduledTask: ScheduleByMonth, Day Last, 12 months, 23:59; next run 2026-11-30 23:59; registered twice; deleted); existing AI Gold tasks untouched

Work

• Part A - Skills foundation + indicators: skill contract and registry, causal harness, two Python environments, indicators-talib, indicators-live, features-catalogue, AI trader input plumbing (OFF), Skills page, documentation (0.28.0: done).
• Part B - Market structure: smc (causal adapter), auction (MarketProfile + OrderFlow-Analysis-Pro port), levels-trendlines (trendln), rules-stolgo (grammar, presets, S3 cross-check), structure worker, reports on the Skills page (0.29.0: done).
• Part C - Macro and news: macro (FRED/ALFRED point-in-time + Yahoo daily), news (official feeds, Google News, GDELT, full text, finBERT), fed (FOMC-RoBERTa Fed index, FOMC history, event study), cot (Legacy history gap), OpenBB evaluation, tasks, dashboard (0.30.0: done; Fed scoring waits for the owner's access to the gated model).
• Part D - AI trader upgrades: trader v5 flow (falsifier, horizon, WAIT, bull/bear debate, Challenger, budget check, decision log with automatic scores), odds-timemachine, knowledge, cot-contrarian, news-reaction-failure, macro-regime, drawdown-breaker and discipline-gate (advisory), post-trade reviews (0.31.0: done).
• Part E - Risk and ML: skills volatility (GARCH), regime (HMM, filtered probabilities), forecast (Chronos-Bolt small, Chronos-2 small, Kronos mini / small) and drift (ADWIN, Page-Hinkley), each research_only with a walk-forward evaluation on the Dukascopy history; LightGBM as a second model type of the ML research with meta-labeling of every strategy's setups under the same registry gates (0.32.0: done; the forecast evaluation is incomplete - H1 done, H4 not done - and TimesFM 2.5 was removed by owner decision).
• Part F - Research lab: vectorbt + Optuna screening of the stolgo rule presets before the discovery search (survivors go to our tick engine; every screened candidate counts in the Bonferroni count), Optuna-tuned LightGBM, Hansen's SPA / StepM and stationary-bootstrap intervals next to the Bonferroni result (gates unchanged), the tsfresh feature factory and the features-catalogue connection, SHAP with leakage flags, offline PELT regimes for research reports, a free-memory check before every heavy job (0.33.0: done; the full-size runs are the owner's on the VPS).
• Part G - Charts, reports and MT5: one shared lightweight-charts chart on Overview, Live Gold and Market Structure (full parity with the old SVG chart, demo AI trades and decisions on it), structure overlays drawn from our skills with the lightweight-charts-indicators renderer, Telegram trade images (dashboard chart, matplotlib fallback), QuantStats tear sheets, the aiomql MT5 read bridge, a read-only MT5 MCP server, the lot-size cross-check and the coding-agent skills. Built in 0.34.0; x2strategy not copied (no LICENSE file).

Blockers / decisions needed

• Forecast evaluation incomplete: H1 done, H4 not done. It continues from its checkpoint (data\skills\forecast) when run again on the VPS (Skills page, forecast card).
• Owner: FRED key and access to the gated FOMC-RoBERTa (Part C); for news-reaction-failure the per-event direction map (empty); a second provider with a working key AND prices for its model in AI Brain (the Groq and Gemini models have no price, so the Challenger uses claude-sonnet-5-5 of the trader's provider).
• Owner: the news window SL/TP limits in the Risk Center are empty, so every demo OPEN inside the USD news window waits.
• Owner: open the MT5 terminal on this PC (closed on 2026-11-10: the demo AI cannot trade and the automatic sync reports 'unavailable'); start the indicators-live, structure, news, reviews, risk, forecast and drift workers when wanted; on the VPS the 'AI Gold ...' tasks run them.
• Part F: the full-size lab runs are the owner's on the VPS (AIGOLD_LAB_SIZE=full, docs/DEPLOYMENT.md 9c): the screening, the feature caches (about 112,000 engine events on three timeframes: many discovery cycles), then the on-demand jobs. On the PC only smoke sizes ran; at smoke size no stolgo candidate passed the screening.
• Owner to decide after the full-size VPS runs: whether StepM replaces or joins the Bonferroni gate (kept as it is now; lab and engine candidates in one count).
• Open item (owner 2026-11-11, not built): discovered rules that use features-catalogue or tsfresh features cannot trade on the demo until a live feature feed exists; such a rule gives no signal.
• Part G on this PC: the skill overlays have no stored result until the structure worker runs (VPS task AI Gold Skills Structure).