Microgrid Peer-to-Peer Energy Settlement

A transactive-energy clearing and settlement engine that runs a uniform double auction across a microgrid every 15 minutes, then books every trade through a fixed-point double-entry ledger that provably cannot leak a cent.

---

Overview

A neighbourhood-scale microgrid is a financial market. Solar prosumers sell surplus into a shared feeder; households buy it at whatever moment they need it; the distribution network charges for the privilege of using the wire; and whatever the local market cannot cover is bought from — or sold to — the utility grid.

This engine is the accounting spine for that market. It implements the Transactive Energy pattern end to end:

1. Clear — aggregate every resting bid and ask, find the single uniform price where the curves cross, and allocate volume pro-rata at the marginal level.
2. Reconcile — compare what each node contracted against what its smart meter actually measured, and flag under-delivery.
3. Settle — post every trade into a double-entry ledger whose postings must sum to exactly zero, then commit atomically to PostgreSQL.
4. Fall back — route residual shortfall to the utility at the retail tariff and residual surplus at the feed-in tariff.

The hard part of distributed energy settlement is not the matching — it is that money must not evaporate. A system that settles 50 nodes every 15 minutes for 96 windows a day will lose cents to floating-point drift, rounding order, and partial failure within a week. This one is built so it cannot: every quantity is decimal.Decimal quantised to 1 Wh / 1 micro-unit, and every posting group is rejected before it reaches the database unless debits equal credits to the last digit.

---

🎯 Key Features

• ⚖️ Uniform Double-Action Clearing — bids ranked descending, asks ascending; the clearing price is the midpoint of the marginal bid and marginal ask, and every trade in the window settles at that one price.
• 🔒 Zero-Loss Fixed-Point Settlement — binary floating point is banned from all balance tables. PostingGroup.total must equal 0.000000 or the write is refused by assert_balanced.
• 🧾 Physical-to-Financial Reconciliation — contracted position vs. metered net injection per node, with a 5% imbalance tolerance; only the short side of a variance is a breach, since surplus is simply exported.
• 🔌 Grid Fallback Rail — residual deficit buys at retail (0.18), residual surplus sells at feed-in (0.075). Residual only — energy already matched to a peer is never settled twice.
• 💸 Wheeling Fees to the MDO — network loss (4%) is applied before the tariff (0.035/kWh), and the fee is routed to the Microgrid Distribution Operator as network revenue, never back to the paying buyer.
• 🕐 Validated 15-Minute Telemetry — meter ticks are decoded from Redis Streams, checked for quarter-hour boundary alignment within 90s clock drift, bounded for physical plausibility, and de-duplicated by (node_id, interval_start, source_event_id).
• ⚙️ Operable Without a Database — the API, clearing daemon, and simulator all degrade to in-process state when PostgreSQL is unreachable, so the whole engine is demoable in one command.

---

🏗️ System Architecture

Diagram source: docs/architecture.md — every node and edge annotated with its file:line evidence.

Component Breakdown

Solid edges are wired runtime paths. Dashed edges are implemented and reachable but not yet on the default runtime path — they are called out honestly below.

Trust & Persistence Boundaries

1. Telemetry is untrusted. Raw stream payloads are decoded and validated before persistence. Boundary misalignment beyond 90s, non-finite or negative volumes, and throughput above the 1000 kWh plausibility ceiling all cause a quarantine, not a silent write.
2. Money is never recomputed from float. Volumes arrive as JSON strings and become Decimal. Every monetary column is NUMERIC(18,6); every Python amount is quantised to 0.000001.
3. Postings are all-or-nothing. persist_matches() opens one ACID transaction covering trade rows, journal entries, and derived cash balances. A leak is detected before the transaction opens.
4. The ledger is append-only. microgrid_nodes.cash_balance is a derived projection updated by delta; ledger_entries is the source of truth, indexed by posting and window.

Known Gap (deliberately visible)

cmd/market_clearing_engine/main.py:52 loads open orders, clears them, and logs the result — it does not yet call SettlementRepository.persist_matches(). The atomic PostgreSQL settlement path is fully implemented (internal/repository/settlement_repo.py:55) and exercised by the database integration fixtures, but the daemon does not invoke it on its default loop. That is the dashed LEDGER → DB edge above. The end-to-end path that is wired today is simulator/microgrid_simulator.py → build_in_memory_ledger().

---

🔁 The 15-Minute Settlement Window

Diagram source: docs/settlement-window.md — every message annotated with its call site.

Step by Step

1. Order entry — A participant submits a BID or ASK with quantity_kwh and limit_price_per_kwh. Pydantic rejects non-finite decimals, quantities ≤ 0, quantities above the 1000 kWh interval ceiling, and non-UUID node ids. The order is parked in the next window's book (current_window().next_window()).
2. Window opens — ClearingWindow enforces a timezone-aware, minute-aligned, minute % 15 == 0 start. The daemon (--interval, default 900s) walks to current_window().next_window() on a fixed cadence.
3. Priority ordering — Bids sort by price descending, asks ascending; heapq gives O(log n) best-price access. Order.with_fill() recomputes FILLED / PARTIALLY_FILLED immutably, so fill state is never mutated in place.
4. Clearing — determine_market_clearing() scans every candidate price and takes max(min(demand@P, supply@P)) as Q*. If the curves never overlap, NoCrossingError degrades to an empty result rather than a bogus price.
5. Uniform price — The marginal price is the last level consumed by the allocation walk on each side; the clearing price is their midpoint, asserted to sit between them.
6. Pro-rata allocation — _pro_rata() floors each share to the 1 Wh quantum, then hands out leftover quantum-steps in price-time priority so the level's total always equals the requested volume exactly. _pair() FIFO-walks both sides' allocations.
7. Posting construction — Per match, four entries: buyer ENERGY_CHARGE_DEBIT, seller ENERGY_REVENUE_CREDIT, buyer WHEELING_DEBIT, MDO WHEELING_REVENUE_CREDIT. The buyer pays energy + wheeling; the seller receives only the energy charge.
8. Balance gate — assert_balanced() requires PostingGroup.total == 0.000000 and raises UnbalancedPostingError otherwise. This runs before the transaction opens.
9. Atomic commit — Trade rows, journal entries, and the cash_balance delta commit in one ACID transaction (internal/repository/settlement_repo.py:76).
10. Reconciliation — contracted_positions() nets sold-minus-bought per node and compares to measured_net_kwh. Nodes with no reading are treated as zero delivery, so an unmetered prosumer surfaces as a breach instead of silently vanishing.
11. Residual routing — residual = measured − contracted keeps a node from being charged twice for the same kWh. route_from_net_position() splits it into retail imports (0.18/kWh) and feed-in exports (0.075/kWh).
12. Statement — StatementBuilder groups journal entries by account and EntryKind into a per-period bill with totals for charges and credits.

---

✅ Verified Results

All figures below are real output from this repository on Python 3.11.

pytest — 133 passed

python -m simulator.microgrid_simulator — 50 nodes × 96 windows

Nodes: 50
Windows cleared: 96
P2P energy traded: 832.426757 kWh
Grid import: 2105.722211 kWh
Grid export: 1631.330399 kWh
Average clearing price: 0.105000
End-of-day trial balance: 0.000000
Grid gateway balance: 256.680222
MDO balance: 27.969545

Counting the same run: 3,087 peer matches across 63 windows with trades, producing 19,282 double-entry ledger entries.

The end-of-day trial balance of 0.000000 is the headline result: 832 kWh of peer trades, 2,106 kWh of grid imports and 1,631 kWh of exports, 19,282 postings — and not one micro-unit created or destroyed. The grid gateway and MDO balances are non-zero because those are genuine external counterparties; that imbalance is accounted for, not lost.

---

🧰 Tech Stack

Standards Alignment

Modelled on the Transactive Energy (TE) framework, with interval semantics conforming to utility settlement periods (IEEE 2030.5 style quarter-hour ticks) and node modelling after CIM/ISO 61968 concepts.

---

📁 Project Structure

.
├── cmd/                                # Executable entry points
│   ├── migrate.py                      # Apply migrations/*.sql (idempotent)
│   ├── market_clearing_engine/
│   │   └── main.py                     # 15-minute clearing daemon
│   └── meter_ingestion_service/
│       └── main.py                     # Redis Streams consumer service
│
├── internal/                           # Application core (packaged)
│   ├── domain/                         # Pure domain model — no I/O
│   │   ├── market.py                   # Order, ClearingWindow, Match, curves
│   │   ├── meter.py                    # MeterInterval + validation rules
│   │   └── ledger.py                   # LedgerEntry, PostingGroup, zero-sum gate
│   ├── auction/                        # Market mechanism
│   │   ├── orderbook.py                # Price-priority heaps, FIFO tie-break
│   │   ├── matching_engine.py          # Uniform double-auction clearing
│   │   └── wheeling.py                 # Network loss + wheeling tariff
│   ├── settlement/                     # Physical ↔ financial resolution
│   │   ├── reconciler.py               # Contracted vs metered imbalance
│   │   └── grid_fallback.py            # Retail import / feed-in export
│   ├── repository/                     # Persistence
│   │   ├── db.py                       # asyncpg pool + migration runner
│   │   ├── meter_repo.py               # Interval upsert + dedupe receipts
│   │   └── settlement_repo.py          # Atomic double-entry settlement
│   └── transport/                      # Edge adapters
│       ├── api/                        # FastAPI: order_routes, account_routes
│       └── stream/                     # meter_consumer.py (Redis Streams)
│
├── migrations/
│   └── 001_init_energy_market_schema.sql   # 6 tables, NUMERIC(18,6) money
│
├── simulator/
│   └── microgrid_simulator.py          # 50 nodes × 96 windows, deterministic
│
├── tests/                              # 133 tests across 7 suites
│
├── docs/                               # Standalone diagram sources
│   ├── architecture.md                 # System architecture (Mermaid)
│   └── settlement-window.md            # 15-minute window lifecycle (Mermaid)
│
├── docker-compose.yml                  # PostgreSQL 16 + Redis 7 + RabbitMQ 3
├── Makefile                            # up / down / migrate / simulate / test
└── pyproject.toml                      # Python 3.11+, deps, strict mypy config

The dependency direction is strictly inward: transport → settlement/auction → domain. internal/domain/ imports nothing from the other layers, which is what makes the money invariants provable in isolation.

---

🚀 Quick Start

Prerequisites

• Python 3.11+
• Docker (for PostgreSQL 16 + Redis 7) — optional, the engine runs fully in-memory without it

1. Clone and install

git clone https://github.com/<your-username>/microgrid-p2p-settlement.git
cd microgrid-p2p-settlement

python -m venv .venv
source .venv/bin/activate        # macOS / Linux
# .venv\Scripts\activate         # Windows PowerShell

pip install -e ".[dev]"

2. Bring up infrastructure

make up        # docker compose up -d, then blocks until pg_isready passes

Without Docker — run completely in-memory

Every entry point degrades gracefully. The API, clearing daemon, and simulator all work with no database at all:

pytest -q
python -m simulator.microgrid_simulator

3. Apply the schema

make migrate
# → Applied migrations:
#     001_init_energy_market_schema.sql

4. Run the test suite

make test       # or: pytest -v

133 passed

5. Run the 50-node simulated day

make simulate

Nodes: 50
Windows cleared: 96
P2P energy traded: 832.426757 kWh
End-of-day trial balance: 0.000000

6. Start the API

uvicorn internal.transport.api.app:app --reload --port 8000

Open http://localhost:8000/docs for interactive OpenAPI.

Exercise the market over HTTP

# Submit a bid into the next clearing window
curl -X POST http://localhost:8000/orders \
  -H "Content-Type: application/json" \
  -d '{
        "node_id": "00000000-0000-0000-0000-000000001000",
        "side": "BID",
        "quantity_kwh": "2.500000",
        "limit_price_per_kwh": "0.160000"
      }'

# Inspect the resting book
curl "http://localhost:8000/orders/book"

# Read a balance and a statement
curl http://localhost:8000/accounts/00000000-0000-0000-0000-000000001000/balance

curl "http://localhost:8000/accounts/00000000-0000-0000-0000-000000001000/statement\
?period_start=2026-03-01T00:00:00Z\
&period_end=2026-03-02T00:00:00Z"

# Liveness
curl http://localhost:8000/health

Quantities and prices are sent as JSON strings so they decode straight into Decimal without ever passing through a binary float.

7. Run the services

# Clear 4 windows on a 5-second cadence (fast demo, ~15s)
python cmd/market_clearing_engine/main.py --interval 5 --iterations 4

# Production cadence: clear every 15 minutes, indefinitely
python cmd/market_clearing_engine/main.py --interval 900

# Clear one specific window and exit
python cmd/market_clearing_engine/main.py --window 2026-03-01T12:00:00Z

# Meter ingestion service (drains the Redis stream)
python cmd/meter_ingestion_service/main.py

Note: the daemon sleeps between windows, so --interval 900 --iterations 4 takes about 45 minutes. Use --interval 5 for a quick demonstration.

Make targets

---

🌐 REST API

Interactive documentation is served at /docs (Swagger) and /redoc.

---

⚙️ Configuration

Core tariff and loss constants live in code and are overridable per instance:

---

🧪 Testing

pytest -v                                     # full suite
pytest tests/test_fixed_point_settlement.py   # money invariants only
pytest --collect-only -q                      # count tests per suite

Database-backed tests use a session-scoped db_pool fixture that runs migrations automatically and skips cleanly when Docker or PostgreSQL is unavailable — which is why the full 133 tests pass on a bare checkout.

make up      # required for the DB integration path
pytest tests/test_phase1_tooling.py -v

---

📐 Design Notes

Why Decimal everywhere. Money in binary floating point does not add up. 0.1 + 0.2 != 0.3, and across the 19,282 postings of a single simulated day that error compounds into visible leakage. Every quantity here is Decimal quantised to Decimal("0.000001") — 1 Wh resolution for energy, 1 micro-unit for money — mirroring the NUMERIC(12,6) / NUMERIC(18,6) column types exactly.

Why assert balance before opening the transaction. assert_balanced() runs in Python before async with connection.transaction(). A leak is a programming error, not a runtime condition, so it should fail in the cheapest possible place rather than after a round trip and a rollback.

Why pro-rata residual distribution is explicit. Flooring shares to the energy quantum leaves a remainder. Handing out the leftover one-quantum steps in price-time priority guarantees the level's allocation equals the requested volume exactly, instead of accumulating dust into an unallocated pool that no longer matches any order.

Why surplus is not an imbalance breach. If a prosumer generates more than it contracted to sell, the extra energy was never promised to anyone — it is exported to the grid at the feed-in tariff. Only the short side of a variance indicates a broken contract, which is why breaches() filters on variance_ratio < -tolerance.

Why the meter ceiling is 1000 kWh. A 100 MW feeder at full duty cycle over 15 minutes is 25 MWh. A 1000 kWh bound is an order of magnitude above any residential or C&I node, so it never rejects a legitimate reading while still quarantining a corrupt meter or a unit error.

---

📄 License

MIT — no LICENSE file is committed yet; add one before treating this as formally released under MIT.

---

Built for engineers who care that the money adds up.

pytest → 133 passed  ·  simulator → trial balance 0.000000  ·  float → banned