NITANSH THAKERNT / SYSTEMS
Research Intern
PHAETHON Centre of Excellence

Reinforcement Learning for Local Energy Markets

Simulation and evaluation of constrained battery and market decisions across transparent baselines and learning-based control.

All work
PHAETHON reinforcement-learning decision loopA diagrammatic reinforcement-learning decision loop for PHAETHON local-energy-market research. Current observations and separate policy constraints inform a constrained policy, which selects one eligible action. The battery and grid environment processes that action, produces a system response, and updates the next observation for the following decision step.OBSERVATIONSCURRENT STATEPV GENERATIONDEMAND / LOADMARKET SIGNALBATTERY SOCCONSTRAINTSSOC BOUNDSACTIONELIGIBILITYCONSTRAINEDPOLICYELIGIBLEACTIONSYSTEM /ENVIRONMENTBATTERY + GRIDSYSTEMRESPONSESTATE TRANSITIONNEXT STATEUPDATEDOBSERVATION
Scroll to inspect the full diagram →
State / policy / constrained actionArchitecture abstraction

The problem

Residential prosumers have to decide how battery energy moves between local demand, photovoltaic production, and the grid while respecting physical operating constraints and changing prices.

System architecture

  • A Python environment modelling photovoltaic generation, demand, battery storage, grid import/export, and time-varying prices.
  • Comparable rule-based, tabular Q-learning, Expected SARSA, and Dueling DQN control paths.
  • Explicit state representations, discrete action spaces, reward functions, exploration behaviour, and terminal handling.
  • Extensions toward price–quantity bidding and local-market interaction.

Engineering decisions

  • Retain a transparent rule-based policy as a behavioural and economic reference.
  • Represent battery state-of-charge and operational constraints inside the decision environment.
  • Treat reproducibility, policy behaviour, and limitations as first-class evaluation outputs.

Testing and verification

  • Simulation outputs and analysis compare economic and operational behaviour against the baseline.
  • Deterministic evaluation pathways are separated from exploratory learning runs.
  • Constraint handling is inspected alongside aggregate behaviour.

Limitations

  • The work is simulation and evaluation, not deployment on a physical robot or live market.
  • It is not represented as federated learning or independently controlled multi-agent reinforcement learning.
  • Unpublished numerical results are intentionally omitted.

What I would improve next

  • Continue studying bidding behaviour under uncertain production, consumption, and prices.
  • Strengthen policy evaluation across seeds, scenarios, and operational edge cases.