Reinforcement Learning

v0.1.7

Q-Learning, SARSA and DQN agents over the GridWorld environment: tabular RL works out of the box, deep RL uses Keras via the dl extra.

Quick Start

qlearning.py

python

import synaptix as sx

env = sx.reinforcement.GridWorld(size=5)      # 5x5 grid, stochastic slips
agent = sx.reinforcement.QLearningAgent(
    n_states=env.n_states,
    n_actions=env.n_actions,
    alpha=0.1,
    gamma=0.95,
    epsilon=0.1,
)

# Train
for episode in range(500):
    state = env.reset()
    done = False
    while not done:
        action = agent.act(state)
        next_state, reward, done = env.step(action)
        agent.learn(state, action, reward, next_state)
        state = next_state

print("Learned policy sample:", agent.policy[:5])

Agents & Environment

Component	Description
GridWorld	2D grid environment with walls, slippery cells, rewards and terminal states.
QLearningAgent	Off-policy tabular agent — learns the optimal action-value function.
SARSAAgent	On-policy tabular agent — updates with the action actually taken.
DQNAgent	Deep Q-Network over Keras (requires synaptix[dl]) for large state spaces.

SARSA Variant

sarsa.py

python

agent = sx.reinforcement.SARSAAgent(
    n_states=env.n_states,
    n_actions=env.n_actions,
    alpha=0.1,
    gamma=0.95,
    epsilon=0.1,
)

for episode in range(300):
    state = env.reset()
    action = agent.act(state)
    done = False
    while not done:
        next_state, reward, done = env.step(action)
        next_action = agent.act(next_state)
        agent.learn(state, action, reward, next_state, next_action)
        state, action = next_state, next_action