Skip to content

W4 - ns3-ai-ntn

Reinforcement-learning agents training in-the-loop
Reinforcement-learning agents training in-the-loop
Shared-memory IPC between ns-3 and Python
Shared-memory IPC between ns-3 and Python
ns3-ai-ntn module architecture
ns3-ai-ntn module architecture

ns3-ai-ntn is the ns3-ntn-toolkit module that modernizes the ns3-ai shared-memory bridge for ns-3.43, Python 3.13, NumPy 2.0 and Gymnasium 1.0, and adds NTN-specific reinforcement learning environments plus an AI-RAN inference contract for satellite and terrestrial networks.

Why it matters. Upstream ns3-ai stopped tracking the modern Python and ns-3 stack: it crashes on NumPy 2.0, fails to import on Python 3.13, and loses pybind11 symbols under ns-3.43's link-time optimization. This fork repairs the bridge and ships four NTN Gymnasium environments and an AI-RAN inference contract, so researchers can train reinforcement learning agents for LEO handover, beam management, slicing and power control directly against ns-3 simulations.

What it simulates

The module pairs a modernized C++ to Python bridge with four NTN reinforcement learning environments:

  • Shared-memory and gym bridges. A Boost.Interprocess shared-memory ring buffer (struct and vector message variants via Ns3AiMsgInterface) and a Gymnasium-API gym interface (OpenGymInterface, OpenGymEnv, with Box, Discrete, Tuple and Dict spaces) for in-the-loop training.
  • AI-RAN inference contract. A gRPC inference path (AiranInferenceClient, AiranInferenceServer, TcpInferenceListener) with a Triton config contract and a deterministic mock runtime, supporting CSI-to-precoder and RSRP-to-beam model classes with Doppler hints for LEO and HAPS.
  • NTN RL tooling. Stable-Baselines3 PPO training, PyTorch Geometric GAT for constellation-graph next-hop learning, and MAPPO/MASAC multi-agent baselines under python_utils/ns3_ai_ntn.
Env Action space Observation Reward
HandoverEnv Discrete (stay or one of N candidate cells) per-cell RSRP · SINR · total TA · TA drift · remaining pass time throughput proxy minus handover and ping-pong cost
BeamMgmtEnv Discrete (beam index over N beams) UE position · per-beam range normalized SNR
SliceEnv Box (PRB share simplex over eMBB/URLLC/mMTC) per-slice demand · satisfaction history · utilization weighted satisfaction minus URLLC latency-tail penalty
PowerCtrlEnv Box (TX power in [-40, +23] dBm) path loss · slow fading · recent BLER · last SNR throughput proxy minus power cost

What ships

ns3_ai_ntn/
├── envs/                       # 4 Gymnasium envs
├── sb3/train_ppo_handover.py   # canonical SB3 trainer
├── gnn/
│   ├── constellation_graph.py  # PyG Data from W1 ISL graph
│   └── gat_topology.py         # GAT for next-hop prediction
├── marl/
│   ├── mappo_handover.py       # multi-UE MAPPO baseline
│   └── masac_beam.py
└── ns3gym_compat.py            # adaptor to canonical ns3-gym API

Validation gates

  • All 4 envs pass gymnasium.utils.env_checker.check_env
  • pytest contrib/ns3-ai-ntn/python_utils/tests/ - 15/15 in 3.1 s
  • PPO baseline beats random on HandoverEnv (gap 116.1 vs σ 27.5 - > 4 σ)
  • GNN converges to 88 % handover-prediction accuracy on Starlink subset (gate ≥ 70 %)

Standards & references

  • O-RAN - canonical slice ordering (eMBB, URLLC, mMTC) and SST mapping used in SliceEnv, plus the AI-RAN inference contract.
  • 3GPP NR FR1 - 100 MHz at 30 kHz subcarrier spacing (273 PRBs) as the SliceEnv default.
  • 3GPP NTN - timing-advance geometry and the [-40, +23] dBm NR UE power class in PowerCtrlEnv.
  • Gymnasium 1.0 - the reinforcement learning environment API all four NTN environments implement.

Use cases

  • Learned LEO handover. Training a policy that weighs RSRP, SINR and timing advance to cut handover failures and ping-pong during a satellite pass.
  • Beam selection. Learning which beam on a Walker shell to serve a moving UE for best SNR.
  • RL slice orchestration. Producing PRB shares that feed the ntn-slice orchestrator via measured satisfaction.
  • Uplink power control. Optimizing NTN UE transmit power against path loss, fading and BLER.
  • AI-RAN inference. Driving CSI-to-precoder or RSRP-to-beam inference through the gRPC contract with a Triton-compatible runtime.

Run it

pip install -e contrib/ns3-ai-ntn/python_utils

python -m ns3_ai_ntn.sb3.train_ppo_handover \
    --total-timesteps 100000 \
    --influx-host localhost

Source