This is an early, experimental series. The environments, numbers and conclusions will change as we go.
PortSimEnv now has v2–v5, built by Edgar Arakelyan (Scale AI): a live port where the week unfolds watch by watch, with pilots and tugs, real weather and shipping lines that answer back. See the live port. Bhavish Pahwa built a second job, RailSimEnv, in What the community built. Now Edgar and @agutmanstein, a licensed pilot, have taken the same idea to airline operations with AirOpsEnv: three agents at the ops, crew and customer desks coordinate through a real storm day. Built at Scale AI using AgentEnv Framework, in collaboration with Hugging Face and FineEnvs.
Simulation RL environments
Most RL environments for language models are puzzles, code, maths or web tasks. A lot of valuable work looks different. Someone keeps a schedule running under real rules while things keep going wrong. I want agents that can do that work, which raises a general question. How do we build simulation environments, digital twins of real work, that are realistic enough to learn from?
Each part of this series takes one real job, rebuilds it from the records it leaves behind, and grades the agent against a proven optimum where tractable, or a disclosed best-known reference and lower bound when it is not.
PortSimEnv v1
PortSimEnv is a deterministic RL environment for berth planning at the Port of Barcelona. It is built from the 1,784 container-ship calls the port recorded in 2024 at its two container terminals, with the real ships, their sizes, berthing times and quay sections. The agent gets one quay, the ships that really called there in a 2024 week, and a week that has just gone wrong. It re-plans the week, deciding for every ship when it docks, where along the quay and with how many cranes.
Every plan plays out on a 3D twin of the real port, built from open data. The agent’s berthing hours, quay sections and crane counts turn into ships, tugs and cranes moving through the week.
The disruptions come from what happens in real ports. Red Sea diversions send ships in late and in bunches, gales stop ships of 300 m or more above 25 knots and every ship above 30, a crane goes down for two days, or an emergency needs a berth now.
Try it
This is the environment’s OpenEnv Space, embedded live. Pick a split and a task index, load the episode and plan it with the same three tools the agent gets. The Playground tab is OpenEnv’s MCP playground. The code is in 07-simulation-environments/portsim-v1 on GitHub.
Why it matters
A ship’s containers only move once it is alongside under a crane, so the berth plan decides how long ships wait, how much fuel they burn at anchor and whether a whole service stays on schedule. Schedules rarely hold, so the plan keeps changing.
Barcelona alone handled 3.9 million TEU from 2,163 container-ship calls in 2024, and in 2025 it changed its rules to give berth priority to regular services during congestion. Rotterdam reports that its port-call app cut waiting by 20%.
The eval
Six models each ran the 50 held-out weeks once, with the same three tools, 12 turns and 32k output tokens per turn.
- GPT-6.1 Sol leads at 0.89 and submits a valid plan every week, but it matches the optimum on only 26 of 50, so there is plenty of headroom.
- Every model scores lower on storm and extreme weeks, which stack gales, closures and emergencies.
- The open models lose most of their score before grading. They spend their output budget reasoning and never call
submit_plan. On the 13 to 25 weeks where they do submit a valid plan, they average 0.75 to 0.97, so getting them to submit is the first thing to train.
Watch a rollout
All 300 eval rollouts replay on the 3D quay in the eval Space, FineEnvs/PortSimEnv-Eval, with the plan at each tool call, the dock chart, the grade and the full transcript.
How the reward works
The reward is deterministic, so the same plan always gets the same score. It is computed from the plan alone, without
an LLM judge or sampling, once per episode when the agent calls submit_plan.
An independent checker re-verifies every plan. Across the 1,100 tasks the naive re-plan never scores above 0.44 and a greedy heuristic never above 0.82, because tasks they nearly solve were dropped when the dataset was built.
In the film’s run, GLM-5.3 checks three drafts that come back with 9, 5 and then 0 rule breaks. Its last edits bring
NAGOYA EXPRESS down from eight cranes to its six-crane limit and move CMA CGM MONTREAL from hour 131 to 133. The plan
it submits costs 87: 72 in weighted delay plus three berth changes at 5 each. That equals the proven optimum, so
the gap is zero and the reward is 0.2 + 0.8 × e⁰ = 1.00.
Built on real Port of Barcelona data
Each task keeps a real week at one quay and the ships that really called, adds disruptions modelled on real events, and gets an answer key from CP-SAT. I drop the tasks a simple heuristic nearly solves, which leaves 1,050 training and 50 eval tasks. No week appears in both.
What is real and what is simulated. The records give us the berth plan the port actually ran, with the ships, their sizes, times and quay sections. They don’t include crane assignments, container manifests or time spent waiting at anchor. Workloads, disruptions and the physical handling are simulated, and the film replays an agent’s plan on that simulation.
From seed data to tasks
- Seed: the real calls of one quay for one to three weeks, with the berth plan the port actually executed.
- Fill the gaps: crane fleets, wind and traffic rules, and a crane rate of 28 moves per hour turn each call’s real time alongside into a workload.
- Disturb: inject the disruptions below, scaled by tier (standard, busy, storm, extreme).
- Solve: CP-SAT proves the optimal re-plan, the answer key; the naive and greedy re-plans set the floor.
- Filter: drop tasks a heuristic nearly solves and keep whole weeks out of training for the eval.
| Disruption | What happens | Real-world model | Share of the 1,100 tasks |
|---|---|---|---|
| Late ships | ships arrive hours or days late | Red Sea diversions, half of ships late | 100% |
| Closures | quay sections closed for repairs or dredging | crane-rail repair, fender repair, dredging | 100% |
| Crane outages | cranes out of service for hours or days | breakdowns, maintenance | 89% |
| Priority cargo | some ships count ×3 | transshipment connections | 86% |
| Emergencies | a ship must dock by a deadline | a reefer ship losing power | 76% |
| Extra calls | unscheduled ships need a berth | ad-hoc calls | 63% |
| Gales | no ship moves; ≥300 m stop above 25 kn, all above 30 | the port’s traffic ordinance | 62% |
| Bunching | several ships arrive together | delayed services catching up | 46% |
| Diverted traffic | ships moved from the other terminal | terminal outages | 15% |
Data sources
| Data | Source | Licence |
|---|---|---|
| 2024 ship calls: 1,784 at BEST and APM, with times, sections, ports | Port of Barcelona open data via berth-allocation-problems | CC BY-SA 4.0 |
| Crane fleets: 13 at BEST, 9 at APM | BEST, APM Terminals | published |
| Wind, traffic and berth rules | BOE-A-2023-6719, BOE-A-2025-5157, port-call procedure | public |
| 3D twin: coast, buildings, yards, cranes | OpenStreetMap | ODbL 1.0 |
| Terrain | Terrain Tiles on AWS | open |
| Anchorage and routes (checked) | Copernicus Sentinel-2 | open |
Links
| Environment: the OpenEnv Space, with the editor and 3D view and an MCP playground | FineEnvs/PortSimEnv |
| Eval: all 300 rollouts, replayed in 3D | FineEnvs/PortSimEnv-Eval |
| Dataset: tasks (train, eval), the source calls, the eval rollouts | datasets/FineEnvs/PortSimEnv |
| Bucket: 3D twin data and raw eval rollouts | buckets/FineEnvs/PortSimEnv |
| Collection | Simulation RL Envs |
| Code | github.com/adithya-s-k/FineEnvs: 07-simulation-environments/portsim-v1 |
| Discussion | GitHub discussion #36 |
Contains data from the Port de Barcelona open data portal. The extracted data and the task pack built from it keep the source’s CC BY-SA 4.0 licence.
The live port: PortSimEnv v2–v5
Edgar Arakelyan at Scale AI took PortSimEnv and made the week happen in order. In v1 the agent sees the whole broken week up front and submits one plan. In his versions it starts on Monday at 00:00 knowing only what the port knows then, and the rest of the week reaches it the way it would reach a real planner. He built four versions on AgentEnv, Scale’s framework for agent environments, each adding another part of the real port. All four keep v1’s quay, weeks, rules checker and reward, so a plan means the same thing in every version. The code is in 07-simulation-environments/portsim-live.
A week in watches
At hour 0 the agent knows the published arrivals and the planned closures. Every other disruption reaches it as a message from whoever would send it: the ship, the harbour master, terminal ops or the line desk. Each message arrives 24 hours before the hour it is about, or 12 hours for an emergency. Messages come in 6-hour bulletins, and each bulletin with news is a watch, between 2 and 9 in a week.
At each watch the agent has four tools:
| tool | what it does |
|---|---|
get_situation() | the watch, its time, the week as known now, the confirmed windows and the ships without one |
check_plan(plan) | rule breaks and cost of a draft, merged over the confirmed windows; confirms nothing |
confirm_berths(plan) | sets or changes windows; each refusal gives the harbour master’s reason |
advance() | ends the watch; after the last one the rest of the week runs and is graded |
- The freeze. A window that starts within 6 hours of the current watch can’t change, so late news can land on a decision that is already locked in.
- A budget.
check_planandconfirm_berthsshare 3 calls a watch. - No time passes while the model thinks. AgentEnv’s gateway keeps a virtual clock. Each
advancemoves it to the next bulletin, and one trigger per watch delivers that bulletin through a tool the agent can’t see. - One grade, at the end. The confirmed windows run against the week as it really happened, scored with v1’s reward. News that lands on an already frozen window is excused. A ship the agent moves into trouble afterwards is not.
v3: pilots and tugs
Every berthing and departure needs a pilot, and bigger ships need tugs. In v3 both come from port-wide pools that the quay shares with the rest of the port’s real 2024 traffic: ferries, tankers, car carriers, cruise ships and container ships at other quays. Edgar set the pools at 7 pilots and 8 tugs, calibrated against that traffic, and the tug company and the pilot station announce cuts.
A plan that worked in v2 can now break. Two ships moving in the same hour may need more tugs than are free, especially inside a wind window, where every move takes one more tug.
v4: real wind, real forecasts
v1 and v2 simulate their gales. v4 uses real weather instead. Each week is moved into a real Barcelona weather week from 2023 to 2025, picked for its storms or for forecast gales that never came. At every watch Port Control sends the latest ECMWF forecast as it was actually issued, adjusted to the port’s anemometer. The agent plans on the forecast, and the grade uses the wind that really blew.
The agent can only know the forecast, so a wind week scores 1.0 at its anchor: the cheaper of the plan in hindsight and a re-planner that follows each forecast as it comes. The 15 weeks are picked so that this re-planner reaches the anchor, while one that ignores the wind scores 0.9 or less. Reading the forecast is what scores.
v5: the shipping lines answer back
In v5 the schedule is only an estimate. Each ship has its agent, the firm the port’s 2024 records name for that call,
running as an AI agent of its own. It reports an ETA 48 and 24 hours out and calls an hour before the ship arrives,
and its ETAs are optimistic. The planner gets a fifth tool, message_agent, to ask a ship for its latest ETA or agree
an arrival hour. Ships with an agreed arrival berth first come, first served. The port berths a late ship wherever
there’s room, and every hour at anchor costs as much as an hour late.
The ships’ agents have no model of their own. They answer from the environment, so the same request always gets the same answer and a week replays exactly.
The eval
Twelve models played 15 weeks of each version, 720 runs in all, graded by the same code.
- Each version is harder. Eight of the twelve models score lower at each step from live to marine to wind, and 11 of 12 score lower on the lines weeks than on the same weeks without the ships’ agents.
- On v5, talking to the ships is what scores. A re-planner that agrees arrival times with the ships’ agents reaches 0.55, and the same planner without asking reaches 0.24. Claude Opus 5.5, GPT-6 Astra and GPT-6.1 Sol agree an arrival for 86% to 90% of a week’s ships.
- Claude Opus 5.5 and GPT-6 Astra lead over all 60 weeks, at 0.857 and 0.851. Qwen3.8-27B, the one model small enough to fine-tune cheaply, finds a valid plan in 1 of 60.
Watch a run
Every run replays in Edgar’s Space on the same 3D quay as v1. Here is week 7 at APM Terminals in each version.
Every run is in the dataset earakely-scale/PortSimEnv-AgentEnv, and the agentenv-portsim plugin has the command line, the sweeps and the Hub tooling.
AirOpsEnv: three desks, one storm day
A flight can have an aircraft and still be unable to leave. Its crew might run out of legal duty time. Waiting a few minutes could save a passenger’s last connection, but delay everyone already on board and the aircraft’s next flight. Then another weather advisory changes the plan.
AirOpsEnv takes FineEnvs from ports to airline operations, in collaboration with Scale AI (@scale_ai) and Hugging Face. It was developed by Edgar Arakelyan (@earakely) with @agutmanstein, a licensed pilot, using Scale AI’s AgentEnv Framework.
The simulation is grounded in real flight records for New York JFK, Boston Logan and the wider JetBlue network, using US Bureau of Transportation Statistics data, FAA traffic-management advisories and NWS weather reports. In v3, three AI agents run the operations, crew and customer desks. They have different tools and must ask each other for changes. The environment executes the decisions and grades the resulting day.
Try the interactive replay · AgentEnv Framework · Environment code in FineEnvs · Source and reproducibility instructions · Pinned data and recorded runs
What the agents actually do
The ops controller changes holds, cancellations and arrival-slot choices. The crew scheduler assigns legal crews, calls reserves and moves crew members between stations. The customer desk identifies endangered connections and asks operations to hold or protect flights. A customer request does not itself change a departure: operations has to accept it. Passenger rebooking is automatic in the simulator; the agents do not directly rebook individual travellers.
Each watch reveals the information available at that point. The desks inspect the situation, check proposed changes, commit decisions and hand over to the next desk. New FAA advisories arrive as the virtual clock advances. Past events cannot be rewritten, and a run must pass the environment’s audit before it receives a score.
The film also follows a crew-agent recovery from the 08:00 watch. The agent confirms that a standby pilot will travel on B6 488 from Orlando to Worcester, then cover B6 487 back to Orlando. It messages ops through request R3 to preserve the positioning flights. That is why the desks need to communicate: cancelling a flight can also remove the crew needed for a later departure. This action comes from the recorded run; the crew roster itself is generated.
Three minutes, three connections
The film follows a recorded run with 637 flights and 81,279 simulated passenger trips. At the 08:00 watch, the customer desk projects that three passengers connecting from B6 126 would miss B6 2902, JFK–Buffalo, by three minutes. It asks ops to hold departure from 10:25 to 10:28. That costs roughly 381 passenger-minutes for the 127 passengers booked on the departing flight, so the answer depends on what happens to the rest of the plan.
Ops checks the downstream effects and accepts. In the executed simulation B6 2902 leaves at 10:28, carrying all 127 booked passengers, with no missed connections recorded on that departure. This is a useful example because the agent’s work is visible: identify a problem, ask another desk, check the trade-off, change the plan, then see the consequences.
Across the whole day the agents make 24 crew moves, hold 31 departures and cancel one flight. The run scores 0.795, with a total cost of 4.24 million passenger-minute equivalents versus 6.01 million for the standard crew baseline: 29.4% lower cost on this particular day. It still ends with delays and stranded passengers. A good score is not a claim that the day was painless.
What is real, and what is simulated?
| Part of the environment | Where it comes from |
|---|---|
| Flight schedules, tails and recorded delays | US Bureau of Transportation Statistics on-time records |
| Aircraft characteristics | FAA registry and BTS Form 41 B-43 |
| Ground stops and ground-delay programs | FAA Command Center advisories, revealed as issued |
| Weather | NWS METARs, forecasts and discussions via Iowa Environmental Mesonet |
| Passenger trips and connections | Generated using BTS T-100 load data and DB1B/DB1C itinerary surveys; domestic itineraries |
| Crews, pairings and reserves | Generated from the schedule under the model’s duty and rest constraints |
| Airport geometry and imagery | OpenStreetMap and USGS; gates, taxi routes, walking paths and the control room are display reconstructions |
These are simulated airline operations grounded in public records. The passenger trips are not private airline itineraries, and the generated crew roster is not JetBlue’s actual roster. The 3D movement explains the result; the simulator’s audited flight, crew and passenger outcomes determine the score. The film’s network board follows recorded departure and arrival times along reconstructed paths; it is not an ADS-B track. Its opening globe uses NASA Blue Marble imagery, a 2004 satellite composite used as geographic context.
From one desk to a passenger-aware network
AirOpsEnv v1 asks one hub desk to recover a short storm window and compares its plan with a proven hindsight optimum. V2 expands to the whole network with operations and crew agents. V3 adds the customer desk and changes the objective to account for passengers reaching their final destinations, including rebooking, nights away and refund exposure, alongside aircraft and crew costs.
The v3 pack has 44 training dates, 24 evaluation dates and two calm controls. The evaluation dates cover separate summer and winter storm events at JFK and Boston. The published board runs six models with three desks on all 24 dates, plus a one-agent comparison for GPT-6.1 Sol: 168 scored runs in total.
The explorer includes all three published evaluations:
- V1: GPT-6.1 Sol on five hand-picked days, reward 0.4191, versus 0.7303 for the online solver follower. This is a first look, not an estimate across the 20-date eval pack.
- V2: Sol with two desks scores 0.5983; one agent at both desks scores 0.5496, over the same 24 dates. Standard crew practice scores 0.3083. The two-desk setup uses about twice the compute.
- V3: six models with three desks plus Sol’s one-agent comparison, all on the same 24 dates. Every configuration and all four published baselines appear above.
All 221 scored runs are available in the run explorer and downloads. The complete attempt ledger also retains seven failed Terra attempts and the superseded v2 attempt; these are excluded from the published means. The registered v2 retry replaces the interrupted run on 9 July 2026. Each row links to the original record and a transcript. These versions change the task and objective, so their rewards should not be read as a progression on one unchanged benchmark.
The leading v3 averages are close: GPT-6 Astra scores 0.742 and GPT-6.1 Sol 0.721, with overlapping confidence intervals. Sol’s three-desk result is 0.044 above its one-agent result, but it uses about twice the compute. This comparison does not isolate the benefit of communication.
For v2 and v3 the reference is the best plan found in hindsight, accompanied by a lower bound where available. Reward is 1.0 at or below that best-found plan; it falls towards 0.2 as cost rises. In v3, 24 of 168 runs beat the published reference and are flagged accordingly. A separate lower-bound reward is retained. The board is a benchmark of this simulator, not a measured improvement over an operating airline.
The recorded requests, tool calls and outcomes are published with the data, so the demo can be replayed and checked without making new model calls. That is the point of this series: real operational constraints, explicit decisions and an outcome that can be inspected.
Film music: “Felicity” by Scott Buckley, licensed under CC BY 4.0. Edited and mixed with original aviation sound effects.What the community built
Within a week of part 1, the community was already building on it. Edgar Arakelyan built the live port, and Bhavish Pahwa took the same recipe to train dispatching. Edgar later developed AirOpsEnv with @agutmanstein, a licensed pilot, at Scale AI. Their environments are in the FineEnvs catalog, credited by name, each with its own eval.
RailSimEnv, by Bhavish Pahwa
In RailSimEnv the agent dispatches Arriva’s trains on the single-track Groningen–Leeuwarden line in the Netherlands after a real 2025 breakdown, collision, or signal or points failure. On a single track, trains in opposite directions can only pass where the line doubles, so one closed section backs up the whole line. The agent holds, short-turns and cancels trains so they cross safely, and every plan is graded against a CP-SAT-proven optimum. It is built from Rijden de Treinen’s open train archive and OpenStreetMap, and it runs on OpenEnv with a 3D replay.
On 45 hard eval tasks, as reported by the author:
| Model | Reward | Legal plans | At the optimum |
|---|---|---|---|
| Qwen3.8-2.4T-A95B | 0.556 | 39 of 45 | 12 |
| DeepSeek V4.1 Flash | 0.534 | 33 of 45 | 12 |
| Claude Sonnet 5.5 | 0.467 | 45 of 45 | 4 |
| GLM-5.3-Flash | 0.360 | 24 of 45 | 6 |
An open model leads here. The one closed model keeps every plan legal but rarely finds the best one. The code is in bp-high/RLSimEnvs.
Add yours
The catalog takes environments whose code lives in FineEnvs and ones that stay in their authors’ own repos. Every entry has a manifest that credits its authors, a short card, and an eval card in a shared format. That shared format is what lets this article draw each environment’s eval the same way, like the PortSimEnv v2–v5 chart above. Here is how to add one.
Ongoing work
Most of the first roadmap is built now, and most of it by the community. A live port with news over time, pilots and tugs, weather history and ship-to-port talk all exist as PortSimEnv v2–v5. RailSimEnv brought train dispatching, and AirOpsEnv now adds airline operations with three desks, real storm records and simulated passenger connections. What is still open: post-training on these environments, more data (AIS tracks, terminal logs, other ports) and more jobs worth simulating from public records.
The question behind the series is still the same. We can build realistic environments from seed data, but how do we get them as close to the real world as possible? If you have an idea or an environment, add it to the catalog or start a thread in the GitHub discussion.