What it does
MWT2 runs the ATLAS production grid workflow: Monte Carlo event generation and simulation, reconstruction, group production and user analysis, scheduled centrally by ATLAS and delivered continuously. Work arrives through the grid rather than through a login prompt, and the facility’s job is to keep accepting it without interruption.
That shapes the design. MWT2 is deliberately bare metal and deliberately conservative, built for long-term support and stability rather than for rapid change. It is the counterweight to the Analysis Facility, which sits a few racks away in the same Chicago data center and is built on Kubernetes for exactly the opposite reason.
Across three universities
The center is split between the University of Chicago, Indiana University and the University of Illinois Urbana-Champaign, with the three sites presenting to ATLAS as a single resource. Compute is distributed across all three; disk storage is served by dCache at Chicago and Indiana.
The three campuses carry very nearly a third of the load each — in jobs, in compute delivered and in events processed — and are joined by a dedicated research network, a Science DMZ running at 2x100 Gbps over ESnet and Internet2, with a shared data layer that lets any site’s storage and compute serve the others. That fabric is not incidental to the Lab’s work: MANIAC Lab has been the primary scientific driver and funding justification for the University’s campus and wide-area network upgrades across two NSF networking awards.
The hardware spans about a decade of procurement cycles, with the compute fleet averaging roughly five years old and the storage fleet closer to three. Equipment is retired as it ages out, and the facility has twice absorbed the relocation of entire data centers without interrupting production.
The Illinois site
The Illinois third of MWT2 works differently from the other two. Rather than a cluster of its own, it is an investment in the Illinois Campus Cluster Program, the university’s campus-wide high-performance computing resource. Under its model, groups and units buy compute and storage and are guaranteed the nodes they paid for; when an owner is not using their share, the cycles go to other Illinois researchers. MWT2 benefits from the same arrangement in reverse. For ATLAS production, which can absorb any amount of opportunistic capacity and does not mind waiting, that is an unusually good fit: the site gets a floor it can count on and a ceiling it does not pay for.
The hardware lives in the National Petascale Computing Facility, which NCSA operates — the 88,000 square foot, LEED Gold building put up for Blue Waters. It is engineered for high power density rather than for general-purpose racks. Computational and storage equipment is water cooled, which NCSA puts at roughly twice the efficiency of air, with direct liquid cooling to the processors, rear-door heat exchangers on the racks and 480-volt distribution to the floor. For much of the year three on-site towers chill that water with outside air instead of drawing on the campus chilled-water loop, and the building runs at the high end of the ASHRAE envelope, rated for 65°F inlet water and 78°F inlet air.
The practical effect for ATLAS is that the Illinois third gets national-facility power, cooling and operations without anyone in the collaboration having to build or run a machine room. NPCF hosts major projects for NSF and industry partners alongside university systems, and leases machine-room space by the square foot. MWT2 occupies a small part of it.
Mark Neubauer, Professor of Physics at Illinois, is the site’s principal investigator.
Compute and storage
Batch scheduling is HTCondor throughout. Storage is dCache, with XRootD providing the wide-area data access that lets ATLAS jobs read from the nearest copy of a dataset rather than waiting for a transfer. Data placement across the grid is managed by Rucio, the ATLAS data management system.
The facility also provides LOCALGROUPDISK, a Rucio storage endpoint where individual physicists keep datasets larger than a shared analysis filesystem can reasonably hold. Because it sits on the same networking infrastructure as the Analysis Facility, analysts can reach that data at local speeds.
What a quarter looks like
In a recent quarter MWT2 completed about 9.2 million jobs at roughly 93% success and moved close to 40 petabytes over the research network. It holds around 88 million files.
The wide-area traffic is a two-way exchange with the rest of the grid rather than a one-way feed: roughly 23 PB in and 17 PB out over the quarter. The heaviest partners in both directions are the Brookhaven Tier-1 and CERN, followed by the European and Canadian Tier-1s — RAL, TRIUMF, IN2P3 — and the other U.S. Tier-2s. Data arrives from a site in Tokyo as readily as from one in New York, which is what a global grid looks like in practice.
Two roles at once
MWT2 is simultaneously a top production site and a heavy analysis endpoint, and the two workloads look nothing alike. Managed production consumes about 71% of the compute. User analysis accounts for 58% of the jobs and reads 94% of all events.
That asymmetry is the real shape of the facility. Production is a smaller number of long, compute-heavy jobs; analysis is a flood of short jobs that touch enormous amounts of data. Designing for either one alone would get the other badly wrong, and it is why storage throughput, not core count, is usually the thing that limits what the site can do.
Beyond ATLAS
MWT2 hosts the bare-metal Elasticsearch cluster behind the Lab’s operations analytics platform, which indexes operational data from across ATLAS distributed computing and serves coordinators, shifters and, increasingly, AI agents. The center also contributes opportunistic cycles to the Open Science Pool when ATLAS is not using them, so capacity procured for the LHC supports researchers well outside high energy physics.
Reliability
ATLAS measures its sites continuously. Over the year from September 2025 to September 2026, MWT2 recorded 99.8% reliability in WLCG’s published figures, while running a near-constant load of tens of thousands of simultaneous jobs.
