Q&A from our talk on Fiber For Breakfast with Gary Bolton
Michael Thomas from our Inside Plant deployment team recently had the opportunity to talk with Gary Bolton of Fiber Broadband Association on their weekly webinar, Fiber for Breakfast. This was such a fun experience because the audience was super-engaged and peppered Michael and Gary with more questions than they could handle during the live show!
To make up for it, we’re happy to share a recap of all the questions submitted by attendees along with Michael’s expert answers. And if you want to listen to the full conversation, find the recorded webinar here.
1. What do the hyperscalers care most about – Fiber type (attenuation, dispersion)- span length – ILA spacing – power availability in rural areas?
Bandwidth, throughput, and stability or uptime. They are focused on metrics that revolve around capacity and stability of the platform like power and redundancy efforts.
2. What do you see for requirements at these micro data centers? Does each customer require their own facility (hut)? Or can they share one hut? Is caging required in these huts?
There is literally every scenario mentioned. Dedicated landlord ILA, caging, and open shared space including locking cabinets and other forms of demarcation.
3. We are hearing that hyperscalers are requiring hundreds of fibers between their large quantum compute sites to existing major data centers. It sounds like the requirements at these micro data center locations might be pretty massive.
The requirements for the ILAs are directly proportional to the number of fibers running on the route. Hyperscale demand is into the thousands and millions of strands—it’s astronomical! However, the fiber count into each individual ILA is driven by the count of used fibers on the associated long-haul route.
4. What size fiber cables are coming into these ILAs?
It really varies. Most of the time we’re dealing with 864s. We haven’t employed 6912s yet, but I’m not saying that’s not outside of the realm of possibility. However, it is unlikely that they will go there—there is a limit on the amount of fibers you can actually amplify just due to the space and power required for the electronics. And understand that these things are in these small 24 by 36 foot huts that have a very limited cooling capacity even when they can be located in very rural or extreme locations.
5. What factors might push the distance between ILAs to be longer or shorter?
Generally speaking ILAs are sited roughly 80km or 50mi apart, but the distance between ILA is not an exact science and many factors impact the actual placement. After determining the general distance where the ILA should be from the previous site, a site acquisition team looks at properties within the acceptable area and ultimately placement decisions hinge on physical site conditions, terrain, power availability or owner’s agreement to sell or lease the property.
6. Typical distance between ILA’s is one thing, but with higher bandwidth requirements (400Gb/s or 800Gb/s or higher) what are the distances required between regeneration points?
So far, in my experience, the loss budget is the same across differing bandwidths. The frequency and attenuation are an attribute of the light itself and not reflected in the amount of data passing through.
7. What’s the footprint of these and how difficult is it to be able to plan routes and drop huts every 50 miles?
The footprint is usually around an acre of land with a compound that is roughly 100’x100’ on average. The ILA shelters will sit on this compound which can and does often house multiple ILA shelters. As far as difficulty, that lies in site acquisition and the effectiveness of the team heading up acquisition. It also has a lot to do with the area. In an urban setting there are fewer options and higher costs than in rural areas.
8. You’re doing the Amps and amplification do you all do Regeneration as well and what’s the typical spacing for doing Regen vs. Amplification? What’s the max length you have seen that signal has been successful before regeneration?
No. The amplifiers replace the regenerator completely. Remember, the regens are channelized, this amplifier boosts or puts gain in the light itself. Just like an analog amplifier in the old days did for electromagnetic signals through load coils. Except now, they are inserting photons to amplify the entire light source.
For more info on Network Connex’s end-to-end ILA services, head over to our ILA Resource Page to learn more.
9. These are largely line amplifiers, so this is really just regenerating the optical signal to transmit, They’re not full of GPU’s or anything like that? You’re not doing any large language models or any kind of generative AI?
No, they aren’t full of GPUs. They’re strictly amplification and connectivity. There has been some proposed use cases for edge computing, but as far as I’ve seen, there has not been nor would there be any AI GPU placements in these sites. We need to understand also that we’re talking about a power budget. If I have a limited amount of power and cooling in each ILA, I’m going to make absolutely certain that it’s doing the task that it needs to do. If I use that power and cooling for some other function, I lose capacity to amplify. So that’s where we need to make sure it stays.
10. As we drive latency into networks — what is the real millisecond speed to be affected, and what is the driving applications and inference requiring true lower latency?
Latency in this application is due to the physical loss. The amplification does not modify the data passing through and therefore has minimal latency impacts. Where as the channelized regeneration of signal from light to electrical and back to light has significant impacts.
11. Given that AI is “eating the world”, are you seeing that the designs of the ILAs have changed much?
The designs of the ILAs have not changed tremendously. We are starting to see some compression in the electronics, like miniaturization, and there is starting to be higher power consumption per cabinet. Again, the technology and the electronics internally are not my area of expertise, but I do know that they have hybrid amplifiers that can shoot over longer distances ultimately increasing the distance between and thusly the number of shelters on a route.
12. What are the power requirements for your ILAs and are you doing any onsite generation?
Typically, they’re relying on commercial power for most everything. The power budget really varies depending on what the size of the shelter is and the design. In fact, the design and engineering actually begins with the power budget. How many cabinets and how many kW per rack drives what the power budget will be.
Once we establish the power budget and what we can get from the commercial power companies, we also size generation onsite for failure. Diesel generators are used primarily, and I’ve seen a couple of gas generators, but there are new technologies coming out for onsite power generation. It’s really exciting that you can wind up having potentially a natural gas or a propane connection at a facility and generate all the power you need for that facility without any connection to commercial power whatsoever.
13. Are you operating a DC power plant on each ILA site with batteries?
DC power is the preferred power of choice and a lot of these things as well just for the stability and kind of getting back into that old telco type mentality with regard to power and utilizing DC because it’s cleaner and more stable.
14. Local Power Generation by Bloomenergy.com, Could be a viable option for on site power generation. Solid Oxide Technology using Natural Gas for fuel. Replaces generators and Gas Turbines.
Onsite power generation is the wave of the future for ILA applications. The ability to manufacture power onsite lifts restrictions on ILA location and sizing.
15. Are you mostly seeing single mode fiber or are you starting to see any demand for hollow core fiber?
Single mode only. Right now hollow-core is in campus environments and not long-haul that would require ILAs.
16. When your fiber comes into the ILA shelters, are these all connectorized or do you guys splice them all in and then are they in fiber cross connect cabinets or how does that all work?
The fiber that is placed is not pre-connectorized. It is run in and pigtails are spliced on for terminating inside the ILA.
16. What type of fiber alarm monitoring do you deploy with these sites typically?
Monitoring on these includes all power, fire, HVAC, and transport. The monitoring is an operational responsibility of the customer or operational owner of the ILA. The level of monitoring and alarming can vary and can be set up however the customer desires.
17. What does the Capex look like on a typical ILA build?
It varies just as much as the size and capacity. On the smaller lower end you could be looking at right around a million, but it goes up from there depending upon the size, scope, location, etc. Some of the design requirements and power requirements from the hyperscale customers can drive significant expense. Use cases can vary so much and it really kind of depends on a use case and the best thing would be to contact somebody that has experience in consulting on these builds so you can get an idea of what that’s going to look like for your company and situation.
18. What technical innovations, you know, optics, power efficiencies, automation do you think will impact the ILA design over the next five years?
There’s going to be so much activity! One of the things that I can see happening, and we’ve had a lot of questions about it today, is that there’s going to be more of a dependency on onsite generated power.
There are also alternative battery solutions that are going to provide some additional support that will result in lower operational costs. As we start to see these things and electronics improve, you might start to see the distance between ILAs grow a little bit or ILAs gaining more bandwidth. The types of cooling that we use are going to have to change to be able to support some of the newer higher density electronics with more power consumption. And again, in these small spaces, that’s a real problem.
Currently, there are four concrete walls with backpack HVAC units that are dependent upon ambient cooling. There’s no really directed cooling in there, no containment. So there are evolutions in technology that will definitely change the scope of the critical infrastructure that supports the electronics.
19. Given that these huts are in rural America where we see the need for access to mobile wireless and Internet, do you see that that that ILA huts might help to provide Internet connections for people?
That wouldn’t surprise me at all and I think that that’s where we are heading. Right now, hyperscalers are driving it, but they’re not really the only people doing it. You’ve got colocation companies big into this as well. If you think about it from their perspective, they’ve got two data centers. They are leasing space in each data center. If they have an interconnected fiber between the two, that’s going to drive real estate in both sides of their business. So, when you start getting Colos in there, then that kind of opens door more for interconnected services along that route as opposed to just at both ends.