PING

APNIC

PING is a podcast for people who want to look behind the scenes into the workings of the Internet. Each fortnight we will chat with people who have built and are improving the health of the Internet. The views expressed by the featured speakers are their own and do not necessarily reflect the views of APNIC.

  1. 11h ago

    The [dismal] economics of AI

    In this episode of PING, APNIC Chief Scientist Geoff Huston reflects on the tremendous economic consequences of the surge in capital investment in Artificial Intelligence (AI) and the associated Data Center (DC) build out. This not involves so much money, it represents a tenth or more of the US annual Gross Domestic Product (GDP) and has significance outside of the application of AI to problems in the world. As technology consumers we’re already used to the idea the cost of basic inputs like RAM, storage devices, the GPU cards which we need to run high speed gaming computers are both in short supply and gone up in price significantly. But the economic effects of the surge in AI go beyond the IT sector. They affect all of us fundamentally. Having a speculative activity like this represent so much of any economies activity means that as it ebbs and flows, or even worse crashes, the same amount of economic activity is removed from other productive investment. Some of the AI money is hypothetical, set into the future on the premise the investments being made now play out, but some of it therefore is highly risky, money which won’t actually exist when the time comes, but the debt side, the cost of raising all this money, spending all this money and time and effort building out the DC. That debt will exist, and will either be recovered in a fire sale of assets, or be distributed around as pain in the pockets of investors who put their money into the proposal. A problem for almost all of us, is that the investment decisions here have been taken out of our hands: Although our apetite for risk may vary, the decision around investment is made mostly on our behalf, by the banks, the investment companies, the “institutional investors” which may seem remote, but when you consider the Australian Superannuation sector represents $4.5 trillion Australian Dollars, and is the fourth largest investment pool in the world, for a nation of only 27 million people, our exposure to this risk is a lot larger than you might think. The same is true for the pension funds of Canada, of the United Kingdom, the European Union, Asia and especially the investment industry of the USA, because the AI companies have joined or are on the brink of joining the pool of investment which is held to be “normal” and a required part of a balanced portfolio. To build and operate a modern AI focussed DC demands a huge investment in construction, in power engineering, in air conditioning and related systems. This investment is so potentially profitable that it can command all the inputs, all the labour required to build it, and afford to pay ahead of the rates normal construction sector projects would expect. It’s contributing directly to a supply chain shortage across the civil engineering sector, and in unrelated construction activity like housing. The diversion of electricity supply to power a DC means that electricity prices for domestic consumers are being pushed up, and the social consequences of where these DC are built effects their immediate surroundings in terms of the amenity and viability as a place to live and work. As Geoff points out, these systems are so big that in the USA, they cannot co-exist in the same state as a competitive sector: They have to be sited independently without a comparable DC investment, lest they overburden the utllity supply systems. This isn’t something the worldwide economy sat down and decided to plan out, it’s emerged into the world with probably unstoppable momentum. The problem exists everywhere, all at once, and the problem is getting larger, as the investor side gets more concerned of their risk of exposure as the cracks in the model emerge. Who “wins and loses” here goes beyond the nameplate on the DC, or the name of the AI company. We’re all standing to suffer economic shocks, as this AI craze works its way through to the inevitable other side of the bubble.

  2. Sep 30

    The October 2026 Root KSK roll

    In this episode of PING we’re hearing from Duane Wessels, a fellow from Verisign who researches DNS and who we’ve had on PING before. This time Duane is discussing the history of changes of top level Key Signing Key (KSK) called “roll-over” of the KSK at the root of the DNS. This month on the 11th of October, we will complete deployment of the 3rd KSK, in the second rollover. It’s a multi-year process. This second key roll was delayed from the first rollover in 2017 by some unexpected circumstances in 2020 and 2023. The initial KSK was deployed in 2011. In October, the new key pair will enter service for the first time. The old key pair will no longer be used to sign, but will remain visible in the root until early 2027. Duane discusses some recent DNS research work by himself and Roy Arendts from ICANN, about the observations of resolver behaviour across a root KSK roll-over. Their most recent blog article “The 2024-2026 Root Zone KSK Rollover: Updates and Observations” from July 2026 is a follow-up to their original article “The 2024-2026 Root Zone KSK Rollover: Initial Observations and Early Trends” from March 2025. It’s worth reading both articles to see how experience gained in the initial observations of the key rollover have informed subsequent research. The DNS KSK at the root is the fundamental trust source for all subsequent DNSSSEC signed data in the global DNS. The processes around management of the private keys of this public-private key pair are described by IANA at their DNSSEC webpage and Verisign play several critical roles in the production, maintenance, distribution and use of the keying materials which come from this activity. Duane monitors this closely, both as a participant in the key ceremonies held under the auspices of ICANN/IANA, and from the vantage points Verisign has into global DNS from operations of the “J” root server anycast cloud. Duane and Roy had the opportunity of re-analysing the behaviour of resolver systems under the key roll with greater clarity, because since the original key roll in 2017 a larger community of resolvers now use a signalling method defined in RFC8145 which indicates which trust anchors they see and are using. This has improved visibility of the uptake of the new trust anchor and shows very clearly in their time-charts of deployment of the new keying materials. Key rolls in the DNS are not a “one and done” process, this is a multi-year activity which is about to reach it’s next most significant date later this month in October. On October the 11th, The new key will begin operations signing over the zone and the older keys will be withdrawn from signing. The new KSK has been published and listed in the root zone since January 2025, and was promoted by RFC5011 signalling in Feburary 2025.

  3. Sep 2

    Measuring the impact of locally served root zone

    In this episode of PING, we talk with Willem Toorop from NLNet Labs and Ilyas Rahimi who recently completed a masters in Security and Network Engineering at the University of Amsterdam (UvA). They’ve been looking at the effects of local root serving. Ilyas was on a masters programme called “OS3” at UvA (from the triplet of Open Standards, Open Software and Open Security). NLNet labs is on the campus of UvA and Willem has been working with the masters programme for some time, pursuing research in DNS and DNSSEC with the students doing their masters thesis component. They conduct 2 1-month original research activities in the masters course, and write the second one up as their masters thesis. Locally served root is a model of operating a DNS resolver where it “pre-fetches” the state of the root zone, the apex of the entire DNS naming model. By fetching this data the idea is that both early termination of unresolvable domain names (names where the final terminal label can be seen not to exist under the root of the DNS) and a degree of privacy in fetching resolvable names can be gained, by looking in the local copy. There is a current proposal in the IETF to make this a “best current practice” or BCP. Ilyas looked at 3 different public code-bases (bind, unbound and the knot DNS resolver) and a total of 4 configurations, exploring both in-band and HTTPS bound fetch of the root zone. As well as identifying a bug in the model unbound used, which accounted for a spike in traffic, Ilyas has been able to show that under the update cycle typically seen for the root there is a surprising increase in apparent network traffic fetching the zone, compared to the many but far smaller events asking for uncached data from the roots. There is more work to do in this space, and with the OS3 masters thesis component now given more time in the degree program, there is hope that Willem can continue to work with UvA students, and carry forward the measurements. WIllem has been exploring how the system behaves if converted to incremental updates (IXFR) rather than an entire zone-fetch.

  4. Aug 19

    DNS Cold Start

    In this episode of PING, APNIC Chief Scientist Geoff Huston and I discuss the behaviour of the DNS system when you have to come up from nothing: the “cold start” where no data exists from prior queries in your “cache”. This stems from a talk given by Ondřej Surý, that Geoff saw at the recent RIPE-92 Meeting held in Edinburgh, in the DNS working group. Cache, is the things you hold on to, from prior work done. It’s a fundamental technique in computer science to avoid the massive disparity in speed between parts of the systems, keeping things you use a lot in “fast” memory close to the CPU, and avoiding having to go to “slow” memory or even worse disk or tape files, to find the data. The size of your cache and it’s speed can have a huge influence on the speed of your program. Comparing CPU performance with and without prior cached data can be very instructive to it’s benefits! -In the DNS, Cache is how you avoid having to wait for a remote system to answer over the network (with all the round-trip delay of question and answer) because you know parts of your answer from the prior queries and answers you hung onto. “Cold Start” is a well known problem in large distributed systems. The problem is not just that industrial systems like coal plants and gas turbines need time to warm up and get sufficient energy to drive the generator, in the so-called “black start” there is the problem of energising the magnetic coils associated with generation of electricity. It’s distinct DC energy, used to make the generator enter the state where the rotational force from the turbine actually makes power. Without a source of power, you cannot make the coils “excite” and so the spinning generator won’t actually make any electrical power. Typically, this DC voltage comes from an independent source like a small diesel generator, or a source of electrical power which hasn’t been affected by the outage. If you contract to supply this contingency power, failure to be available is a major issue. The DNS isn’t an electrical generation network, but it does have this kind of complex dependency, mapping from DNS fully qualified name, to the specific IP addresses bound to the name. The “D” part of DNS stands for “Domain” and the elements of the name in a sequence form “domains” of control (or at least potentially do) separated by the dots in the name which may have a distinct “name server” detailing how things are named under-neath that domain boundary. So a single question “what is www.potaroo.net” has the potential to invoke at least 3 if not more questions, for each “domain” in the name. When you have a cache. The cache of known name-to-address pairs includes the data you need to know the addresses of the nameservers for each domain you have seen, which can tell you the name-to-address pairs for the other things you are asking: either directly or indirectly by passing you on to another system. But if you have no cache, when you ask (for example) the nameservers for .NET and are told they are A.NET and B.NET you are no wiser: how do you find these hosts, when you don’t know how to find .NET? Geoff has been chasing down what he sees exploring this “cold start” behaviour, and what it tells us about how people chose to name their hosts and services, how they provide the “name servers” behind these hosts and services, and how intruding intermediaries, or even trying to work around the risks of a cold start can increase the query burden on the clients worldwide.

  5. Aug 5

    Canonical Cache Representation

    In this episode of PING, we talk with Job Snijders about secure Internet routing again, focusing on an approach to preserving the state of RPKI as a longterm historical record for research and analysis. Job was last on PING to discuss the “Erik” protocol. Job has been running an archive of RPKI state for some time, as a volunteer activity but an exercise which was feasible when BGP speakers producing ROA objects was measured in the hundreds to low thousands is significantly more expensive when the population of ROA producers is a more realistic percentage of the around 80,000 AS holders worldwide. Because Public Key Cryptography depends on a regular re-signing, and re-cataloging (as another signed object, the RPKI “Manifest” file) even when there is no substantive change in the state of signed information, a large amount of “churn” can be seen in the data, and even a compressed form of this state in turn incurs a huge overhead in storage of hard-to-compress data. Job reached the limits of his free activity, and started to explore a more compact and useful representation. This has emerged in the IETF standards process as two related activities. The Canonical Cache Representation or CCR, and the aggregation over this for data represented as the RPKI Spool data model for “materialising” the state of RPKI objects. Along with his fellow IETF draft authors Bart Bakker , Tim Bruijnzeels, Theo Buehler and Fedor Vompe Job has managed to define a remarkably compact, highly compressible representation of RPKI validated objects, and the cryptographic payloads. These systems have been designed to leverage well known UNIX and other techniques for data management such as the “tar” format for spooled data, ASN.1 for binary encoding, Merkle Trees to hold compact hash state, and data compression using the zstd encoding. It’s a nice approach to solving the scaling problem.

  6. Jul 22

    DNS Query Duplication

    In this episode of PING, APNIC Chief Scientist Geoff Huston and I discuss a traffic behaviour in the DNS which Geoff has noticed in the labs advertising based experimental data capture. Virtually every DNS query Geoff sees, he sees twice (or more). For a cohort of about 150 million unique DNS labels on a given day, The Labs system is collecting 270 million incoming DNS queries. Thats a lot of duplication. What's going on? The advertising data collection depends on issuing unique DNS queries, which in turn generate unique web page serves. This allows measurement of internet-wide behaviour on about 30 million browsers, games, devices every day. Not to say that these names and web URLs are not routinely seen by more than one entity, intermediary systems such as caches and proxies as well as re-visiting old open website tabs on a browser can cause this. The point is that after the "first" fetch, the subsequent fetches can usually be held to be re-presentations of the same experiment and can therefore often be discarded (in the case of the web). But, for the DNS which has always had an element of unreliable transport, and which in turn invites measurement of features like DNSSEC which cause SERVFAIL messages, and demand repeated attempts to "find the DNS name-to-address mapping, re-fetching is itself something under test. How many resolvers lie behind a given users systems? How many kinds of resolver (DNSSEC enabled, or not) does the user depend on? Finding that almost all DNS queries are repeated, and it turns out repeated very quickly invited Geoff to have a look at what's going on "under the covers". There are some patterns behind what's being seen which Geoff explored in the APNIC Labs Blog.

  7. Jul 8

    Testing RSSAC 028 DNS root server name choices

    In this episode of PING, we talk with Willem Toorop about a measurement of 6 choices for naming the root servers of the global DNS. This measurement was motivated by RSSAC 028, a 2017 technical analysis for ICANN of the naming scheme used by the root DNS servers. Root name servers are part of the global DNS system, and they provide basic bootstrapping for every other DNS resolver and authoritative name server worldwide. This is their primary role, but they also handle query load relating to the global DNS system all day and every day. There are 13 distinctly named root servers, each one identified by a letter from A to M under the domain root-servers.net. Although there are only 13 distinct "labels" there are in fact thousands of machines providing this service as independently operated clusters behind each of these letters, using the BGP "anycast" method we have discussed on PING before. The special zone, root servers.net was instantiated in 1995. It's delegated as normal under the .net zone, which in turn delegates from the root "dot" zone. If you are bootstrapping a new resolver, to find a root server you appear to need to know about the delegation of .net in order to find root servers.net, to find the given server. This is a circular dependency because you don't know how to find .net until you have asked a root server instance. The circular dependency is resolved by the contents of an initial "priming" response. This response provides the additional "glue" information to seed direct knowledge of how to find each of these named root servers as is, without all the intermediate logic of the circular dependency. This glue is inherently not signed in DNSSEC. It's insecure. The priming response is very carefully curated for both its size and the content but it would be good to give it some content security and increase trust in the message. RSSAC 028 explored the impact of applying DNSSEC over this domain, which in turn would add DNSSEC signatures to the priming response and impact a goal of keeping this a small 512 byte backwards compatible UDP packet. By keeping this response small, it would continue to work on all legacy systems worldwide. Nothing happens quickly in DNS evolution, and this naming question has now been under consideration for almost a decade. Willem and his colleagues at NLNet Labs and SIDN produced two reports reflecting on the candidate naming schemes, application of DNSSEC, and the impact on the bootstrap fetch. There are some surprising outcomes!

Ratings & Reviews

5
out of 5
4 Ratings

About

PING is a podcast for people who want to look behind the scenes into the workings of the Internet. Each fortnight we will chat with people who have built and are improving the health of the Internet. The views expressed by the featured speakers are their own and do not necessarily reflect the views of APNIC.

You Might Also Like