© Internet Society / Nyani Quarmyne / Panos Pictures

Consolidation in the Internet Economy

The Internet economy, as understood in this report, is broadly defined as the economic activities that either support the Internet or are fundamentally dependent on the Internet’s existence. This section presents an overview of current trends of consolidation in its affiliated markets, including growing forces of concentration, vertical and horizontal integration, and fewer opportunities for market entry and competition. The purpose is to give an overview of different markets, and to provide the analytical basis for the following sections that look closer at how these trends may influence the Internet’s evolution.

Organised around three separate but closely linked economic domains (Internet Applications; Access Provision; and Service Infrastructure) this section is intended to provide a snapshot of various markets, and to describe how they relate to the Internet as a whole. As such, it is not intended to provide an in-depth economic analysis, nor is it exhaustive in terms of encompassing all relevant markets and services. Rather, it provides a starting point for further discussion.

Internet Economy

The Internet economy, as understood in this report, is broadly defined as the economic activities that either support the Internet or are fundamentally dependent on the Internet’s existence.

“One-stop shops”

© Frederic Courbet / Panos Pictures

The top 5 companies in the application layer today are Alphabet (parent company of Google), Amazon, Tencent, Facebook, and Alibaba:2Investopedia (2018). World's Top 10 Internet Companies.  Available at: https://www.investopedia.com/articles/personal-finance/030415/worlds-top-10-internet-companies.asp

These global tech companies build digital environments that comprise multiple platforms across a diversity of spaces, and are thus often described as digital conglomerates or digital giants14The New York Times (2017).  Conglomerates Didn’t Die. They Look Like Amazon.  Available at:    https://www.nytimes.com/2017/06/19/business/dealbook/amazon-conglomerate.html    and  The Economist (2018). Competition in the digital age: How to tame the tech titans.  Available at: https://www.economist.com/leaders/2018/01/18/how-to-tame-the-tech-titans. They are largely present in the application layer of the Internet, although they increasingly offer cloud services and infrastructure, as we will explore further below. They are not just the biggest companies on the Internet; they are some of the most valuable companies in the world.

Digital platforms tend to be two-sided markets, meaning they develop useful online spaces or platforms. On one side, people acquire desired products and services, and on the other side, businesses can find customers. These platforms gain market share in their respective markets due to data control and network effects (where the service’s value to the user increases with the number of other users).

© Internet Society / Nyani Quarmyne / Panos Pictures

54.5% of our survey respondents saw users in their region having a limited choice of applications and services.

Where data is concerned, companies use the openness and cross-border nature of the Internet, along with valuable data about their customer base, to match customers with products and services, and to continuously improve such offerings. The more customers using a service, the more data they generate for the platform. Platforms use data they collect across a range of devices and services to target advertising, generate extra revenue streams from data analytics, improve their offerings, and evolve strategies for moving into new services and markets. Platforms also increasingly use data to feed artificial intelligence (AI) processes designed to anticipate people’s purchasing and other behavioural patterns.

Network effects are also central to the value and power of these platforms. The more people use a marketplace like souq.com or a communications app like WhatsApp, the more valuable and attractive it is to current and prospective users. Some platforms open up a limited set of application programming interfaces (APIs) and software development kits (SDKs) to enable external developers to build on a particular functionality or sell to users via the platform (e.g., Google Maps or Apple’s App Store). These two-sided markets link innovators and customers while adding value and a stream of innovation to the platform itself.

Platform companies are also effective at attracting and retaining customers because they use their resources – and the data they continue to gather – to provide and continuously improve a one-stop shop. This means that based on what data shows about potential user preferences and needs, platform companies add features that address and test user desires. And data about projected user needs and preferences also guide their acquisition (or replication15The Economist (2018). ibid) of novel and potentially competitive services. For example, Facebook acquired WhatsApp and Instagram in order to maintain leadership in the social messaging and photo sharing markets.

© Internet Society / Nyani Quarmyne / Panos Pictures

At the core of the platform-user relationship is often the platform’s drive to keep extending user time on the platform and deepening their engagement. Platforms retain customers through a virtuous cycle: data that is generated through the use of applications and services feeds the data engine, which in turn creates a better and more tailored Internet experience. This keeps the customer engaged and on the platforms, which generates further data, and so on.

© Atul Loke / Panos for Internet Society

Move fast and leverage dominance

The platforms’ dominance in their original domains or markets enables them to rapidly extend into others. Amazon and Alibaba run e-commerce platforms, but also offer cloud platforms. Google’s search platform provides most of the company’s current revenue, via advertising, but its parent organisation, Alphabet, owns an ever-widening array of companies:16Business Insider (2016).  Google's parent company, Alphabet, explained in one chart.  Available at: http://uk.businessinsider.com/chart-of-alphabet-google-parent-company-infographic-x-gv-2016-1

Internet-Society-Alphabet-Graphic

Similarly, Alibaba and Tencent have diversified into new areas: 17 Forbes (2018).  Who Can Go The Distance In Alibaba And Tencent's Battle For Southeast Asia And India?  Available at: https://www.forbes.com/sites/vinnielauria/2018/05/09/alibaba-and-tencent-vie-for-dominance-in-southeast-asia-and-india-who-can-go-the-distance/#781ee5f45154

InternetSociety-Alibaba-Tencent

Like Alphabet, Alibaba and Tencent have expanded well beyond their initial core service. The diagram shows merely a selection of their interests across multiple industry sectors. For example, the Alibaba group owns multiple online market places including Lazada and also Taobao, a consumer-to-consumer platform with more than 600 million active monthly users18Alibaba Group (2018).  Alibaba Group Announces June Quarter 2018 Results.  Available at: https://www.alibabagroup.com/en/news/press_pdf/p180823.pdf. Baidu, another large Internet company operating mainly in China, runs China’s largest search-engine19GlobalStats statcounter (2018). Search Engine Market Share in China - November 2018.  Available at: http://gs.statcounter.com/search-engine-market-share/all/china, owns multiple content, advertising and app services20Reuters (2018). Profile: Baidu Inc (BIDU.OQ).  Available at:  https://www.reuters.com/finance/stocks/company-profile/BIDU.OQ, and also leads the Apollo Project, a leading autonomous driving and AI project21TC (2017).  Baidu’s Apollo platform becomes the ‘Android of the autonomous driving industry’.  Available at:  https://techcrunch.com/2017/07/05/baidus-apollo-platform-becomes-the-android-of-the-autonomous-driving-industry/.

Streaming, mobile, and voice

© Internet Society / Nyani Quarmyne / Panos Pictures

Across the platforms, content is driving online experiences, from YouTube to Amazon Prime to Twitch, Spotify, Netflix and India’s Hotstar. Gaming is growing, especially on mobile devices, and game-streaming services like Twitch respond to the 18 to 35-year-old gamer demographic, who tend to spend more time watching other people play video games than they do watching traditional sports on television22Limelight Networks (2018).  The State of Online Gaming – 2018.  Available at: https://www.limelight.com/resources/white-paper/state-of-online-gaming-2018/#watching. In the UK, people already spend more leisure time online than they do watching television23EY (2017).  Decoding the UK digital home: EY’s annual survey of 2,500 UK consumers reveals the diverse nature of today’s digital household.  Available at: https://www.ey.com/uk/en/services/specialty-services/ey-are-consumers-content-with-your-content. Streaming video and gaming on mobile devices is changing the usage and content viewing behaviours of mobile customers and is one of a number of drivers bringing about mergers in the communications, media, and content industries.

Driven by content, mobile is estimated to account for more than half of all web traffic and is still growing.24Smart Insights (2018).  Global social media research summary 2018.  Available at: https://www.smartinsights.com/social-media-marketing/social-media-strategy/new-global-social-media-research/ Mobile apps are thriving, and expected to reach a total of 258 billion downloads valued at $157 billion (USD) in 2022.25App Annie (2017). The 2017-2022 App Economy Forecast: 6 Billion Devices, $157 Billion in Spend & More.  Available at: https://www.appannie.com/en/insights/market-data/app-annie-2017-2022-forecast/ Additionally, people are increasingly using voice to interact with devices and software, especially through the use of smart assistants and devices such as Amazon’s Alexa, or Google Assistant, which powers Google Home, a smart-home Internet of Things (IoT) platform.26Gartner (2016). Gartner Says Digital Assistants Will Serve as the Primary Interface to the Connected Home. Available at:  https://www.gartner.com/newsroom/id/3352117

© Internet Society / Nyani Quarmyne / Panos Pictures

Overall trends

Consolidation in markets at the application layer currently appears to be centred around a relatively small number of firms, predominantly from the US and China. Users and businesses benefit from this trend because they enjoy the convenience and markets that these platforms provide. The same companies are also expanding their operations into new markets through acquisitions and new product developments, further expanding their ownership to an even broader set of services that will shape users’ Internet experience.

The Internet is growing and shrinking at the same time: users and traffic are both increasing, but most of the interactions are with a smaller number of dominant one-stop-shop players. This raises a question: How will the increase in time spent online, combined with the extension of the application layer into domestic and public life (e.g., smart homes), impact the Internet and its users?

Supply side consolidation

To gain access to the Internet, users interact with providers on the supply side of access provision, chiefly mobile Internet or fixed broadband operators. National and regional circumstances vary greatly and have a significant impact on consolidation trends in the access layer in a specific region, making it difficult to identify a widespread trend of consolidation in the access market. But in certain markets, the access market is dominated by a small number of suppliers.32e.g. GSMA (2016). Assessing the impact of mobile consolidation on innovation and quality. An evaluation of the Hutchison/Orange merger in Austria. Available at: www.gsma.com/publicpolicy/wp-content/uploads/2017/07/GSMA_Assessing-the-impact-of-mobile-consolidation-on-innovation-and-quality_36pp_WEB.pdf.33e.g. WEF & Accenture (2017). White paper. Digital Transformation Initiative. Telecommunications Industry. Available at: http://reports.weforum.org/digital-transformation/wp-content/blogs.dir/94/mp/files/pages/files/dti-telecommunications-industry-white-paper.pdf; McKinsey & Company (2016). Telecommunications industry at cliff’s edge: Time for bold decisions. Available at: www.mckinsey.com/~/media/McKinsey/Industries/Telecommunications/Our%20Insights/Winning%20the%20rush%20for%20data%20services%20in%20the%20Middle%20East%20and%20Africa/Telecommunications%20industry%20at%20cliffs%20edge%20Time%20for%20bold%20decisions_June2016.ashx Where this is the case, such dominance tends to be driven by certain structural factors like high fixed costs and significant barriers to entry (often for regulatory reasons), which often means that providers need significant scale to become and remain commercially viable34 c.f. Deloitte (2014). The future of Telecoms in Africa. The “blueprint for the brave”. Available at: www2.deloitte.com/content/dam/Deloitte/fpc/Documents/secteurs/technologies-medias-et-telecommunications/deloitte_the-future-of-telecoms-in-africa_2014.pdf; Capgemini (2014). Communications Industry: On the verge of massive consolidation. www.capgemini.com/wp-content/uploads/2017/07/ma_pov_05082014.pdf. . This need for economy of scale, encapsulating the ability to offer a lower price than that of most competitors, can naturally lead to consolidation. There is also evidence that some markets that were liberalised from historic governmental monopolies are again becoming consolidated, sometimes leading to higher prices35c.f. BEREC (2018). BEREC Report on Post-Merger Market Developments - Price Effects of Mobile Mergers in Austria, Ireland and Germany. Available at: https://berec.europa.eu/eng/document_register/subject_matter/berec/reports/8168-berec-report-on-post-merger-market-developments-price-effects-of-mobile-mergers-in-austria-ireland-and-germany .

Trends: Developed markets are experiencing a trend towards consolidation by dominant operators, with several instances of markets with 4 mobile operators moving to 3, via mergers36The Economist (2016, Feb 4). Three’s a Crowd. Available at: https://www.economist.com/britain/2016/02/04/threes-a-crowd; Tyagi, K. (2018) Four-to-Three Telecoms Mergers: Substantial Issues in EU Merger Control in the Mobile Telecommunications Sector. IIC-International Review of Intellectual Property and Competition Law, vol. 49(2): 185-220. Available at: https://link.springer.com/article/10.1007/s40319-018-0677-3.. Access providers face lower prices for data transport, a maturing subscriber base, competition between fixed and mobile that is driving down prices, and the erosion of voice and SMS revenues by Voice over IP (VoIP) or IP-based instant messaging (IM) services37e.g. Deloitte, 2014, ibid; McKinsey & Company, 2016, ibid.. In response, some large access providers are moving into other layers by purchasing or allying with media, IT services, or software companies, and merging with other operators.

Some access providers are responding to the challenges of lower prices, infrastructure investment, and the erosion of revenues by sharing infrastructure38e.g. Deloitte, 2014, ibid.. Sharing towers, masts, and ducts can reduce costs, ensure coverage, and still promote competition39c.f. OECD (2018). OECD Reviews of Digital Transformation: Going Digital in Sweden. Available at: http://www.oecd.org/sweden/oecd-reviews-of-digital-transformation-going-digital-in-sweden-9789264302259-en.htm. . This has proven successful in the highly regulated markets of the UK, Poland, and Romania40McKinsey & Company, 2016, ibid.. Other responses include ventures into other businesses and layers, such as operators purchasing digital media, IT services, or software players with an eye toward vertically or horizontally expanding into sectors that could, over time, become parts of their core business41c.f. WEF & Accenture, 2017, ibid; McKinsey & Company, 2016, ibid; Capgemini, 2014, ibid. .

© Internet Society / Nyani Quarmyne / Panos Pictures

Spectrum – cash cow or country enabler?

Mobile providers need allocations of radio spectrum to give users wireless access to the Internet. Access to affordable spectrum is therefore a foundational principle for ensuring access to ICTs and future network development; however, wireless access networks could be hampered, especially in developing countries, by the lack of optimal and affordable spectrum42e.g. Lewis, C. (2018). Lessons From Spectrum Auctions: A Benchmark Approach. Policy Brief for DTPS. Available at: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3185752; Song, S. (2015). How To Let GSM Serve The People That Other Networks Can’t Reach. Available at: https://manypossibilities.net/2015/04/how-to-let-gsm-serve-the-people-that-other-networks-cant-reach/.

When governments decide to use spectrum auctions to generate revenue, incumbents tend to prevail over potential new entrants.43Lewis, 2018, ibid; ITU (2012). Exploring the value and economic valuation of spectrum. Available at: https://www.itu.int/ITU-D/treg/broadband/ITU-BB-Reports_SpectrumValue.pdf This is at least partly because auctions often fail to attract new entrants into the market when reserve prices are set too high or when the amount of spectrum on offer is too limited.44 ITU, 2012, ibid. In Ghana and Brazil, auctioned spectrum for 4G has gone to incumbents and market concentration has increased.45Lewis, 2018, ibid. Governments, however, may deliberately incentivise the creation of new market entrants. In some cases, those new entrants may be purchased by incumbents further down the line.

Trends: Incumbent operators already had first-mover advantage by occupying the optimal parts of existing spectrum allocations. Instead of attracting new entrants to diversify markets, spectrum auctions may be helping dominant players gain even more market share by only making it feasible or attractive for them, and not for new entrants, to participate in competitive bids.

© Internet Society / Nyani Quarmyne / Panos Pictures

For many users, Facebook is the Internet

© Internet Society / Nyani Quarmyne / Panos Pictures

Three billion people are currently unconnected to the Internet, a little less than half of the world’s population.46ITU (2017). Facts & Figures: 2017. Available at: https://www.itu.int/en/ITU-D/Statistics/Documents/facts/ICTFactsFigures2017.pdf There are still significant gaps between developed and developing regions, urban and rural, men and women, wealthy and poor, young and elderly, and literate and illiterate people.47e.g. GSMA (2018). The Mobile Gender Gap Report. Available at: www.gsma.com/mobilefordevelopment/programme/connected-women/bridging-gender-gap-mobile-access-usage-low-middle-income-countries/; Van der Spuy & Souter (2018). Women’s Digital Inclusion. ISOC & APC. Available at: https://www.apc.org/en/pubs/womens-digital-inclusion-background-paper-g20; Chair, C. (2017). Internet use barriers and user strategies: perspectives from Kenya, Nigeria, South Africa and Rwanda. Available at: www.researchictafrica.net/docs/RIA 2016 Comparative FGD study_Final_Web version.pdf;

APC (2016). Ending digital exclusion: Why the access divide persists and how to close it. Available at: www.apc.org/en/pubs/issue.   Quality, price and availability of both fixed and mobile broadband services vary greatly between countries and regions.48 e.g. see #AfterAccess data across three regions in the Global South: www.afteraccess.net. That said, some approaches to connecting the unconnected may risk locking users into current application platforms and exposing users only to a limited version of the Internet.49e.g. Marda, V. (2016). Facebook Free Basics: Gatekeeping powers extend to manipulating public discourse.  Centre for Internet and Society. Available at: https://cis-india.org/internet-governance/blog/catchnews-january-6-2016-vidushi-marda-facebook-free-basics-gatekeeping-powers-extend-to-manipulating-public-discourse; Belli, L. (2016). Net neutrality, zero rating and the minitelisation of the Internet. Journal of Cyber Policy, vol. 2. Available at: http://internet-governance.fgv.br/sites/internet-governance.fgv.br/files/publicacoes/net_neutrality_zero-rating_the_minitelisation_of_the_internet_final.pdf.

Developed country users largely enjoy higher bandwidth, faster connections, and lower latency, thanks to peering between networks and locally hosted content.50Kende, M. & Rose, K. (2015). Promoting Local Content Hosting to Develop the Internet Ecosystem. Internet Society. Available at: https://www.afpif.org/wp-content/uploads/sites/26/2017/10/Promoting-Local-Content-Hosting-to-Develop-the-Internet-Ecosystem.pdf While most developed countries have near-universal fixed networks to support the widespread introduction of broadband, in Africa, for instance, fixed line networks reach barely 1% of the population.51 ITU, 2017, ibid. Although the total number of mobile phone subscriptions globally is greater than the number of people on Earth, many people in developing countries still do not use a mobile phone.52 See #AfterAccess data across three regions in the Global South: www.afteraccess.net. And when the unconnected do gain access, such access may not be of adequate quality or low enough cost to be meaningful to them.53Souter & Van der Spuy (2018). W20 Digital Inclusion background paper. London: GSMA. Available at: www.gsma.com/mobilefordevelopment/wp-content/uploads/2018/06/GSMA_narrative_VF.pdf

© Roopa Gogineni / Panos Pictures

78% of our survey respondents identifying as belonging to the “Business” stakeholder group agree that there is a trend of consolidation in the Internet Economy.

Providing access via a walled gardens of content gets users online and could drive demand for a wider range of what the Internet offers (i.e., outside walled gardens). For example, Facebook’s Free Basics – a partnership with mobile network operators – gives users in more than 60 developing countries free access to content curated by Facebook, including news, employment, health, and local information.54See: https://info.internet.org/en/story/where-weve-launched/ But while these services are technically free, as no money is exchanged, users could be confined to one company’s platform, as many users cannot afford the costs of accessing data from other services or are simply unaware that other services are available.55e.g. Romanosky, J. & Chetty, M. (2018). Understanding the Use and Impact of the Zero-Rated Free Basics Platform in South Africa. CHI ’18. Available at: https://hci.princeton.edu/wp-content/uploads/sites/459/2018/01/ZeroRatedCHI-25.pdf; Sen, R.; Ahmad, S; Phokeer, A.; Farooq, Z.A.; Qazi, I.A.; Choffnes, D.R.; Gummadi, K.P. Inside the Walled Garden: Deconstructing Facebook’s Free Basics Program. Computer Communication Review. Available at: https://www.semanticscholar.org/paper/Inside-the-Walled-Garden%3A-Deconstructing-Facebook's-Sen-Ahmad/8d75e606e4b8b5ed90b189d5f42419421065e697 On the other hand, zero-rated initiatives can offer useful ways of serving areas and populations that have traditionally been underserved for profitability reasons. A study conducted in Ghana, Kenya, Nigeria, and South Africa found that zero-rating cannot only provide a gateway to the Internet for first-time and price-sensitive users, but when the practice is adopted by non-dominant mobile network operators, zero-rating can enhance competition.56Research ICT Africa (2016). Much ado about nothing? Zero rating in the African context. Available at: https://www.researchictafrica.net/publications/Other_publications/2016_RIA_Zero-Rating_Policy_Paper_-_Much_ado_about_nothing.pdf

© Internet Society / Nyani Quarmyne / Panos Pictures

Overall trends

To summarise, different market maturity levels, regulations, and institutions in each country significantly impact competitive market structures for access provision and how open they are to new entrants. These factors make it difficult to discern overall global trends of consolidation. Although national conditions vary, traditional access providers face pressure, particularly with the continued growth of mobile, a rising demand for data, and decreasing profit margins. In response, some operators share infrastructure, find ways to better use available spectrum, form strategic alliances and mergers and acquisitions, and seek higher returns in adjacent markets such as content and services. Examples of these actions include operators purchasing digital media, IT services or software players, with an eye toward vertically or horizontally expanding into sectors that could, over time, become parts of their core business.

At the same time, while platform-driven efforts to connect the unconnected are being driven by narrower interests than may be compatible with the porous and collaborative nature of the open Internet, the impact on local innovation and opportunity is unclear.

This raises questions about whether the access provision domain almost intrinsically tends to dominance due to the impact of economies of scale and other operating factors, and whether the entry of digital actors traditionally operating in other Internet domains (e.g., the app layer) might improve competition within 1 layer, while only entrenching their overall dominance cross-domain.

While the access markets differ considerably across the globe, to what degree will evolving access technologies bring the unconnected online faster? And, to what degree will the Internet platforms seek to gain a foothold in the access markets?

As the demand for streaming content grows around the globe, how will the networks that provide such services meet the bandwidth demands, particularly in those countries in which there are significant connectivity challenges?

"Do-it-yourself" transit?

Transit is how large communications providers interconnect to ensure data goes where it is needed around the world. The Internet depends on a fabric of business relationships that structure the exchange of data. For similarly sized networks, connectivity between them is often enabled via settlement-free peering, where no money is exchanged because the relationship has roughly equal value to both parties. Smaller networks must buy access to the global Internet from larger networks that market, sell, and operate transit service.

A small number of Tier 1 providers dominate the market for international transit. Every network that connects to the Internet is assigned an Autonomous System Number (ASN). Some of these Autonomous Systems (ASes) – usually very large networks – are known as Tier 1 providers. They do not need to buy transit from anyone else; rather, they sell it. A recent analysis of the interdependencies among ASes showed that a small group of Tier 1 providers play a significant role in connecting remote networks of the Internet.61Fontugne, R.; Shah, A. & Aben, E. (2017). “The (thin) Bridges of AS Connectivity: Measuring Dependency using AS Hegemony”  In: Beverly R., Smaragdakis G., Feldmann A. (Eds.) Passive and Active Measurement. PAM 2018. Lecture Notes in Computer Science, vol. 10771. Available at: https://arxiv.org/abs/1711.02805 The largest Tier 1 provider, Level 3, is estimated to serve nearly 53% of all ASNs in their customer cone.62See: http://as-rank.caida.org/asns; See definition of customer cone; http://www.caida.org/data/as-relationships/

Trends: Although a small number of Tier 1 operators dominate global transit, there is a flattening of the Internet’s hierarchy, as increasingly, networks interconnect via Internet Exchange Points (IXPs) and use Content Delivery Networks, thereby relying less on transit.

IXPs reduce access costs and latency by keeping more traffic local, and have seen significant growth in recent years, with the total number of IXPs increasing to 543 from 375 in the past 12 months.63 Packet Clearing House (n.d.). IXP Directory (reports). Available at: http://wwww.pch.net/ixpdir/summary Transit prices have been falling steadily, with some estimates indicating that global prices have decreased an average of 27% compounded annually between 2015 and 2018.64Boudreau, B. (2018) Outlook for IP Transit Prices in 2018. TeleGeography. Available at: https://blog.telegeography.com/outlook-for-ip-transit-prices-in-2018

These market trends are reinforced by the continued growth of Content Delivery Networks (CDNs) and large content providers investing in their own infrastructure for international connectivity. Google, for example, has laid its own oceanic submarine cables.65Sloss, B.T. (2018). Expanding our global infrastructure with new regions and subsea cables. Google (blog). Available at: https://www.blog.google/products/google-cloud/expanding-our-global-infrastructure-new-regions-and-subsea-cables/

InternetSociety-Content-Providers-Submarine-Cables

Data from TeleGeography66Mauldin, A. (2017). A Complete List of Content Providers' Submarine Cable Holdings. TeleGeography. Available at:
https://blog.telegeography.com/telegeographys-content-providers-submarine-cable-holdings-list

Overall, transit providers are under pressure from several directions. As IXPs keep more traffic local, economic and performance considerations will favour peering and non-monetised arrangements over transit arrangements. The evolution of CDNs using private networks to distribute content internationally also helps drive down demand for transit. Diminishing profitability of transit service provision may drive acquisitions and mergers intended to increase scale and reduce costs.

Content Delivery Networks (CDNs) – local hosting, global brands

© Internet Society / Nyani Quarmyne / Panos Pictures

CDNs are specialised networks of servers designed to cache and serve content in geographically diverse locations closer to the user. They store a copy of the content (e.g., cloud storage capacity or popular films) close to the edge of the network serving the content consumer. CDNs can increase reliability, reduce latency and also reduce demand for transit.

Trends: CDNs are now a fundamental infrastructure component for building high-performance, reliable web services. Some big content providers even operate their own CDNs (e.g., Netflix Open Connect67See: https://openconnect.netflix.com/en/). Among the top 1,000 websites globally, CDN use grew to an estimated 87.5% in August 2018 from 50% in June 2014. Of the websites in the sample that use CDNs, 27% use Amazon Cloudfront and 27% use Akamai. While some content providers make use of multiple CDN providers, the fact that 474 of the top 1,000 global websites use one of these 2 CDN providers indicates that they have significant market share.68Oracle Dyn (2014). Dyn Research: CDN Adoption By The Numbers. Available at: https://dyn.com/blog/dyn-research-cdn-adoption-by-the-numbers/

InternetSociety-Alexa-top-2018

Source: Datanyze69See: https://www.datanyze.com/market-share/cdn/Alexa%20top%201K/ (data last accessed 2018, August 1

Beyond the top 2 players, companies such as Microsoft and Alibaba are expanding their own CDN services, both organically and through acquisitions. Similarly, Alibaba is now estimated to have one of the largest CDNs in China after acquiring the online video platform Youku in 2016.70 Lee, E. (2016). Alibaba Finalizes Youku Tudou Buyout. Technode. Available at: https://technode.com/2016/04/07/alibaba-youku-buyout/

© Internet Society / Nyani Quarmyne / Panos Pictures

Cloud computing grows; as does big players’ market share

With the ability to access and manage servers remotely, from anywhere on the Internet, new businesses have emerged that specialise in renting out space and processing on their servers. Today, cloud computing services dominate this arena. They use large data-centres with expertise and economies of scale, offering their specialised services globally. Customers tend to access these resources as needed instead of buying and managing servers themselves.

Cloud computing essentially means the delivery of computing and storage services over the Internet, with companies offering these services called cloud providers. These providers offer functions as a service. These are functions to rent, including Infrastructure as a Service (IaaS), Platform as a Service (PaaS), and Software as a Service (SaaS). IaaS customers outsource basic computing requirements, such as processing and storage, to cloud providers, but manage everything else themselves, from operating systems to deploying applications. PaaS providers offer additional elements of the technological stack, including operating systems and functions like development tools, database management, and analytics. Large providers may even offer AI/machine-learning capabilities such as natural language-processing and image recognition71Dignan, L. Top cloud providers 2018: How AWS, Microsoft, Google Cloud Platform, IBM Cloud, Oracle, Alibaba stack up. ZDNet. Available at: https://www.zdnet.com/article/cloud-providers-ranking-2018-how-aws-microsoft-google-cloud-platform-ibm-cloud-oracle-alibaba-stack/, for instance. SaaS, in turn, largely covers applications discussed in Section 1.1 on Applications and Services.

Trends: Cloud computing is growing fast. The market for cloud computing services like IaaS and PaaS is expected to almost triple to $110.8 billion in 2021 from $41.9 billion in 2016.72Gartner (2018). Gartner Forecasts Worldwide Public Cloud Revenue to Grow 21.4 Percent in 2018. (press release). Available at: https://www.gartner.com/newsroom/id/3871416 The IaaS market has long been dominated by Amazon Web Services (AWS), with Gartner estimating AWS’s global market share of IaaS to be 51.8% in 2018.73Ibid. The IaaS market now appears to be consolidating around a small set of large providers: AWS, Microsoft, Google, IBM and Alibaba.74Gartner (2017). Gartner Says Worldwide IaaS Public Cloud Services Market Grew 29.5 Percent in 2017. (press release). Available at: https://www.gartner.com/en/newsroom/press-releases/2018-08-01-gartner-says-worldwide-iaas-public-cloud-services-market-grew-30-percent-in-2017 The current top 10 providers are expected to increase their market share even further, to 70% from 50%, by 2021.75Gartner, 2018, ibid.

The market for PaaS features the same players – AWS, Microsoft, Google, IBM and Alibaba – but also includes established companies like Oracle and Salesforce. As large investments are required in order to provide the full range of capabilities and resources, the IaaS and PaaS markets appear to be consolidating around the major players.

© Internet Society / Nyani Quarmyne / Panos Pictures

DNS services – growing concentration on all sides

© Internet Society / Nyani Quarmyne / Panos Pictures

The Domain Name System (DNS) translates reader-friendly domain names (e.g., example.com) into numeric Internet Protocol (IP) addresses. The DNS comprises many actors and functions, including registries, registrars, and root zone operators, but we focus here on two functions: recursive DNS and DNS hosting services. In both areas, we see trends of concentration towards a smaller set of global providers.

Recursive DNS services

Recursive DNS servers perform an IP address lookup in the DNS on the user’s behalf. They have traditionally been supplied by the user’s Internet Service Provider (ISP), but now new options are available, with public recursive DNS servers, available to anyone, increasingly being used. ISPs and enterprise administrators can outsource recursive DNS service to one of these public servers. Tech-savvy individuals may also choose to configure their devices to make use of public DNS servers, for example, to circumvent a local DNS-based censorship regime.76The recently standardised DNS over HTTPS technology (DoH – see RFC8484) may have some significant impacts in this area. The deployment of DoH is discussed later in this report. Hoffman, P. & McManus, P. (2018). RFC8404. Available at: https://www.rfc-editor.org/info/rfc8484

Trends: Since Google entered the market with 8.8.8.8, several other public recursive DNS providers have emerged (e.g., Cloudflare’s 1.1.1.1 and Quad9). Google’s public DNS is still considered the largest among these services, with estimates from APNIC indicating that nearly 15% of all Internet users rely on the service.77See: https://stats.labs.apnic.net/dnssec?s=Uses+Google+Public+DNS&d=Auto&w=30&t=40 (data last accessed 2018, September 18).

© Internet Society / Nyani Quarmyne / Panos Pictures

40% of our survey respondents see consolidation being a global trend, while 26.5% see it as more visible in certain countries.

DNS hosting services

Dedicated DNS hosting services are typically used either as part of a bundle of hosting services offered to smaller websites and individuals, or as more specialised DNS hosting providers used by large content providers and businesses, as well as cloud service providers with high-traffic domains, which need to avoid costly website downtime. CDNs use the DNS to direct traffic to content instances in order to improve reliability and latency for content providers.

Trends: A study of the top 1,000 domains using .com, .net and, .org indicates a trend towards consolidation in the DNS space with 4 providers: Dyn, Akamai, AWS, and Cloudflare. These providers had an estimated combined market share of 50% in May 2017.78Bates, S.; Bowers, J.; Greenstein, S.; Weinstock, J. & Zittrain, J. (2018). Evidence of Decreasing Internet Entropy: The Lack of Redundancy in DNS Resolution by Major Websites and Services. Working paper. https://dash.harvard.edu/bitstream/handle/1/35979525/DNS_NBER_Working_Paper.pdf?sequence=1 This is being driven both by the move away from self-hosting and toward cloud-based site-hosting and management platforms, which use the DNS for traffic engineering purposes, and also by consolidation in the broader hosting market. While website owners may benefit from vertical bundling of services and access to expertise, bundling also potentially impacts DNS diversity as single points of failure can reduce overall network resilience in the event of outages or other service failures. Single points of failure can similarly be exploited in attacks, such as the 2016 DDoS attack against Dyn.79 See, for instance: https://en.wikipedia.org/wiki/2016_Dyn_cyberattack

© Internet Society / Nyani Quarmyne / Panos Pictures

Overall trends

Transit is changing in ways that can reduce demand for traditional Tier 1 international transit. As IXPs are more widely used, CDN use grows and data is stored closer to users, and large platforms build their own infrastructure. Price pressure in international transit may drive market consolidation amongst Tier 1 providers as they try to deal with lower margins and the ongoing need for infrastructure investment. The changes to transit are mostly driven by developments in other areas of service infrastructure, particularly CDN and cloud-computing. These, in turn, are driven largely by the growth and investment by major Internet platforms and technology companies.

A small number of players in different parts of the technology stack increasingly dominate service infrastructure. The decentralised nature of the Internet is changing, and at some levels, more concentrated arrangements are beginning to emerge, particularly in DNS services and CDNs.

This changing topology of the Internet raises a number of critical questions:

What is the benefit or impact, if any, for competition and the open and interoperable Internet when the Internet platforms secure a significant presence across the application and services layers?

Does the flattening of the Internet change the fundamental characteristics of the global infrastructure – and in particular those of global reach and integrity, and interoperability and mutual agreement?