[ Watch on YouTube ]

The Wednesday Keynote Address was delivered by Greg Bell, Division Director of the Energy Sciences Network (ESnet), the advanced network serving more than 40 Department of Energy research sites, including all national laboratories. Bell gave attendees a meaty overview of ESnet itself, a new way to view advanced networks as instruments for discovery, and the relevance of these new outlooks to California. Before moving into his main topics, Bell took time to address the critically important relationship between ESnet and CENIC, and that he looks forward to an increasing collaborative relationship.

An Overview of ESnet

He then gave attendees a general overview of ESnet as a 26 year old mission network in support of the Department of Energy (DoE) Office of Science. They have roughly $5billion in grants each year, much of them in the physical sciences but bio- and materials sciences are becoming a larger part of the grants pie in recent year. As a research network, ESnet is, as Bell put it, "too big for a university to own," and he drew an historical parallel between ESnet and the UC Berkeley cyclotron, which expanded beyond the ability of a single university to manage it and was moved to a location where the Department of Energy could manage it. The Department of Energy often oversees such "too big for a university" facilities, and ESnet is considered to be one such facility. Its mission is to accelerate research and discovery for other DoE missions.

Bell then showed attendees a slide that demonstrated dramatically the importance of the relationship between ESnet and CENIC, specifying that California supports more national labs than any other state, including SLAC (home of LCLS and SSRL), the Lawrence Berkeley National Lab home of NERSC, ALS, the Molecular Foundry, NCEM, the Joint Genome Institute, and ESnet itself), and General Atomics (home of the DIII-D Tokamak).

As well as interconnecting the US national labs, ESnet also connects to hundreds of other advanced networks. ESnet itself is already optimized for massive data flows but is nonetheless feeling pressure to grow. They received $62million in stimulus funding and partnered with Internet2 to undergo a major transition to an optical infrastructure. Thanks to this, Bell stated that they "lit up" the first continent-scale 100G network in the fall of 2012. Readers can learn more at fasterdata.es.net and my.es.net.

Bell then delved more deeply into ESnet optimization for large data flows, explaining the difference between "elephants" and "mice," where "elephants" are defined as large, bursty data flows which are difficult to predict and do not "average out" in terms of network demands, unlike a million "mice," which do average out. ESnet has been, in his words, optimized for the elephants -- spiky, less predictable patterns of network traffic that can put sudden significant demands on the network. Elephants are also more powerfully affected by packet loss; where even a 0.0046% packet loss will not affect many mice, it will cause an 80x reduction in data transfer for an elephant at the kinds of distances that concern the DoE. This has caused ESnet to be designed for lossless operation, meaning more than ample network capacity, carefully chosen infrastructure, deep packet buffers, automatic and continual verification of network health, and the creation of "fast lanes."

And the elephants are getting bigger over time. One of the most impactful slides shown by Bell (see Fig. 1) was a graph of ESnet network traffic over the past 23 years, demonstrating what he called an "unrelenting growth in traffic." The following slides compared various predictions for traffic growth made by other study groups, and showed that the ESnet actual growth outstripped them all. Further it also showed that the growth in science-related traffic outstripped -- and continues to do so -- global IP traffic. Clearly networks that serve science "elephants" will continue to operate on the bleeding edge.

Figure 1: ESnet Network Traffic Growing Logarithmically

An example of only one kind of a new "elephant" was cited by Bell: x-ray beams. Beam resolution at the Advanced Light Source (ALS) at LBNL is increasing quickly, creating what Bell called a compounded Moore's Law growth rate as both frame resolution and frame rate increase. Four such new beams are being installed at ALS, and 40 such exist, with 8 to 10 Gb/s generated by each. Such beamlines are often used to image arts and historical documents as well, creating more "big data" disciplines in the social sciences and humanities and creating a whole new populations of elephants into new territories.

Bell also took time to state that most of the elephants wandered off of the ESnet network itself -- that 80% of all traffic on ESnet moves beyond the DoE complex of sites. (The number of these off-ESnet sites that are located in California further underscores the vital relationship between ESnet and CENIC.)

The Network as Instrument

Bell then moved into a second topic -- the consideration of the network as an instrument for discovery, as opposed to an infrastructure connecting endpoints at which discovery occurs. Three particular events function as historical inflection points in his view. The first is the creation of abundant network capacity, that extreme bandwidth has changed how networks are used. The second is software-defined networking, that the network can be a dynamic thing actively responding to the stresses placed on it. The third is the new campus cyberinfrastructures that have enabled the network endpoints to truly take full advantage of the network itself. These three advances, taken together, have made of the advanced network more than simply a more powerful version of its historical self, but have caused it to become an instrument for discovery.

Bell then described the discovery of a particle consistent with current understanding of the Higgs boson as an example. While the particle creation occurred at the Large Hadron Collider, the "discovery" itself took place on the multi-tiered global network of sites between which data is shared and analyzed. Bell pointed out that the just-in-time fetching of data required to perform these kinds of analyses and discoveries "represents greatly increased faith in global science networks." Another example is that of the Square Kilometre Array (SKA), highlighted by a quote from Roshene McCool stating that, "Networks are at the heart of the telescope." In Bell's words, the question of, "where does discovery occur is a little difficult to localize." The SKA in particular also highlights the creation of a new breed of network "elephant"; Bell stated that the data rates from this one project will exceed the current total capacity of the commercial Internet.

Bell then turned to California examples, such as the UC Santa Cruz Cancer Genomics Hub (a 2013 Innovations in Networking Award winner), the discovery of the August 24, 2011 supernova -- the youngest to be "caught on film" and caught through a California workflow called the Palomar Transient Factory, Photosystem II at SLAC, and the ESnet/CENIC enabled connection of the SDSC Gordon supercomputer and NERSC's Edison. All are examples of entire distributed discovery facilities which rely on advanced networks to function as their central nervous systems. The new model put forth by Bell for the network(s) that underlie discovery today is shown below in Fig. 2.

This view, in his words, "inspires us" and prompts network specialists and researchers to enable new workflows, offer APIs for many functions, and decouple data acquisition, storage, and computation from one another -- finally realizing ESnet's vision of "a world where discovery is completely unconstrained by geography."

Figure 2: Network as Infrastructure or Instrument

Faster Data for California

The third part of Bell's keynote address focused on faster data for California, and some of the ways in which this has been and can be enabled for California institutions. He touched on multiple topics in this area, including the building of a community knowledge base, the creation of an abstract Science DMZ design pattern for data transfer and specific tools to move this model forward such as the Globus Online data transfer tool, the design of a typical Science DMZ, and the use of perfSONAR for network monitoring.

Bell then gave the audiences a glimpse into the future for ESnet and its coming priorities, which include greatly increasing partnerships, science engagement, and outreach to the facilities it serves. These will encompass tactical and strategic objectives respectively with the creation of "tiger teams" that will engage facilities in discussions about data mobility, and the targeting of facilities and collaborations to help build a foundation for lasting impact. ESnet is also pursuing the creation of real-time network discovery and visualization tools (the public face of which is my.es.net) as well as looking ahead to the world beyond 100G.

This last is particularly important as the projection of the historical traffic graphs imply that the current optical platform will be exhausted by 2020.

Bell closed his address by reminding attendees that these challenges cannot be met by ESnet alone, and that the relationship between it and CENIC will play a big part of realizing the future vision.

About CENIC and How to Change Your Subscription:

California's education and research communities leverage their networking resources under CENIC, the Corporation for Education Network Initiatives in California, in order to obtain cost-effective, high-bandwidth networking to support their missions and answer the needs of their faculty, staff, and students. CENIC designs, implements, and operates CalREN, the California Research and Education Network, a high-bandwidth, high-capacity Internet network specially designed to meet the unique requirements of these communities, and to which the vast majority of the state's K-20 educational institutions are connected. In order to facilitate collaboration in education and research, CENIC also provides connectivity to non-California institutions and industry research organizations with which CENIC's Associate researchers and educators are engaged.

CENIC is governed by its member institutions. Representatives from these institutions also donate expertise through their participation in various committees designed to ensure that CENIC is managed effectively and efficiently, and to support the continued evolution of the network as technology advances.

For more information, visit www.cenic.org.

Subscription Information: You can subscribe and unsubscribe to CENIC Updates at http://lists.cenic.org/mailman/listinfo/cenic-announce.