The LONAP AGM 2019 was held at IET, London on 14th January 2020.

The Agenda was as follows:

• 15:00 Introduction
• 15:05 Review of the year [RI]
• 15:25 Technical update [WH]
• 15:45 Technical update [IC]
• 16:00 Start of AGM [TD/PT]
• 16:30 Technical update [RL]
• 17:00 Euro-IX Database [AD]
• 17:15 LONAP Social

It’s pretty much the same agenda each year with the occasional modification, but why change a winning recipe?

Review of the Year - Richard Irving

In brief it is everything up and to the right. The exchange peaked at 507 Gbps and ended the year with a total connected capacity of 4.36 Tbps, up from 3.21 Tbps a year ago. We saw a net growth of 10 members. Turnover increased from a run rate of £839k to £958k year on year.

Will Hargrave talked about scaling

In his talk Will discussed the work we have been doing to scale the LONAP network. He began by discussing the new bandwidth peaks already mentioned by Richard and moved on to discussing the work that has been done on Intersite Link Sizing.

Link sizing and demand is not the easiest thing to forecast. These links must deliver traffic demand even during fault conditions. The sum of per-site connected capacity is not very useful due to the wide variation in port utilisation.

In 2019 we moved to a dual-core network with ECMP multipathing. All Telehouse and Harbour Exchange locations are now connected with minimum dual 2x100G. Demand at Interxion and Sovereign House has been historically lower and these sites remain connected with 8-12x10G links. During 1H 2020, these will move to multiples of 100G. In January 2020 we will be adding a second link from Equinix Slough to Telehouse North with 2x100G initially.

2019 LONAP core diagram:

ECMP of traffic via the core nodes in Telehouse North (THN) and Telehouse West (THW) is working very well. Fortunately our more distant sites are not so far geographically. Equinix LD8 (HEX) will be added as a core site during 2020. Non-Telehouse sites (HEX/EQS/INX/SOV/etc.) will connect to new cores in HEX and THN. Care needs to be taken to ensure traffic is balanced.

The resultant 2020 core design is very elegant in its simplicity.

It is worth looking at the timeline for 100 Gbps+ connectivity in the metro area.

Pre 2016 we had n * 10GE links scaling to around 160G in our current setup. This had difficulties with “too many moving parts” at this scale.

2016 saw the lighting of 100G on dark fibre pairs/ 1310nm waves. Still had questions over scalability.

2018-19 there was still no straightforward 100G DWDM option. We investigated Coherent transponder solutions and purchased Smartoptics / Inphi “ColorZ” PAM4 solution. This was price-competitive compared with additional dark fibre in Docklands metro.

The Inphi “ColorZ” offers 2x50G PAM4 λ encoded in a single 100GHz DWDM channel with standard QSFP28 in switch. Arista EOS supports DOM extensions to instrument on-optic FEC DSP.

Optical specifications include output power of -8dBm and min input power of -2dBm, dispersion tolerance +/-6 km on G.652 fibre. A dedicated optical line system is required to address these challenges.

Many of our links are in the <10km range. Where a line system is not appropriate we have been replacing existing 100GBase-LR4 optics with 4WDM-10 (purple tab). A CWDM4 grid with on-switch RS-FEC is optically superior to standard 100GBase-LR4 and a little cheaper. There are few other immediate options to reduce price in this area of the curve.

The DCP-M8-PAM4 box has:

• support for 8 x ColorZ PAM4 wavelengths
• Automatic fibre distance measurement and dispersion compensation setting
• Complete zero-touch automatic optical setup, just like using a passive multiplexer
• Optical performance, 0-80km, 0-20dB PAM4 Only

For the future where metro connectivity is concerned, technologies developed for 400G may be useful here. Pricing is very competitive - a 100GBase-LR (single λ 100G, 1311nm, 10km reach) is already price-competitive with “standard” 4-λ 100GBase-LR4. 400GBase-LR4 (4x100, CWDM grid, 10km reach) optics are entering the market priced at 4x price of 100G. This is good news. Nothing we can use yet, but closely monitoring this fast-moving situation.

From a switching platform perspective during 2020 we are likely to reach 100G port density capacity on existing Arista 7280QR platforms. The Core (Tomahawk 32*100G) has spare ports. Most LONAP edge is original Jericho, launched in 2016 with plenty of 10G ports spare.

In 2020 Broadcom the Jericho 2 platform starts volume deliveries with switching capacity of 1080G -> 4.8T. In Arista-land, these are the 7280R3 boxes of which the most useful form-factor for LONAP is likely to be the 32x100G+4x400G variant.

In conclusion, Will expressed the sentiment that all was going well. We continue to scale the platform. Faster switches and choosing “right-sized” optical solutions are key to meeting member growth requirements.

Ian Chilton talked about DevOps activities

Ian joined us in 2019 and has already been making a great contribution towards updating systems and automating processes. Improvements so far include:

• Staff VPN endpoints and “jump boxes” for privileged access to our internal services and infrastructure.
• Self hosted GitLab instance for source code hosting and CI.
• Configuration management - repeatable deployments and provisioning of our hosts & services with Ansible.
• VLANs for separation of internal services, external services, those with “third party access”.
• VM Hosts Overhauled.
• Route Servers Redeployed.

VM Hosts

Quite a lot of much needed activity here. Two new hosts have been added. Existing hosts have been re-provisioned and some very old hosts upgraded and moved out to Slough for development, testing, backups etc. These are provisioned with Ansible and have 1 Gbps and 10 Gbps connections to both management (OOB) network and main peering LAN. Guest (VM) installations are now automated.</p>

Route Servers

Our Route Servers were in need of an overhaul. Although functioning fine there had been one occasion where we saw a crash when a peer with many thousands of routes connected. The old hardware struggled to cope with this and newer versions had since been released (BIRD v1.x). The Route Servers had been manually installed, configured, updated, tweaked over many years. There was no support for BGP Large Communities (RFC8092) and no support for RPKI.

Clearly there was some Route Server work to be done. We now have VMs on two separate and redundant VM clusters. We are running the latest stable release of Debian Linux (buster) and the latest release of BIRD v2.x all deployed using Ansible configuration management tooling. We also have BGP Large Communities support together with RPKI support.

LONAP AGM

The formal LONAP AGM started at 4pm.

The LONAP AGM was conducted by Chairman Trefor Davies and Director with responsibility for Finance, Pete Taphouse. AGMs are short affairs and follow a set pattern. The minutes for the 2018 AGM were approved. The financials were presented by PT and approved by those present. It was noted that we had moved to using the Xero accounting package which had resulted in some of the accounts being presented differently in particular in respect of allocation of expenditure.

The last bit of the AGM was the Board Election. This year Jody Botham was standing down due to work commitments of a new job and we had two people standing for this position. We are very fortunate in having some great people stepping up to the plate and this year was no exception. Ronan Mullaly (Akamai) and David Croft (Sargasso) gave short speeches and David Croft polled the most votes. We are extremely grateful to Ronan for putting himself up as a candidate and offer our sincerest thanks to Jody for the work he put in during his time on the board.

Rob Lister then gave his technical update

Rob covered support activities, an update of ticketing types and numbers. He also gave us some Route Server statistics (over 85k routes and around 80k networks reached) and described our work with MANRS (Mutually Agreed Norm for Routing Security). This includes coverage of areas such as Filtering, Anti-Spoofing, Global Validation of data and coordination between network operators globally. Rob’s content was much more detailed than provided here.

Guest Speaker

Andy Davidson of Asteroid and the Euro-IX board gave a talk on the Euro-IX database (IXPDB) before we all decamped to the Coal Hole pub for food and refreshments.