Post-Event Report on:

The 13th edition of Nigerian Network Operators’ Group (ngNOG) Workshops and Meetings.

Venue:

Lagos, Nigeria

Dates:

29th October – 2nd November 2018.

Theme:

AFFORDABLE BROADBAND FOR THE NEXT 100 MILLION NIGERIANS.

Executive Summary:

The ngNOG Workshops & Conference is Nigeria’s foremost capacity-building and human networking activity for ICT personnel and other Internet stakeholders.

Organized by the Nigeria ICT Forum annually for the past thirteen years, it has enabled the training of over 1000 network engineers, managers and librarians in network operations, systems administration, library systems, content management systems, power systems, and management policy.

With a large and growing number of stakeholders, it also serves as a platform for collaboration and sharing of experience within the Nigerian communication technology ecosystem. Internet companies seeking to explore opportunities with Nigerian entities also see it as a one-week one-stop location to have productive engagement with a wide variety of stakeholders.

Sponsors of the Conference made it possible to host the local community in a conducive environment without charging an entry fee. Special thanks are due to Facebook, The Internet Society, Google, Netflix, DE-CIX and AFRINIC for supporting financially and in kind. Other sponsors and supporters include MDXI (Main One), Medallion Communications, The Nigerian Communications Commission(NCC), Internet Exchange Point of Nigeria (IXPN), Meed Networks, NisTech, Nigeria Internet Registration Association (NIRA), KKONTech, the Bandwidth Consortium, the team of volunteer instructors, presenters, panelists and moderators.

The annual ngNOG week usually includes a Conference day (during which all workshop tracks are paused). Panel discussions are held on issues critical to the operation of networks and delivery of Internet services in Nigeria.

The 2018 ngNOG conference panel sessions touched on successes and challenges in: Multimedia Content distribution

In-Country Web Hosting services

Quality of Service Provision to end users 

Broadband Policy regime and impact 

Strengthening of Internet service providers

Strategic Partnership between local Content providers and ISPs 

Implementation of best practices to improve routing Security (MANRS) 

Deepening broadband penetration by peering at local exchange points

Awareness on relevant global Internet policy discussions

There were also presentations and lightning talks aimed at enlightening and sharing experiences in operation of reliable Internet networks.

A networking cocktail enabled more interaction among delegates in the evening right after the conference. Nice music and delicious refreshments were in abundant supply. The day’s events played host to some 411 participants from 11 countries and 184 organisations.

The session recordings are available at http://nog.ng/resources

In addition to the Conference, ngNOG 2018 had hands-on technical workshop tracks

running simultaneously as usual. The tracks included

• Internet Services
• Internet Infrastructure
• Domain Name Business (in collaboration with NIRA), and
• Systems Administration (merged with Content Development).

The 2018 workshops also included two specialty tracks –

• Peering and Interconnection (in collaboration with ISOC and IXPN) – to increase the interconnectivity levels and access to locally available content by network operators, and
• IP version 6 Deployment Hackathon (in collaboration with AFRINIC) – to assist network operators who were willing to enable wider use of globally routable IP addresses on their networks.

The 2019 ngNOG conference will re-introduce a full day of tutorials during which delegates will learn more from their peers and case studies from the Internet industry. This will further increase the value gained by attending ngNOG.

The following pages go into more details about the 2018 conference and workshop sessions.

WELCOME ADDRESS

The conference commenced with an opening address by Dr. Adewale Adedokun, the National event Coordinator who gave a brief history of ngNOG and appreciated all the volunteer instructors drawn from various higher education institutions and private organizations. He also appreciated the participants and the ngNOG community at large who have kept the event active over the years. He thanked all the sponsors and partners for their continuous support in training technical staff from member organizations over the years.

Dewole Ajao, President of the Internet Society Nigeria Chapter ( https://isoc.ng/ ) highlighted the agenda while stressing that the most important part of the conference was for attendees to interact well and share experiences during and outside the sessions. All the sessions were targeted at enabling reliable broadband access to users across the country.

KEYNOTE

The President of Nigerian Internet Registration Association (NIRA, https://www.nira.org.ng/ ), Rev. Sunday Folayan, delivered a keynote address in which he emphasized the need to recognize the importance of ICT and re-align Nigeria’s strategic ICT direction through the steady pursuit of strong policies and programmes to drive socio-economic development if we are to achieve our goal of connecting the next 100 million Nigerians. He concluded by encouraging all developing partners to hold steady and keep the faith as Nigeria is on the right path to building a community of people who can work together.

PANEL SESSION 1

OFFLINE AND ONLINE CONTENT DISTRIBUTION

Moderator: Chidi Uhuru(Big Cabal Media)

Onsite Panelists: Mo Feleta (AVIVA), Olakunle Ogungbaila (Iroko TV)

Remote Panelist: Nina Bargisen (Netflix)

Session Summary:

The panelists gave an overview of their various businesses factoring online and offline content distribution in regards to cost and content development. Iroko TV stating the use of user data and feedback to create content. Discussions were on content development based on data generated by users and suggestions were made in making users part of the process of content development to make user see the use of the content when it is Hosted on the Internet. It was proposed that collaboration between the content providers and the Telecommunications operators could enable the consumer to get the contents easier, cheaper and faster. Content providers present were also advised to host their contents locally as the risks have been mitigated not forgetting that their users and customers are in Nigeria.

SESSION 2

THE CASE FOR LOCAL HOSTING OF CONTENT

Moderator: Mr. Seun Odedeji

Onsite Panelists: Bimbo Ayo (Whogohost), Akinbo Adeola(ISOC), Olatunji Suleiman (Medallion), Vremudia Oghene-Ruemu (MDXi/MainOne)

Session Summary:

The panelists gave an overview on local hosting of content. Whogohost partnership with MainOne makes them enjoy the advantages of hosting their content locally which can be identified as flexibility, security of Information and also foster seamless transfer.

When local content meets a certain standard it becomes global, however content not hosted locally, becomes a market, basically giving out the value of the data to whoever is hosting the content. Hosting content locally also increases speed when it comes to bandwidth usage.

Local hosting is key to data privacy and control. It has encouraged to host locally especially when sensitive data is involved.

MDXI a subsidiary of MainOne is focused on data centers services and interconnection. A key thing about local data hosting is taking advantage of the economics of between internet transit and peering. Nigeria has a large population which a business scale for any venture in technology. Data retrieval for content hosted outside Nigeria incurs a higher cost which as always end up on the end user. Having contents closers is a great business case for local hosting as quality local contents are now available. Local content is key to developing a nation.

You should tell your story yourself by hosting locally.

PANEL SESSION 3

LAST MILE TO THE CONSUMER

Moderator: Mr. Dele Amire (Nis-Tech)

Onsite Panelists: Femi Omoniyi (Fibre One), Florence Lavroff (Google), Ifeanyi Okonkwo (Tizeti)

Session Summary:

The panelist which consist of Service providers explored some of the challenges and successes they face on reaching consumers.

Some challenges identified are language barriers, inability to focus on end users, lack of hosting local content and the high cost of data. Florence stated Google’s priority is

the end users, however most of the challenges faced cannot be isolated from the others and all have to be addressed at the same time which itself is a challenge.

Google has successfully brought Edge closer to Lagos as against previously being hosted in places like Western Europe. This impact is easily felt by ISP and users in the difference in cost of content.

Google is also collaborating with Local ISP to host Google Cache so content are fetched locally. And there are also working with the Internet Exchange Point to bring content closer to end users.

They now have a new POP located with Main One in Lagos and it will be far more efficient and effective to Peer with them at the Lagos POP.

Fibre-One – though being quite young in the Fibre sector some challenges faced are right of way, cable theft, cable cuts.

Tizeti has been successfully brought access to the internet to about 10,000 customers, provided job creation to over 200 youths. One major challenge faced is acquiring licenses, DNS challenges and working towards hosting servers locally. Through a partnership with Facebook, they have about 900 hotspots around Lagos which goes for as low as N50 for 100Mb.

Tizeti currently building their infrastructure starting from Lagos and are already heading towards Ogun state. Each access point is aimed at providing to a particular niche of the market. They plan on building infrastructure In other parts of the country.

Session ended with the need for more collaboration between ISP’s to lower cost to end users and the need to look beyond internet reaching the end users but also access to content as the internet is becoming the social necessities.

PANEL SESSION 4

BROADBAND POLICIES AND OVER-THE-TOP SERVICES

Moderator: Dr. Adedokun Adewale

Onsite Panelists: Funke Opeke (MainOne- Operator), Henry Ojiokpota (Nigerian Communications Commission (NCC- Regulator), Prof. Muazu (Ahmadu Bello University- End User)

Session Summary:

This session started by a goodwill message from the EVC NCC through his representative Dr Henry, who stated NCC ’s commitment to improving internet services, local content hosting and IPV6 deployment. The EVC also commended the Nigeria ICT Forum for its commitment to ICT development.

Mrs Funke Opeke stated Nigeria does have a five year policy framework referred to as National Broadband policies since 2013 although some part of the policies are yet to be realized. Thus policies may not be the major issues affecting accelerating broadband penetration, affordable internet or quality of access.

One area of challenge is difficulty in deploying infrastructure required to facilitate broadband in Nigeria since the 2015 recession. Some major issues limiting access on the supply end is, the macro-economic situation has made it increasingly expensive to deploy. Secondly is the challenges (security, logistics, absence of power) of doing business in-country adds to the overhead and the efforts required makes it more difficult for investment of the required capital. This had caused concentration of infrastructures in the same high network urban areas thus people at the fringes are underserved in terms of access.

INFRACOS are meant to help resolve this imbalance but unfortunately, they are also faced with the challenges of accessing permits to deploy infrastructures from State governments. This requires policies intervention to enable INFRACOs gets permits at reasonable rates. She however appreciated NCC efforts on these issues. Opeke ended with a call for a better sharing in terms of already built infrastructure and open access framework.

Prof. Muazu stated that for the end user their major concern is the cost of last mile. End users are mainly pushing for policies that would reflect a difference in cost and access.. He also called an action to deal with certain fundamental issues which reflects a drop in prices for the end users.

Dr Henry stated that NCC promotes accessibility, affordability and availability of services to end users. He said that the INFRACO model was put in place to enable services to get to the end users at affordable rate.

These models are aimed at the six geo-political zones, two zones have already been completed and licensing is also in process. The commission is also implementing the National roaming and infrastructure sharing model which will enable operators provide service in areas where they don’t have existence network. NCC also has the University intercampus connectivity.

He further said NCC promotes strategic partnership as they operate an open door policy for all the operators, however these collaborations can be limited by certain commercial issues.

SECURING THE INTERNET THROUGH MANRS

Speaker: Michuki Mwangi (Internet Society)

Session Summary:

Michuki started by stating MANRS is specifically about routing security. Traffic movement on the internet is largely based on trust which involves trusting how packet moves from one network to another.

Generally, we don’t verify identities of networks due to how the internet was built and this has led to some challenges of routing outages which has affected network availability to end users.

These routing challenges/outages on average could last up to 19 hours, and these are issues difficult to resolve as it involves a lot of processes and collaborations to identify and resolve. These challenges can go as far as affecting an entire country.

Though routing security challenges may be considered as a minor issue, it has a significant impact on end users and how we experience the internet. Thus the importance of how traffic is routed from a security point of view- MANRS.

Three main threats have been identified which Prefix or Route hijacking, IP address spoofing. However these can be easily resolved through collaborations. The internet is designed is such a way that it is easily affected by “tragedy of the commons” which simply translates to best impact is achieved when solutions are practiced by all involved.

MANRS simply serves to encourage all network service providers to implement best practices to help address internet routing securities.

Four Main actions are required to help fix the immediate issues like IP spoofing and also prevent potential problems. These actions are filtering, anti-spoofing, coordination and global validation. Every Internet user benefits with MANRS for better routing security and a more stable internet.

Unfortunately, only three networks currently in Africa has subscribed to MANRS actions. Thus the need to for more enlightenment on MANRS for the benefit of all.

Guidelines and online trainings on MANRS are available https://www.internetsociety.org/issues/manrs/

FACEBOOK – BEING OPEN

Speaker: Ben Ryall (Facebook)

Session Summary:

The presentation was about Facebook, its edge and how they deliver traffic to the people using their various platforms. Ben talked about the traffic pattern and how they try to move traffic closer to the edge. He went on to explain the Global Controller Architecture which is used to achieve closeness of friends on Facebook.

There were some suggestions of implementation from the user/local network side that will improve user experience such as Caching, local and regional peering. Facebook always works to ensure the security of all users.

He ending by inviting group and individuals to schedule more detailed talked with the Facebook team, as they are available to hear comments and ways to provide better services.

CONNECTING THE NEXT 100 MILLION – THE ROLE OF IXPN

Speaker: Muhammed Rudman (Internet Exchange Point of Nigeria (IXPN)

Muhammed Rudman has been an advocate of local content and appreciates the event especially the audiences’ persistent demand for creation and hosting of local contents.

Rudman stated that the internet exchange points are the physical locations where multiple networks interconnect. The IXPN handles critical infrastructure and contents for service providers.

The IXPN is active in five geo-political zones in Nigeria and the north-east zone being the sixth zone is underway.. The idea is to localize traffic across the country and joining the IXPN is open to all.

IXPN started in 2008 with a traffic of less than 1MB by 2012 to date it has increased to about 120GB/second this translates to about a ten thousand percent increase within

five years. Currently over 50 different service and content providers are connected to the IXPN.

The cost of internet has significantly dropped in Nigeria based on the cost of ports IXPN offers, this is also made possible due to presence of players like Google, Facebook, Akamai.

Google will also host their cache at the exchange point in Abuja and possibly other locations.

It was also announced by Mr Rudman that the Nigeria Internet Exchange Point has won the bid to host the West African exchange point with a grant from the African Union has been given for affordable internet.

AFRINIC SERVICES UPDATE

Speaker:Madhvi Gokool (AFRINIC)

Madhvi Gokool emphasized on the need to update AFRINIC details to avoid service interruptions and to avoid membership account closure. Members are required to manage their resources via the myafrinic , whois web client and main client interfaces. AFRINIC continuously supports internet exchange points in its service regions. There are also membership discounts enjoined by exchange points (100%) and educational institutions (50%).

It is also imperative to follow Afrinic Policy discussions as it affects all network operators which affects the use of number of resources in the region. Some Afrinic policy proposals ranges from the Inter-RIR resource transfer, the clarification on IPV6 sub assignment , the abuse contact policy update etc. More details on AfriNIC services available https://www.afrinic.net/fr/services

CLOUDFLARE

Speaker: Martin Levy ( Cloudflare INC)

Martin Levy stated that cloudflare has plugged in their CDN node in Lagos connecting it to the internet exchange point of Nigeria.

Cloudflare has an public DNS resolver using the IP address 1.1.1.1 as a global service. The address was generated from APNIC. The DNS server can reduce the amount of information and secure capacity on data policy if the client’s IP address is not stored.

DNS was never a protocol built with privacy in mind. One of IETF standards is to understand how to build a secure encrypted first hub capability. Cloudflare provides that on the 1.1.1.1 system.

He said that the IPV4 1-1-1-1 , with a back up no of 1-0-0-1 are easy to remember and tend to be used in different environments. Unfortunately it’s not available in V6. Only about 7.13% traffic useful adoption of 1-1-1-1 has been granted and it was announced in April 2018. The downsides of using popular IP address is that it tends to route leaks. Here BGP security is quite relevant here.

In general, it is easy to remember a fast IP address. It is built for privacy, as a service for the internet which was quite an interesting project for CloudFlare. More details are available https://www.cloudflare.com/learning/dns/what-is-1.1.1.1/

Professor Oluleye Ayodeji gave the closing remark and vote of thanks, and was happy to see the seed sown many years ago blossoming. He thanked all the sponsors for their participation. He concluded by encouraging everyone who participated To ensure that the said objectives of the event are met.

The Internet Society Nigeria Chapter successfully held its AGM towards the end of the event. A lucky participant Mr Dele Amire won the Oculus Go Virtual Reality VR via a raffle draw courtesy of the Facebook team. The event concluded with a networking cocktail.

ngNOG Workshops

TRACK TITLE- Internet Services

Instructors:: Cletus Okolie – AMCON , Adewale Oronti – Federal University of Technology Akure

Track started with a combined session on Virtualization and containerization with the System Administration and Application Content and Development Track. Installation of Apache Web server, creation of Virtual hosting web servers; SSL certificate and its implementation was also practiced. The web session was followed by Freeradius setup for authentication over Mysql. Participants were also shown how to add a radius client to the AAA server and vice-versa. A radius web-frontend was later implemented to make management free radius server easier when participants get back to their respective places.

TRACK TITLE- Internet Infrastructure

Instructors:: Dele Amire-NISTech , Auwal Tata-Bayero University Kano, Emmanuel Onowojo-University of Benin, Mr Seun Oyedeji Federal University Oye-Ekiti

This track was carefully designed to empower participants with skills to select the appropriate Layer technologies, given specific requirements. Also to help participants understand principles of network architecture (Core, distribution & edge networks, redundancy & scalability), carry out network monitoring, optimization and capacity planning, Secure a network from intrusion and abuse, Produce an IP plan for their institutions, Interconnect with other networks (EGPs)

Setup and manage gateway security (firewalls, Intrusion Detection and Prevention, UTM) and Carry out Service monitoring (NAGIOS, cacti, SMOKEPING).

TRACK TITLE- Application & Content Development/ System Administration Instructors:: Christiana Olayinka-Yaba College of Technology , Christopher Anthony- Ahmadu Bello University

The Application & Content Development track (ACD) was designed for application developers , webmaster, database administrators, library staff who are involved in open distance learning management system, those deploying and management of web services in their institutions. This year the track was combined with the System Administration track which present real-life Unix experience to administrators with an objective to let participants perform various hands-on system administration activities.

TRACK TITLE- Domain Name Business

Instructors:: Muhammed Rudman, Bar Emmanuel Edet, Dr Eugene Ohu, Mrs Edith Udeagu, Mr Shola Akinsanya, Mr Moses Ajayi, Mr Toba Obaniyi, Mr Kemi Adepoju, Mr C Fred Agbata(Jnr), Mr Salvation Alibor

This track was sponsored by the Nigeria Internet Registration Association (NIRA) which covered content distribution, domain industry and web hosting. The track focuses on helping participants understand the business side of Internet services, common tools for managing and supporting web hosting and domain name businesses as well as the roles of global Internet coordinating bodies. It pays particular attention to the domain name system, how to acquire and resell.

TRACK TITLE- Peering and Interconnection

Instructors::             Michuki Mwangi-ISOC, Jacob Dagunduro-IxPN, Ghislain Nkeramugaba-Rinex/ISOC, Anibe Onuche-IxPN

The first of its kind in Nigeria was an opportunity to interact, learn from one another and share experiences helpful to network operations and businesses. The track had in attendance 36 participants from 20 organizations ranging from ISPs, MNO, Content Delivery Networks, and Data Centre operators both local and international.

Participants in the track were exposed to the benefits of peering and interconnection which ranges from Lowering the cost of transit, Users end up with a better experience

as they have a direct path between two networks, and how peering increases redundancy and resiliency. It covered interactive discussions on peering economics, peering policies, Routing security.

Most participants were for the first time aware of the work being done on the Mutuals Agreed Norms for Routing Security (MANRS) which is fast becoming a globally accepted standard on routing security.

They also learnt on the inner operations of an Internet Exchange Point (IXP) in general and using the Internet Exchange Point of Nigeria in particular. Participants got to know how to contact and connect to the IXPN as well as prospective peers. The track also served as an avenue to network with some major global internet players like google, facebook, cloudflare and other service providers.

Upon completion of the two day training, the University of Jos participant (Mr Anaobi Ishaku) won the fellowship prize for the Africa Internet Summit 2019 in Kampala after updating all the route objects of his organization in AFRINIC database. Courtesy of ISOC

Also up for a grab is a fellowship to next African Peering and Interconnection Forum AFPIF event for the first member network to become MANRS (Courtesy ISOC)

An Oculus Go Virtual Reality VR to first member network to implement IPv6 and peer with the exchange wins (Courtesy Facebook)

Appreciation

Our sincere appreciation goes to the various organizations who participated in the conference as presenters, moderators, panellist. To our sponsors and partners who made the event a success.

A very big thank you to the ngNOG instructors, secretariat members, interns , event organizers and most importantly the attendees of both the conference and workshop