It’s finally here, your dream job as a Network Designer in a newly established local service provider. You received the acceptance e-mail yesterday, and today is your first day on the job.

After you take your time to explore your new workplace, you receive a call from your CTO asking for an urgent meeting. Hastily, you go and receive your first task.

“Helping your CTO convert business requirements to a conceptual architecture before translating it into an optimal design for IGP (Interior Routing Protocol) area design”.

You leave the meeting room after the [not so brief] meeting, mashing your head thinking about what should you do? what’re the first steps you should take to tackle that task?

Questions such as ‘which elements you should consider in your design’ are starting to arise, and you start thinking how to size the IGP area?

You’re probably a bit nervous at this point and thinking about running to the closest computer to devour information on what should be your next move.

Well, we’re here to help. This article was specifically written to help you (and any network designer in that matter) to organize your thoughts and find answers for the questions above.

First off, almost all existing large Service Providers and Telco use OSPF (Open Shortest Path First) or ISIS (Intermediate System-Intermediate System) as their main IGP protocols.

As for why they choose these two protocols? and how would you choose between them? we will outline it in detail in another blog post soon enough.

In our opinion, there are many areas you need to tackle to assure the optimal IGP design and area size, so the first thing you should do is to prepare a list of considerations that you need to take care of while designing your network, then you are ready for the meeting with the CTO.

The list came to life as per below:

 

First: Business Requirements

As a Top-Down network design approach, you need to be sure that it is aligned with the organization’s vision and goals.

And so, first, you need to understand the business requirements and goals. that includes their strategy and which services they are planning to carry over this IP core network

(please note: that services such as voice, or mobile calls can dramatically change your design because of their high requirements from the underlying network when it comes to delays, jitter and packet loss).

Another important point for future growth is that you need to ask yourself an important question. if your core Network will start with 50 Routers, what are the expectation in  3, 5, and 10 years respectively? Will you reach 500 or 1000’s routers after 10 years? How to assure your network is scalable?

Note that core area design, ISIS is easier to expand backbone area compared to OSPF

Scalability: Your designs need to be scalable, flexible, and adaptable to the most situation, you have to keep in mind that there are many types of end-users that your network has to cater for.

 

Second: Services & Solutions

As highlighted in the first consideration – Business Requirements and identifying the services running over this proposed network is a key for you Routing Domain design and size.

Consider new trends in networking, that includes entertainment, videos, IoT (M2M), virtual reality and augmented reality. All these applications are bandwidth hungry, and some of them requires real-time interaction. you have to keep in mind how to carry traffic from the edges of the network to end-users through your core network.

Other traditional Services such as L2VPN, L3VPN, and Voice also require special care of their requirements in regards to:

  • Availability
  • Delay
  • Jitter
  • Packet loss ratio

Third: Devices Capabilities & Feature Parity

 

The third consideration is the Routers’ capabilities. This includes (CPU, Memory, packet per second, line-cards architecture). This was the dominant consideration in the 80’s, and 90’s, then with the revolution in hardware which solved many of hardware limitations, this consideration is still there, but it’s not as critical as it was previously.

 

 

In the virtualized world we live in; we have similar limitations when it comes to the host where the “Virtual Router” resides in.

This virtual resource plays a key role in designing your routing domain, virtual resources to be allocated to the VNF (i.e. vCPU, RAM, and storage) must be carefully calculated; a good advice would be

allowing a room to upgrade these virtual resources seamlessly and smoothly without the need to shut down the VNF (this depends on the VIM/Hypervisor capabilities)

 

Fourth: Resiliency

When you design your routing domain and any other areas of your network, you should consider High Availability. This is tightly-coupled with SLA you’re committed to providing to your customers, which is getting higher and higher every year due to the tough competition between service providers and advanced hardware and solutions architectures.

Nowadays, it is common to say five nines SLA (99.999% availability per month or year), while some other service providers offer 100%!!

So, you need considering Redundancy on all your solutions pillars and layers, that includes – but not limited to – Hardware Redundancy (like power, fans, line cards, …), Roles and functions resiliency (deploying two ABR routers per area), Number of backup/redundant links between neighbors

 

This will lead to increase of the number of devices per routing domain. but on the other hand, it will assure better resiliency and SLA.

Fifth: Adjacencies

The fifth consideration is Number of neighbors within an area, you may have 100 routers within an area, but you have to plan and design it optimally to minimize the number of adjacencies.

Contrarily you may have 20 routers in an area but your design leads to a huge number of adjacencies, which affects routers resources (such as Memory and CPU) and increase the number of SPF runs in case of failures.

This also leads to exponential growth in signaling between these neighbors for routing and status updates.

A useful design tip in regards to the number of adjacencies is using point-to-point when possible, and if a full-mesh is required, you’d better use enhancement such as mesh-group, update throttling, and flood reduction.

 

 

Sixth: Type of Area

Another important consideration is area type, Mainly for OSPF. We have to be aware of the huge difference in routing information each router in a specific area need to carry if it resides in a normal, Stub, NSSA, or a Totally stubby area.

From a design perspective, it is always recommended to use stub, and a totally stubby area where you can; this leads to minimize number our routes within the area, hiding information thanks to summarization, and assure local changes/failures in other areas won’t affect your own area.

An exception to this recommendation is Backbone area (area 0) which must be a normal area to assure whole domain reachability and route exchange.

Seventh: Geographical Coverage

Seventh Consideration is your Network coverage, you need to think about and try to find answers to the below questions:

  • How big is your network?
  • Does it cover multiple countries?
  • What about international connections speed and feed?
  • What is the transmission (Layer-1 technology) used?
  • Does your company own the transmission and fiber?
  • What are your Service Provider PoP design and facility tier?

By answering the above question, you will be able to better understand your underlying passive network and assure transmission redundancy and capabilities.

Based on how big your network is, you can decide area boundary while considering the distance in order to avoid large delays within an area which affects the time to share updates and calculate alternate path using SPF.

Another recommended design principle is grouping routers with same capabilities and links bandwidth on the same area to assure slow routers won’t affect the performance of the whole area.

 

Eighth: Team’s skillset

A critical consideration of which many people believe it’s on the easier side of the equation is the Team’s Skillset and competence. That includes the Design team itself, the Implementation team, and Most importantly, the operations team.

You must consider people and process your design, and take care who can affect your decision

Some of the design I have done myself, I had to choose OSPF over ISIS for that particular reason.

 

 

 

Ninth: Links Efficiency

Finally, the last consideration in our list, which most designer just forget about it: is Link Efficiency and reliability.

This is solely depending on Transmission, Access technology (X25, FR, ATM, Ethernet, Dark Fiber, SDH, and/or DWDM).

Each of these Transmission and access technologies has its own Operations and Management Techniques (OAM), error control, transmission control, and so on.

Some of these technologies such as X25, adds vast amount of overhead to assure transmission control, and Error correction because usually link efficiency is not that good and could lead to numerous SPF runs, sharing a lot of updated between adjacencies à that’s why in such circumstances, it is better to go for small area with less number of routers (in magnitude of 10’s routers).

On the other hand, some other protocols add neglectable overhead, and it is very reliable (such as DWDM and ATM) in this case, we can easily use bigger areas (around 50’s to 100’s).

 

Examples

After covering the considerations to size your routing domain, here are two examples for the two widely deployed routing protocols:

 

ISIS

You can start with a single level-2 area (within the Core), this assures flexibility and eases the expansion by just adding L2 routers in the core, or introducing new areas connected to the core (L1/L2 as a border and L1 routers within the new area)

On Service Provider Network, the below guidelines can be used:

  • P / Core Layer: L2 routers
  • N-PE Aggregation Layer: L2/L1 Routers
  • PE/U-PE Access/Edge layer: L1 Routers

 

 

 

OSPF

As a common best practice, the number of routers should be between 50 – 150 maximum per OSPF area. Limit number of ABR per area to two routers.

And for each ABR it shouldn’t participate in more than 3 areas.

Plan number our routes/prefix per area to few hundreds. As we mentioned earlier always consider the use of stub/NSSA area where you can.

 

Conclusion

In this article I tried to explain the consideration you must include in your planning and designing phase of a Network Domain sizing and segmentation.

These considerations include non-technical ones: Business Requirements, Services, Coverage, and Team Skills.

Other considerations related to Link State protocol itself: such as Number of Adjacencies, Links within an area and links efficiency.

Finally, there are considerations related to the routing TIN itself (namely the router) which includes hardware capabilities, Resiliency, and area type.

What is the optimal number of routers per area, and how to size Network Domain; The Magic Answer is à “It Depends” J

This article provided some considerations of which you must study during your planning and designing phase, and also included some design tips and guidelines.

In future articles, we can address these design guidelines in details, so stay tuned !!