SIG: Solution to TCP Incast in SDN Network Based Openflow Protocol

Jiang Xu,Hongwei Guo,Jihuai Wu,Jenshan Lin,Dongxu Zhang,Gang Chen,Xingping Zhang,Chao Chen

DOI: https://doi.org/10.1364/acp.2013.aw4i.5

2013-01-01

Abstract:TCP incast refers to the drastic goodput collapse in many-to-one communication prototype. We propose and demonstrate a solution named SIG (sending in group) to address incast in SDN network, and stable goodput is achieved.

What problem does this paper attempt to address?

Taming TCP Incast Throughput Collapse in Data Center Networks

Jiao Zhang,Fengyuan Ren,Li Tang,Chuang Lin

DOI: https://doi.org/10.1109/icnp.2013.6733609

2013-01-01

Abstract:The TCP incast problem attracts a lot of attention due to its wide existence in cloud services and catastrophic performance degradation. Some effort has been made to solve it. However, the industry is still struggling with it, such as Facebook. Based on the investigation that the TCP incast problem is mainly caused by the TimeOuts (TOs) occurring at the boundary of the stripe units, this paper presents a simple and effective TCP enhanced mechanism, called GIP (Guarantee Important Packets), for the applications with the TCP incast problem. The main idea is making TCP aware of the boundaries of the stripe units, and reducing the congestion window of each flow at the start of each stripe unit as well as redundantly transmitting the last packet of each stripe unit. GIP modifies TCP a little at the end hosts, thus it can be easily implemented. Also, it poses no impact on the other TCP-based applications. The results of both experiments on our testbed and simulations on the ns-2 platform demonstrate that TCP with GIP can avoid almost all of the TOs and achieve high goodput for applications with the incast communication pattern.
Modeling and Understanding Tcp Incast in Data Center Networks

Jiao Zhang,Fengyuan Ren,Chuang Lin

DOI: https://doi.org/10.1109/infcom.2011.5934923

2011-01-01

Abstract:Recently, TCP incast problem attracts increasing attention since the receiver suffers drastic goodput drop when it simultaneously strips data over multiple servers. Lots of attempts have been made to address the problem through experiments and simulations. However, to the best of our knowledge, few solutions can solve it fundamentally at low cost. In this paper, a good put model of TCP incast is built to understand why good put collapse occurs. We conclude that TCP incast goodput deterioration is mainly caused by two types of timeouts, one happens at the tail of a data block and dominates the good put when the number of senders is small, while the other one at the head of a data block and governs the goodput when the number of senders is large. The proposed model describes the causes of these two types of timeouts which are related to the incast communication pattern, block size, bottleneck buffer and so on. We validate the proposed model by comparing with simulation data, finding that it can well characterize the features of TCP incast. We also discuss the impact of most parameters on the goodput of TCP incast.
Analysis for TCP in Data Center Networks: Outcast and Incast

Yang Qin,Weihong Yang,Yibin Ye,Yao Shi

DOI: https://doi.org/10.1016/j.jnca.2016.04.014

IF: 7.574

2016-01-01

Journal of Network and Computer Applications

Abstract:The unfairness caused by bandwidth sharing via TCP in data center networks is called TCP Outcast problem. Some researchers show that the throughput of a flow with small Round Trip Time (RTT) is less than that with large RTT which is completely contrary to the classic Transmission Control Protocol (TCP) protocol, and they believe that the Outcast problem is caused by port blackout in data center. However, we find that there are more important reasons for outcast which are an unfair distribution of the flows with different RTT on physical link and the differences in congestion window size when finishing the transmission of current block. TCP Incast is a throughput collapse that occurs when many flows arrive at the switch destined for the same output port. Incast deteriorates the performance of networks by increasing the queuing delay of flows and decreasing the throughput of applications. In this paper, we develop an analytical model for goodput when TCP Outcast occurs. Then we propose a new protocol based on window notification mechanism called TCP congestion window replacement (TCP-CWR), which can improve the goodput of the flows with small RTT to solve the outcast problem. Second, we propose a new transport layer protocol based on acknowledgment (ACK) reply changing rate to solve the Incast problem named TCP with Acknowledgment Changing Rate (TCP-ACR). The TCP-ACR protocol adjusts the current congestion window according to the changing rate of ACK and the theoretical maximum congestion window, and it can alleviate the congestion in data center networks. Last, we conduct simulation experiments to verify our proposed schemes. The results show that our analysis of Outcast is correct and our proposed protocols for Outcast and Incast are effective and practical.
Safe Reconfiguring Data Plane Via Supervision over Resource and Flow States

Gao Wen,Zhou Boyang,Wu Chunming,Zhou Haifeng,Jiang Ming,Hong Xiaoyan

DOI: https://doi.org/10.1049/cje.2015.07.035

IF: 1.019

2015-01-01

Chinese Journal of Electronics

Abstract:In the Software-defined networking（SDN）,when multiple control domains are used in control services,the transient state problem can occur causing the service flow interruption when border switches are updated asynchronously. We analyze the uniqueness of this problem for SDN, and propose a light-weight protocol for safe reconfiguration of the border switches in order to improve the availability of the services running in SDN. Our solution is designed as a generic supervision layer added in the control plane to support different types of services. To demonstrate the benefits, we implement a prototype of Informationcentric networks（ICN） with the protocol, and conduct experiments using Planet Lab. The results show the ICN service can continuously serve high volume requests for contents despite the congestions built by the heavy background traffic. The performance gains in terms of the mean and standard deviation of the content retrieve delay are40.5% and 21.56%.
Modeling and Solving Tcp Incast Problem in Data Center Networks

Jiao Zhang,Fengyuan Ren,Li Tang,Chuang Lin

DOI: https://doi.org/10.1109/tpds.2014.2310210

IF: 5.3

2015-01-01

IEEE Transactions on Parallel and Distributed Systems

Abstract:TCP Incast problem attracts much attention due to the catastrophic goodput drop. In this paper, a goodput model of the problem is built to understand why goodput collapse occurs and a solution to the problem based on the theoretical analysis is proposed. We found that the TCP Incast goodput deterioration is mainly caused by two types of timeouts, one happens at the tail of data blocks and dominates the goodput when the number of senders is small, while the other one at the head of data blocks and governs the goodput when the number of senders is large. The proposed model describes the relationship between these two types of timeouts and the Incast communication pattern, block size, bottleneck buffer size, and so on. The simulation results indicate that the model well characterizes the features of the TCP Incast problem. Enlightened by the analysis, a PRiority-based solution to the TCP INcast problem (PRIN) is proposed, which avoids timeouts at the head of blocks by reducing TCP send window and prevents timeouts at the tail of blocks by leveraging priority technology. The experimental results show that PRIN solves the TCP Incast problem.
A Delay-Based Tcp For Solving Incast Problem In Data Centers

Feng Zheng,Yongfeng Huang,Kevin Yin

DOI: https://doi.org/10.1109/ChinaSIP.2015.7230534

2015-01-01

Abstract:TCP incast problem has become a severe problem in datacenters due to the catastrophic collapse of goodput at the receiver side. Many solutions have been proposed, however, none of them solved it fundamentally. In this paper, we take a deep dive into the causes of incast problem, and find out that the root cause of TCP incast is the droptails induced by TCP congestion control algorithm based on packet losses. According to the root cause, we design a new TCP based on FAST TCP, a delay based congestion control algorithm, to radically solve TCP incast problem. The new TCP aims at maintaining a small and relatively constant queue length that does not exceed the switch buffer and meanwhile fully occupy the bottleneck link. The evaluation results in both simulation and real system demonstrate that our method cuts off most of the timeouts and attains high goodput under various conditions.
Session-aware congestion control for TCP Incast in datacenter networks

Dagang Li,Shuyi Li,Zheng Du

DOI: https://doi.org/10.1109/ISCC.2016.7543904

2016-01-01

Abstract:TCP Incast is one of the typical datacenter problems that may affect network efficiency in a many to one transmission session, in which the completion time of the whole session depends on the last finishing flow. In this paper a session-aware mechanism is proposed that intelligently chooses only the leading flows to slow down at the presence of congestion, so the lagging ones will have a higher chance to catch up to achieve better session completion time and goodput. Compared to existing solutions, the proposed one introduces no change to the TCP protocol and no extra messaging for the internal status of either TCP or the intermediate switches. Experimental results show that the proposed mechanism can indeed balance the progress among Incast flows and achieve a shorter session time and higher goodput.
Designing a new TCP based on FAST TCP for datacenter

Feng Zheng,Yongfeng Huang,Donghong Sun

DOI: https://doi.org/10.1109/ICC.2014.6883815

2014-01-01

Abstract:TCP incast problem has become a severe problem in datacenters due to the catastrophic collapse of goodput at the receiver side. Many solutions have been proposed, however, none of them solved it fundamentally. In this paper, we take a deep dive into the causes of incast problem, and find out that the root cause of TCP incast is the droptails induced by TCP congestion control algorithm based on packet losses. According to the root cause, we design a new TCP based on FAST TCP, a delay based congestion control algorithm, to radically solve TCP incast problem. The new TCP aims at maintaining a relatively small queue length that does not exceed the switch buffer and meanwhile fully occupy the bottleneck link. The simulation results demonstrate that our method cuts off most of the timeouts and attains high goodput under various conditions.
CW-HSTCP: Fair TCP in High-Speed Networks

Pan Xue-zeng,Su Fan-jun,Lü Yong,Ping Ling-di

DOI: https://doi.org/10.1631/jzus.2006.a0172

2006-01-01

Journal of Zhejiang University SCIENCE A

Abstract:The congestion control mechanisms of the current standard TCP constrain the congestion windows that can be achieved by TCP in high-speed networks, which leads to low link utilization. HSTCP is one solution to solve this problem by modifying the congestion control mechanism to have the characteristics of TCP friendliness in high loss rate environment and high scalability in low loss rate environment. However, experiments revealed that HSTCP has severe RTT unfairness. After analyzing the RTT unfairness in HSTCP with a model, we proposed CW-HSTCP, which added a fair factor to decrease the difference of congestion window caused by different RTT. Fair factor of long RTT flows can cause a sharp window increment that is easy to cause a bursty traffic, so a method called block-pacing was adopted. Simulation results showed that our new proposal could alleviate the RTT unfairness while keeping advantages of HSTCP.
Research on TCP Incast in Data Center Networks

Yajun YU,Zheng LIU,Mingwei XU

DOI: https://doi.org/10.3778/j.issn.1673-9418.1706034

2017-01-01

Abstract:Because the traditional TCP protocol does not apply to the operating mode of data center, the TCP Incast problem occurs when there are common many-to-one traffic patterns in the data center, causing a visible throughput collapse of the application layer. Considering the characteristics of the data center, putting forward a comprehensive solution is the research objective of TCP Incast problem. This paper analyzes the root causes of the problem, enu-merates the challenges of the problem, introduces the mathematical model based on the problem, analyzes and sum-marizes the recent solutions which are classified into link layer, transport layer and application layer, then from the effectiveness, deployment and other different aspects, makes a comprehensive comparison, finding that current solu-tions based on some specific points almost have drawbacks in different degree. Finally, this paper puts forward some feasible solutions to study the problem, and focuses on combining the technology of SDN and machine learning and designing a new transport protocol.
A Hybrid Simulating Framework of TCP Traffic at Aggregated Level

Yufeng Chen,Yabo Dong,Zhengtao Xiang,Dongming Lu

DOI: https://doi.org/10.1109/chinacom.2008.4685032

2008-01-01

Abstract:To obtain a trade-off between veracity and complexity when simulating TCP traffic, a hybrid simulating framework is proposed based on the packet-level simulation and aggregated control. The aggregated TCP traffic of access network in the gateway is proposed as simulated object, instead of the traffic generated at individual host or session level. The framework is divided into four parts: traffic generator, transmission controller, traffic sink and network. The traffic with self-similarity or multi-fractal property at application level is generated by traffic generator. The transmission controller sends packets to traffic sink based on aggregated control. The transmission controller also sends control information to traffic generator according to the status of network, and the traffic generator decides how to change the generating mode of traffic. The simulation results of non-congestion case and congestion case show the validity of the framework. The time cost shows our simulating framework possesses better performance.
Preventing TCP Incast Throughput Collapse at the Initiation, Continuation, and Termination

Adrian Sai-Wah Tam,Kang Xi,Yang Xu,H. Jonathan Chao

DOI: https://doi.org/10.1109/iwqos.2012.6245995

2012-01-01

Abstract:Incast applications have grown in popularity with the advancement of data center technology. It is found that the TCP incast may suffer from the throughput collapse problem, as a consequence of TCP retransmission timeouts when the bottleneck buffer is overwhelmed and causes the packet losses. This is critical to the Quality of Service of cloud computing applications. While some previous literature has proposed solutions, we still see the problem not completely solved. In this paper, we investigate the three root causes for the poor performance of TCP incast flows and propose three solutions, one for each at the beginning, the middle and the end of a TCP connection. The three solutions are: admission control to TCP flows so that the flow population would not exceed the network's capacity; retransmission based on timestamp to detect loss of retransmitted packets; and reiterated FIN packets to keep the TCP connection active until the the termination of a session is acknowledged. The orchestration of these solutions prevents the throughput collapse. The main idea of these solutions is to ensure all the on-going TCP incast flows can maintain the self-clocking, thus eliminates the need to resort to retransmission timeout for recovery. We evaluate these solutions and find them work well in preventing the retransmission timeout of TCP incast flows, hence also preventing the throughput collapse.
Adaptive rate control for TCP Incast based on selective ECN-marking

Li Shuyi,Li Dagang,Du Zheng

DOI: https://doi.org/10.1109/ICSESS.2016.7883084

2016-01-01

Abstract:This paper presents a practical solution to a typical datacenter problem which may deteriorate network efficiency in a many to one transmission session, which is called the TCP Incast problem. In Incast, the completion time of the whole session depends on the last finishing flow. To address such problem, we propose an adaptation mechanism which always selects sufficient leading flows to slow down to mitigate congestion, so the lagging flows of the session can catch up. The proposed solution makes use of the existing ECN mechanism without modifying the TCP protocol. Results from the experiments show that the proposed mechanism can achieve a faster transmission time and better goodput result as well as balancing the progress among Incast flows.
Floodgate: taming incast in datacenter networks

Kexin Liu,Chen Tian,Qingyue Wang,Hao Zheng,Peiwen Yu,Wenhao Sun,Yonghui Xu,Ke Meng,Lei Han,Jie Fu,Wanchun Dou,Guihai Chen

DOI: https://doi.org/10.1145/3485983.3494854

2021-01-01

Abstract:ABSTRACTIncast occurs frequently in datacenter networks where a large number of senders send data to a single receiver simultaneously, which makes the last hop the network bottleneck. Incast can hurt flows' performance. However, congestion control protocols are not effective at handling incast. One key insight is that it is too late to handle incast packets after they have already piled up at the last hop. Instead, we should avoid incast as early as possible. Inspired by flood control in Hydrologic Engineering, we propose Floodgate, a novel switch-based per-hop flow control to handle incast. Floodgate is compatible with existing congestion control protocols. We integrate it with practical congestion control approaches such as DCQCN, TIMELY, and HPCC. We evaluate Floodgate both in our implementations and large-scale simulations. Compared with state of the art, Floodgate reduces the buffer occupancy by a factor of 6.6x, as well as the queuing delay. Therefore, the average FCT and tail latency are greatly reduced.
Comprehensive understanding of TCP Incast problem

Wen Chen,Fengyuan Ren,Jing Xie,Chuang Lin,Kevin Yin,Fred Baker

DOI: https://doi.org/10.1109/INFOCOM.2015.7218549

2015-01-01

Abstract:Since TCP Incast has been identified as a catastrophic problem in many typical data center applications, a lot of efforts have been made to analyze or solve it. The analysis work intends to model Incast problem from certain perspective, and the solutions try to solve the problem through designing enhanced mechanisms or algorithms. However, the proposed models are either closely coupled with particular protocol version or dependent on empirical observations, and the solutions cannot eliminate Incast problem entirely because the underlying issues are not identified completely. There is little work which attempts to close the gap between “analyzing” and “solving”, and present a comprehensive understanding. In this paper, we provide an in-depth understanding of how TCP Incast problem happens. We build up an interpretive model which emphasizes particularly on describing qualitatively how various factors, including system parameters and mechanism variables, affect network performances in Incast traffic pattern, but not on calculating the accurate throughput. With this model, we give plausible explanations why the various solutions for TCP Incast problem can help, but do not solve it entirely.
Osdt: A Scalable Application-Level Scheduling Scheme For Tcp Incast Problem

Shuli Zhang,Yan Zhang,Yifang Qin,Yanni Han,Zhijun Zhao,Song Ci

DOI: https://doi.org/10.1109/ICC.2015.7248342

2015-01-01

Abstract:TCP Incast refers to the phenomenon of goodput collapse when multiple synchronized servers send data to the same client in parallel. In this paper, we propose a novel application-level scheduling approach named OSDT (Optimal Staggering Data Transfers) for TCP Incast problem. OSDT limits the number of concurrent TCP flows as well as servers' sending rate to optimal values so that the utilization of link capacity can be maximized without any packet losses. To achieve this, we build an optimization model with the usage of network and application information. Based on this model we can get the optimal values for the parameters in OSDT. Simulation results indicate that OSDT can achieve the highest goodput among all existing application-level scheduling approaches in a wide range of network and application parameters, and its performance is stable. So OSDT can be seen as an effective and scalable solution for TCP Incast problem.
SIM: Sub-RTT-based Incast Mitigation in Data Center Networks

Haoran Chen,Hang Wang,Guanglei Chen,Peilin Hong

DOI: https://doi.org/10.1109/iscc61673.2024.10733561

2024-01-01

Abstract:Recently, many Remote Direct Memory Access (RDMA) congestion control (CC) algorithms have been proposed to ensure the performance of high-speed Data Center Networks (DCNs). However, there is an inherent feedback delay in end-to-end CC, resulting in the inability to exert control over each flow within its first RTT. Upon many-to-one (Incast), the immediate high queue introduces significant latency to the short flows. In this paper, we propose SIM, a Sub-RTT-based Incast mitigation scheme to enhance congestion control. SIM offers two key functionalities: it enables the detection and notification of Incast within Sub-RTT, and it provides an adaptive adjustment algorithm that responds based on the severity of the Incast. Simulation results show that SIM fundamentally reduces the peak queue of Incast and shortens 99% FCT slowdown of short flows by up to 70% and 47% compared to the state-of-the-art congestion control schemes in DCNs, respectively.
TCP-FITDC: an Adaptive Approach to TCP Incast Avoidance for Data Center Applications

Jun Zhang,Jiangtao Wen,Jingyuan Wang,Wenlai Zhao

DOI: https://doi.org/10.1109/iccnc.2013.6504236

2013-01-01

Abstract:TCP incast is an issue of great importance to the performance of many-to-one communication sessions in data centers. It is resulted from the mismatch between the RTO mechanism of existing TCP congestion control algorithms and the requirements and parameters in data center applications, leading to gross under-utilization of link capacity, high latency and packets loss rates. In this paper, we propose a novel and adaptive delay-based mechanism for TCP incast prevention in data center applications. Theoretical analysis and simulation results show significant performance improvement as compared with existing methods.
A Coding-based Approach to Mitigate TCP Incast in Data Center Networks

Changlin Jiang,Dan Li,Mingwei Xu,Kai Zheng

DOI: https://doi.org/10.1109/ICDCSW.2012.70

2012-01-01

Abstract:As the key infrastructure of cloud computing, data center network provides routing and transport services for cloud applications. To ensure reliable data delivery, TCP has been used in data centers as the de facto transport layer protocol. However, recently researchers found that when a client has a barrier-synchronized requirement and requests data from many servers, the good put may experience collapse. This phenomenon is well-known as TCP In cast. The main cause of TCP In cast is the scare buffer size in low end switches as well as the mismatch between RTO and RTT in data centers. So existing solutions focus on either minimizing the effect of TCP's RTO, or preventing flows from aggressively utilizing the switch buffer. Different from the previous approaches, in this paper we employ coding-based UDP to replace TCP as the transport protocol for the many-to one applications. Specifically, we use LT Codes to ensure reliable data delivery, and adopt TCP Friendly Rate Control (TFRC) to achieve congestion control. The NS based simulation shows that our approach can significantly enhance the application good put in a typical TCP In cast scenario.
Throughput Optimization of TCP Incast Congestion Control in Large-Scale Datacenter Networks

Lei Xu,Ke Xu,Yong Jiang,Fengyuan Ren,Haiyang Wang

DOI: https://doi.org/10.1016/j.comnet.2017.06.004

IF: 5.493

2017-01-01

Computer Networks

Abstract:The many-to-one traffic pattern in datacenter networks leads to Transmission Control Protocol (TCP) incast congestion and puts unprecedented pressure to cloud service providers. The abnormal TCP behaviors in incast increase system response time and unavoidably reduce the applicability of cloud-based system deployments. This paper proposes Receiver-oriented Congestion Control (RCC) to address heavy incast in large-scale datacenter networks. RCC is motivated by oscillatory queue size of switch when handling heavy incast and substantial potential of receiver in congestion control when using TCP. RCC makes effective use of centralized scheduler and Explicit Congestion Notification (ECN) at receiver. The RCC prototype is realized in network simulator 3 (ns3) which implements TCP exactly. This paper details the RCC design and evaluates its performance in diverse and heavy workloads. The evaluation results indicate that RCC has an average decreases of 47.5% in the mean queue size and 51.2% in the 99th-percentile latency in the heavy incast over TCP.

SIG: Solution to TCP Incast in SDN Network Based Openflow Protocol

Taming TCP Incast Throughput Collapse in Data Center Networks

Modeling and Understanding Tcp Incast in Data Center Networks

Analysis for TCP in Data Center Networks: Outcast and Incast

Safe Reconfiguring Data Plane Via Supervision over Resource and Flow States

Modeling and Solving Tcp Incast Problem in Data Center Networks

A Delay-Based Tcp For Solving Incast Problem In Data Centers

Session-aware congestion control for TCP Incast in datacenter networks

Designing a new TCP based on FAST TCP for datacenter

CW-HSTCP: Fair TCP in High-Speed Networks

Research on TCP Incast in Data Center Networks

A Hybrid Simulating Framework of TCP Traffic at Aggregated Level

Preventing TCP Incast Throughput Collapse at the Initiation, Continuation, and Termination

Adaptive rate control for TCP Incast based on selective ECN-marking

Floodgate: taming incast in datacenter networks

Comprehensive understanding of TCP Incast problem

Osdt: A Scalable Application-Level Scheduling Scheme For Tcp Incast Problem

SIM: Sub-RTT-based Incast Mitigation in Data Center Networks

TCP-FITDC: an Adaptive Approach to TCP Incast Avoidance for Data Center Applications

A Coding-based Approach to Mitigate TCP Incast in Data Center Networks

Throughput Optimization of TCP Incast Congestion Control in Large-Scale Datacenter Networks