Thursday, June 18, 2020

Troubleshooting packet drops in Solarflare NIC

Recently on one of our servers we saw that the 10G solarflare NIC card was dropping packets. We could also detect the same within our application. We had client application listening to video streams send over the network via multicast traffic. Things were all good and green untill we increased the channels being subscribed by our client application.

We went around troubleshooting the problem in the following fashion:

  • Check that the application buffers were not the bottleneck while reading the packets 
  • Check that the NIC network rx buffer were set to maximum
  • Ascertain that traffic on the network was within the limit of what the network lan could handle.
  • Loading onload drivers with different pre-defined profiles (like latency, etc). 


Well nothing seemed to help. Then we reached out to the support guys at solarflare. And the guys there do an amazing job. We provided them a onload_stackdump with the running application.
And let me tell you it was huge (~ xx lines).
After analyzing they told us that the application read buffers were not sufficient to handle the input stream data packet, so why not increase the no of rx ring buffer size. Well I told them that we had already done that via the ethtool -g <> rx 4096 and that it did not work.

And the big surprise was while applications are running in kernel bypass mode, those settings dont come into picture. Instead they asked us to set rx_packets to 35999 MAX (out of which around 75% is used for rx buffers packets and rest 25% for tx packets).

Well that too didnt help solve the problem. Next we tried resetting the ring buffer size from the default 512 to 4096 (though they claimed its default 512 has been running for now 4+ years). 
But I anyhow did it and YEAH.. I saw absolutely no packet drops since then. 

Wednesday, June 17, 2020

Visual Studio Error: Fails to launch Debugger for ASP.NET web project

Visual Studio Debugger launch error : One or more error occurred
I recently ran into the following error while trying to launch Visual studio debugger for one of my ASP.NET web project


Simple web application which does some data rendering connected to a COSMOS DB. well it was working fine for some time, and then suddenly it started failing.
The error reads :

One or more errors occurred. 

Failed to Launch Debug adapter. Additional Information may be available in the output window. 

Cannot connect to runtime process, timeout after 10000 ms - Reason : Cannot connect to target: connect ECONNREFUSED 127.0.0.1:61479



Turned out that the problem was that during the last debug session/run, the chrome instance had not died/closed (which it usually does by itself). So this time launching chrome was giving the error. Well the error is cryptic and not much helpful, unless you look it up.
so go ahead, locate your previuosly launched debug browser window and close it. And press debug again and you should be good to go.



Section 80C TMT 2023(PA) – Standard Deduction amounting to a maximum of Rs 8670 under IT rule

With the recently concluded Tax filing season (which if I may draw parallel to the Christmas holiday season enjoyed by one and all), Indians...