Ping pong test with Ansible

A quick and sweet command to ping your inventory/servers in ansible: 

[parag@centosVM ops-dir]$ ansible all -m ping -i production
PodAServer18 | SUCCESS => {
    "changed": false,
    "ping": "pong"
}
PodAServer93 | SUCCESS => {
    "changed": false,
    "ping": "pong"
}
PodAServer71 | SUCCESS => {
    "changed": false,
    "ping": "pong"
}
PodAServer73 | SUCCESS => {
    "changed": false,
    "ping": "pong"
}
PodBServer151 | UNREACHABLE! => {
    "changed": false,
    "msg": "Failed to connect to the host via ssh: Connection timed out during banner exchange\r\n",
    "unreachable": true
}
PodBServer48 | UNREACHABLE! => {
    "changed": false,
    "msg": "Failed to connect to the host via ssh: Connection timed out during banner exchange\r\n",
    "unreachable": true
}

The above command pings all the server in your production to check for their availability/accessibility/to check whether they are reachable or not. 

In the output above servers : PodAServer18, PodAServer93, PodAServer71, PodAServer73  are accessible and PodBServer151, PodBServer48 are not reachable. 

This can be modified to check for specific sub-section/groups in the server inventory file. 

The following command limits the ping test to the emailServers group defined in the staging inventory file. 

[parag@centosVM ops-dir]$ ansible all -m ping -i staging--limit emailServers
server43 | UNREACHABLE! => {
    "changed": false,
    "msg": "Failed to connect to the host via ssh: Connection timed out during banner exchange\r\n",
    "unreachable": true
}
Server35 | UNREACHABLE! => {
    "changed": false,
    "msg": "Failed to connect to the host via ssh: Connection timed out during banner exchange\r\n",
    "unreachable": true
}
Rack3Server45 | UNREACHABLE! => {
    "changed": false,
    "msg": "Failed to connect to the host via ssh: Connection timed out during banner exchange\r\n",
    "unreachable": true
}
[parag@centosVM ops-dir]

aa

Troubleshooting Packet drops in Solarflare 10G network card

If you experience packet drops while listening to UDP packet broadcast then the following extracts from the onload_stackdump command utility can help you
identify the Onload stack under memory pressure.

[root@paragpc ~]  onload_stackdump lots

You can monitor the rate of these drops for the Onload stack using the above command, for example in one of my machine the stackdump reported:

[root@paragpc ~] onload_stackdump lots | grep memory_pressure_drops

memory_pressure_drops: 81381

If this is increasing during bursts of data then increasing the value of EF_MAX_PACKETS to 65000 or higher could resolve the issue by providing more buffering but if this value is increasing steadily then it would only delay the point at which drops happen. You can set this by setting “EF_MAX_PACKETS=65000” in the environment or prefix the ‘onload’ command, e.g.:

[root@paragpc ~] EF_MAX_PACKETS=65000 onload <app_cmd_line>

Note that this is further restricted for TX and RX in order to avoid potential deadlocks where all the packet buffers are used on the RX and can’t be freed because data needs to be sent to do this. By default there is a 75% limit on each so if you use the maximum for RX it would leave 25% for TX.

Read more about the how to troubleshoot and configure onload here.

C++: Looking at source code while debugging in GDB

Sometimes when I am debugging my code for issues/bugs, I want to look at the surrounding piece of code where my debugger is executing, or may be I want to look at the some lines of code in a particular file. Its quite time consuming to toggle windows, switch source code editor/IDE locating code(sometimes switch desktops). Worry no more, use the following commands, to help you navigate through the code, while debugging.

Scenario: after you have set your breakpoint and the GDB debugger hits the section, and you want to look around those line, type list

[Parag@paragpc:/home/Parag/Workspace/Utils]gdb sample
GNU gdb (GDB) Red Hat Enterprise Linux (7.2-60.el6)
Copyright (C) 2010 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from /home/Parag/Workspace/Utils/sample...done.
(gdb) break countLines
Breakpoint 1 at 0x400cf2: file MultiUseHeader.hpp, line 21.
(gdb) run
Starting program: /home/sid/Parag/Workspace/Utils/samp 
inside registerQuickExitHandlers()

Breakpoint 1, countLines () at MultiUseHeader.hpp:21
warning: Source file is more recent than executable.
21     std::ifstream myfile(strFileName);
Missing separate debuginfos, use: debuginfo-install glibc-2.12-1.107.el6.x86_64 libgcc-4.4.7-11.el6.x86_64 libstdc++-4.4.7-11.el6.x86_64
(gdb) list
16  char cArr[2];
17 };
18 
19 int countLines(std::string& strFileName) {
20 
21     std::ifstream myfile(strFileName);
22 
23     // new lines will be skipped unless we stop it from happening:    
24     myfile.unsetf(std::ios_base::skipws);
25 
(gdb) n
24     myfile.unsetf(std::ios_base::skipws);
(gdb) n
30         '\n');
(gdb) list
25 
26     // count the newlines with an algorithm specialized for counting:
27     unsigned line_count = std::count(
28         std::istream_iterator<char>(myfile),
29         std::istream_iterator<char>(), 
30         '\n');
31 
32     std::cout << "Lines: " << line_count << "\n";
33     return 0;
34 }
(gdb) 

Similarly if you want to look at some specific lines in a source file, type:

(gdb) list MultiUseHeader.hpp:32
27     unsigned line_count = std::count(
28         std::istream_iterator<char>(myfile),
29         std::istream_iterator<char>(), 
30         '\n');
31 
32     std::cout << "Lines: " << line_count << "\n";
33     return 0;
34 }
35 
36 
(gdb) 

Example Boost MultiIndex Container Insertion and Iteration

Here is another easy example to get you started with boost multi index containers:

#include <string>
#include <iostream>
#include <boost/multi_index_container.hpp>
#include <boost/multi_index/member.hpp>
#include <boost/multi_index/ordered_index.hpp>
#include <boost/date_time/posix_time/posix_time.hpp>

using boost::multi_index::multi_index_container;
using boost::multi_index::ordered_non_unique;
using boost::multi_index::ordered_unique;
using boost::multi_index::indexed_by;
using boost::multi_index::member;
using boost::multi_index::nth_index;
using boost::multi_index::get;

struct employee_entry
{
 employee_entry( const std::string& first,
   const std::string& last,
   long id):
  first_name_(first),
  last_name_(last),
  id_(id)
 {}
 std::string first_name_;
 std::string last_name_;
 long id_;
};

typedef multi_index_container<
 employee_entry, 
 indexed_by<
  ordered_unique<
       member<employee_entry, std::string, &employee_entry::first_name_> 
  >, 
  ordered_non_unique<
       member<employee_entry, std::string, &employee_entry::last_name_> 
  >, 
  ordered_non_unique<
        member<employee_entry, long, &employee_entry::id_> 
  >
 >
> employee_set;

//employee set.... multi-index
employee_set m_employees;

using namespace std;

//Define the different type of iterators
typedef nth_index<employee_set, 0>::type fname_view;
fname_view& fdv = get <0> (m_employees);

typedef nth_index<employee_set, 1>::type lname_view;
lname_view& lnamev = get <1> (m_employees);


typedef nth_index<employee_set, 2>::type id_view;
id_view& idv = get <2> (m_employees);

void PrintLnameWise()
{
 ///get employees sorted by lname
 cout<<"Printing sorted by Lname... \n";
 for(lname_view::iterator it = lnamev.begin(), it_end(lnamev.end()); it != it_end; ++it)
 {
  std::cout << it->first_name_  <<" "
   << it->last_name_ << ":"
   << it->id_ << std::endl;
 }

   const std::string str(40, '-');
   std::cout << str << std::endl;
}

void PrintFnameWise()
{
 ///get employees sorted by first name
 cout<<"Printing sorted by Fname... \n";

   for(fname_view::iterator it = fdv.begin(), it_end(fdv.end()); it != it_end; ++it)
   {
       std::cout << it->first_name_  <<" "
                 << it->last_name_ << ":"
                 << it->id_ << std::endl;
   }
  
   const std::string str(40, '-');
   std::cout << str << std::endl;
}


void PrintIdWise()
{
   cout<<"Printing sorted by Id... \n";
   
   for(id_view::reverse_iterator it = idv.rbegin(), it_end(idv.rend()); it != it_end; ++it)
   {
       std::cout << it->first_name_  <<" "
                 << it->last_name_ << ":"
                 << it->id_ << std::endl;
   }

   const std::string str(40, '-');
   std::cout << str << std::endl;
}

int main()
{

   typedef nth_index<employee_set, 0>::type first_name_view;
   first_name_view& fnv = get<0>(m_employees);

   fnv.insert(employee_entry("John", "Smith", 110));
   fnv.insert(employee_entry("Fudge", "Hunk", 97));
   fnv.insert(employee_entry("Tolem", "Bathi", 87));
   fnv.insert(employee_entry("Prisha", "Agrawal", 1));
   fnv.insert(employee_entry("Shilpi", "Jain", 25));

   std::cout << "Count Structure after 5 inserts:" << m_employees.size() << std::endl;

   //Inserting via the ID Index iterator
   idv.insert(employee_entry("Carlos", "Linus", 140));
   idv.insert(employee_entry("Donald", "gates", 7));
   std::cout << "Count Structure after 2 more inserts:" << m_employees.size() << std::endl;
  
   //Inserting via the lname iterator/index
   lnamev.insert(employee_entry("Sharad", "Smith", 10));
   lnamev.insert(employee_entry("Parag", "Agrawal", 28));
   std::cout << "Final Count Structure after 2 more inserts:" << m_employees.size() << std::endl;
  
   const std::string str(40, '-');
   std::cout << str << std::endl;

   PrintIdWise();
   PrintLnameWise();
   PrintFnameWise();



   return 0;
}

Example Boost MultiI ndex Container

Here is a quick example to get to you started with boost::multi_index_container. Multi_index containers enables defining containers maintaining one or more indices with different sorting and access semantics.
Here is a sample program to get you started:

#include <boost/multi_index_container.hpp>
#include <boost/multi_index/member.hpp>
#include <boost/multi_index/hashed_index.hpp>

#include <iostream>

class CartItem{

public:

 int64_t cartItemId;
 int barcode;
 char symbol[10];        
 int64_t validityDate;     
 float price;

};


struct CartItemByCartItemIdIndexTag{};
struct CartItemByBarcodeIndexTag{};

typedef boost::multi_index_container<
 CartItem,
 boost::multi_index::indexed_by<
  boost::multi_index::hashed_unique<
   boost::multi_index::tag<CartItemByCartItemIdIndexTag>,
   boost::multi_index::member<CartItem, int64_t, &CartItem::cartItemId>
  >,
  boost::multi_index::hashed_unique<
   boost::multi_index::tag<CartItemByBarcodeIndexTag>,
   boost::multi_index::member<CartItem, int, &CartItem::barcode>  
  >
 >
> CartItemMapContainer;


typedef CartItemMapContainer::index<CartItemByCartItemIdIndexTag>::type CartItemByCartItemIdIndex;
typedef CartItemMapContainer::index<CartItemByBarcodeIndexTag>::type CartItemByBarcodeIndex;

//Global Variable
CartItemMapContainer  itemmultiIndexContainer;

void insertItem(int64_t cartItemId, int barcode){


 CartItem c;
 c.cartItemId = cartItemId;
 c.barcode = barcode;

 //Insert operation
 CartItemByCartItemIdIndex& index = itemmultiIndexContainer.get<CartItemByCartItemIdIndexTag>();

 if(!index.insert(c).second){
  std::cout << "Error inserting item";

 }
 else
  std::cout << "Insert successful\n";

}

void searchByItemId(int64_t cartItemId){

 CartItem d;
 //Find Operation by CartItemId
 CartItemByCartItemIdIndex& idx = itemmultiIndexContainer.get<CartItemByCartItemIdIndexTag>();
 CartItemByCartItemIdIndex::iterator itrb = idx.find(cartItemId);

 if(itrb == idx.end()){
  std::cout << "Look up By CartItemId failed";
  return;
 }
 else
  d = (*itrb);

 std::cout << "CartItemId:" << d.cartItemId << ", Barcode:" << d.barcode << std::endl;
}

void searchByBarcode(int barcode){

 CartItem e;
 CartItemByBarcodeIndex& indexByBarCode = itemmultiIndexContainer.get<CartItemByBarcodeIndexTag>();
 CartItemByBarcodeIndex::iterator itr = indexByBarCode.find(barcode);

 if(itr == indexByBarCode.end()){
  std::cout << "look up By BarCode failed";
  return;
 }
 else
  e = *itr;

 std::cout << "CartItemId:" << e.cartItemId << ", Barcode:" << e.barcode << std::endl;

}

int main(){

 insertItem(1,2);
 insertItem(2,4);
 insertItem(3,6);
 searchByItemId(1);
 searchByBarcode(6);


 return 0;

}

Bash Script to Create users in a DB

Many a times when running unit test on a clean build, I want to skip the user creation part, or may be reset the user details/profile for a particular user(s) in Development environment.
Of course there are unit test suite written to automate the flow, but sometimes when you are in the flow or designing the test cases itself, you need something to do the little rub-scrub-and-setup the little tiny jobs.
Here is my small script which deletes two very specific users from the my_dev_db db and creates them again. The details of the user are in file UserProd.txt which is tab delimited.  Enjoy.

[parag@paragcentosvm ~]# cat populateUsers.sh
#!/bin/sh

mysql -u parag_dev -pSOMEPASSWORD -e "DELETE FROM User_Table where UserId in (110,111);" -D my_dev_db;
mysql -u parag_dev -pSOMEPASSWORD -e "LOAD DATA LOCAL INFILE '/home/Parag/UserProd.txt' INTO TABLE User_Table;" -D my_dev_db;

Boost Mutex scoped_lock example

Here is a quick example showing how mutex helps in doing a synchronous access on shared resource. We are using boost::mutex::scoped_lock here, which ensures that the ownership of the mutex object is relinquished even in case the code following the lock throws an exception, thus preventing possible thread deadlocks.

#include <boost/thread/thread.hpp>
#include <boost/thread/mutex.hpp>
#include <boost/bind.hpp>
#include <iostream>

//Compile using : g++ program.cpp -lboost_thread

boost::mutex io_mutex;

void count(int id)
{
  for (int i = 0; i < 5; ++i)
  {
    boost::mutex::scoped_lock lock(io_mutex);
    std::cout << id << ": " << i << std::endl;
  }
}


void countWithoutMutex(int id)
{
  for (int i = 0; i < 5; ++i)
  {
    std::cout << id << ": " << i << std::endl;
  }
}


int main(int argc, char* argv[])
{
std::cout << "The wrong way\n";
  boost::thread thrd1(
    boost::bind(&countWithoutMutex, 1));
  boost::thread thrd2(
    boost::bind(&countWithoutMutex, 2));
  thrd1.join();
  thrd2.join();
std::cout << "Now the right way\n";
   boost::thread thrd3(
    boost::bind(&count, 3));
  boost::thread thrd4(
    boost::bind(&count, 4));
  thrd3.join();
  thrd4.join();
  

  return 0;
}

Output:

The wrong way
2: 01: 0
1: 1
1: 2
1: 3
1: 4

2: 1
2: 22
: 3
2: 4
Now the right way
3: 0
3: 1
3: 2
3: 3
3: 4
4: 0
4: 1
4: 2
4: 3
4: 4

R : Webcrawler Parser with Try-Catch

Well, I felt the need to do some analysis over content hosted on some initial set of web-sites and then aggregate it, plot. I created a very simple and easy parser which would crawl and parse the data read from these websites. I used the try-catch block for fault tolerance and resilience from error (website down, unavailable or trust error, etc). Here is a sample code, where I have used tryCatch and readLines methods :

  
>myUrlStats <- function(urlToCrawl) {
    statData <- tryCatch(
               {
   dataReadFromUrls <- readLines(con=urlToCrawl)
   myWebCrawlParser(dataReadFromUrls)
  },
        error=function(errorMessageStr) {
            message(paste("URL does not seem to exist:", urlToCrawl))
            message("Error message:")
            message(errorMessageStr)
            return(-1)
        },
        warning=function(warningMessageStr) {
            message(paste("URL caused a warning:", urlToCrawl))
            message("Warning message:")
            message(warningMessageStr)
            # Choose a return value in case of warning
            return(NULL)
        },
        finally={
   ##Clean up code
  }
 )
    return(statData)
}


> myWebCrawlParser <- function(dataReadFromURL){
# Do your analysis parsing here
# also like you can mine other outlinks from this data read for further traversing the web-links
return(1)
}


> urlToCrawl <- c(
  "http://superdevresources.com",
     "http://superbloggingresources.com"
     )
> finalReslt<- mapply(myUrlStats, urlToCrawl)

Happy programming!

R: Alternative and easy approach to Data concatenation

Sometimes in R you have the requirement of concatenating/appending data from two sources to create one Super Set, for eg. You may have a list of countries and economic growth indicators, and you want to create a data set which is a super set of this data (AxB), here is how you can do it via simple apply operation:

> myCustColNames
  MyColNames
1          A
2          B
3          C
4          D
5          E
> growthIndicator
[1] "GDP"        "Inflation"  "Population"
> apply(myCustColNames,1, paste, growthIndicator, sep="_")
     [,1]           [,2]           [,3]           [,4]           [,5]          
[1,] "A_GDP"        "B_GDP"        "C_GDP"        "D_GDP"        "E_GDP"       
[2,] "A_Inflation"  "B_Inflation"  "C_Inflation"  "D_Inflation"  "E_Inflation" 
[3,] "A_Population" "B_Population" "C_Population" "D_Population" "E_Population"
> 

Another example can be:

> myCustColNames<-data.frame(MyColNames=LETTERS[1:5])
> myCustColNames
  MyColNames
1          A
2          B
3          C
4          D
5          E
 > apply(myCustColNames,1, paste,seq(1:6), sep="")
     [,1] [,2] [,3] [,4] [,5]
[1,] "A1" "B1" "C1" "D1" "E1"
[2,] "A2" "B2" "C2" "D2" "E2"
[3,] "A3" "B3" "C3" "D3" "E3"
[4,] "A4" "B4" "C4" "D4" "E4"
[5,] "A5" "B5" "C5" "D5" "E5"
[6,] "A6" "B6" "C6" "D6" "E6"
> 

R: Converting a list to a dataframe

Sometimes a function returns a list data type, which you often need to convert to a data.frame data type. To convert the list to data.frame ran the following statement:

 compositeRowOfDataFrame<-do.call(rbind.data.frame, listData)

R deleting rows in a dataframe

To remove all rows from a data frame in R, you can use any of the following ways:

> b
   x  y   Result
1  1  1   B
2  1  2   B
3  1  3   C
4  1  4   A
5  1  5   B
6  1  6   A
7  1  7   C
8  1  8   C
9  1  9   B
10 1 10   C
> deleteRowsVec=c(FALSE)
> b[deleteRowsVec,]
[1] x   y   Result
<0 rows> (or 0-length row.names)

R: How to get current working directory and change it

If you are working on a R project, and you wish to load/save your data in some file, it is important to know which directory you are currently working in, so that you can easily browse or save your data to that directory or may be change it to a specific location. Here are the list of commands to get you going :

> getwd()
[1] "C:/Users/parag/Documents"
> ?setwd
> setwd("X:/Share")
> getwd()
[1] "X:/Share"
> 

WhatsApp call feature is out

The much awaited call feature of WhatsApp is out now and available to all Android users.  After installing the update and opening WhatsApp, you see a new tab for Calls next to the Chats and Contacts tab. Tap on the Calls tab and select the name of the person that you want to call. 
I made some quick phone call in successions to some of my friends and family and found the voice quality at par with similar services like Skype. Currently its available only to Android users. IPhone and Windows phone users would need to wait for some time. 

Windows Server 2003 End of Support

Starting July 14, 2015, Microsoft will end all support and updates for Windows Server 2003. Microsoft hosting an exclusive series of webinars to help you with a smooth transition from Windows Server 2003.

Presenter(s):	Manpreet Madaan and Vishal Mitbawkar.
Language(s):	English.
Product(s):	Microsoft server product portfolio.
Audience(s):	IT Decision Maker, IT Manager, Tech Influencing BDM and Tech Support - Partners.

Webinar Details:

Date: April 02, 2015   Time: 3:00 pm to 4:00 pm

Register here.

Microsoft IOT DevCamp in Pune & Hyderabad

Microsoft DevCamps are no-fluff events for developers, by developers. You learn from experts in an interactive way and then get to apply what you’ve learned.

.

Our upcoming IoT DevCamp will show how to use Microsoft's Azure services (specifically Event Hub, Streaming Analytics, and Web Sites) in the context of a full end-to-end IoT solution. The lab currently includes a sensor/gateway approach with low-cost Arduino devices as the sensor end-point, and Raspberry Pi as the gateway. A dummy Windows console gateway is provided for those who wish to focus solely on the Azure solution.

.

Participants are also encouraged to bring their own devices to hack, and learn to connect to Azure. At the end of the session, you will have a good end-to-end understanding of a typical IoT solution.

.

Topics that will be covered:

.

IoT overview, architecture, Microsoft platform Event Hubs  Azure NRT Devices (Arduino, Raspberry Pi, etc.)

.

Instruction, demos, equipment and hands on labs will be provided in class.

When & Where

.

. City: . Pune . Date: . 29th April 2015 . Timings: . 9:00 am - 6:00 pm . Venue: . The O Hotel, North Main Road, Koregaon Park, Pune 411001 . . City: . Hyderabad . Date: . 24th April 2015 . Timings: . 9:00 am - 6:00 pm . Venue: . Radisson Hyderabad Hitec City, Gachibowli, Miyapur Road, Hyderabad 500032 .

. Pre-requisites for pre-event online briefing: . You should be IoT Start-ups or IoT enthusiasts having a working knowledge of Embedded/IoT/System on Chips devices. . Details about the pre-event online briefing will be sent by email a few days after you register. Attendance is required. . . Pre-Requisites for the Event: . A Laptop with Visual Studio 2013 Update 4 . Microsoft Azure Subscription (MSDN, BizSpark, Trial, Pay-As-You-Go or Company/Individual Account)

Explore the world of Azure with IoT

Microsoft Office 2013 - Upto 15% off

Today is the last day to register for Microsoft Office 2013 and get discounts of up to 15%. This is valid only for Small and Medium Businesses. Find your cloud partner here.

Microsoft Azure App Service Now available

Microsoft recently announced the Azure App Service which helps developers deliver cross-platform, cloud-connected apps faster. This solution integrates the Microsoft Azure Websites, Mobile Services and Biztalk Services into a single service with a common app hosting, runtime and extensibility model, enabling simplified integration with popular consumer and commercial services and on-premises systems, all for one low price.
Read more here. 

Windows 10: Windows Insider Hub

I have been using Microsoft Windows 10 for over two month now. Cortana is a big help and right now I am more or less using it as my "Search" cum "Run-as" feature. The side notification center situated on the task bar is a big help. Quick look up and turning features and settings of my machine on-off. The outlook and calender apps are the most used one at the moment as I could not find much exciting otherwise there. The good news Microsoft will update this to the RTM version of Windows 10 for free. Anyways with VS2014 community edition now also free its good for developers. 

One great thing about the Windows 10 preview built is the Windows Insider Hub app, through which you can give feedback and suggestions directly to the Microsoft team, which is really really cool. I can up vote on features requested by others which they would like to see. There are different categories under which you can directly provide your views and must haves directly to the concerned teams. 

R: Selecting unique values out of dataset

Many a times you have a dataset or a column which has a range of values and you wish to know the domain or range of values the column assumes. To do so run the following command:

>  y<-c('A', 'A','A','A','B', 'B','B','B','B','B','B','B','B', 'C','C','D','D', 'E','E', 'E','E','E' ,'E','E','E' ,'E','F','F','F', 'F','F', 'F','F', 'F','F')
> levels(factor(y))
[1] "A" "B" "C" "D" "E" "F"
> unique(y)
[1] "A" "B" "C" "D" "E" "F"

Windows 10 preview build Updates

I must say that when I installed and ran the first publicly available build of windows 10 announced in January this year, I was a little upset with the performance of the system. It ran so many apps and processes in the background that the machine felt sluggish and slow and non-responsive.

You had to keep it running for few minutes after boot before you could start working on it. This was a total turn off. With the recent updates it started running a lot better. No more background CPU hogging processes. Hope the improvements keep on coming.