Sunday, April 16, 2017

Using Docker Containers to build C++ Projects

With everyone moving to the docker bandwagon, why should you feel left behind. If you have your project hosted on BitBucket.org, make sure that you do make use of the pipeline feature to ensure that the project/product is always in the sane state.

Here is a sample bitbucket-pipelines.yml file to get you going with building C++ projects.

# This is a sample build configuration for C++ – Make.
# Check our guides at https://confluence.atlassian.com/x/5Q4SMw for more examples.
# Only use spaces to indent your .yml configuration.
# -----
# You can specify a custom docker image from Docker Hub as your build environment.
image: gcc:6.1

pipelines:
  default:
    - step:
        script: # Modify the commands below to build your repository.
          - make
          - make test

Well this is not going to be very useful if you use other libraries like boost, or qt in your project. Because the basic default image that BitBucket uses has tools like gcc, python, maven, npm, java etc(Read more about the image here). It does not back with it other third party libraries (like boost).

What do you do? You have the following options at hand for specifying the image:
  1. the default image bitbucket provides(which is not very useful. Read here about the image), 
  2. your own image from docker hub (public/private) 
  3. specify am image hosted in private registry. (for detailed instruction read here)
Well for our little POC project where we were testing the feasibility of making use of Pipeline features we decided to use ilssim/cmake-boost-qt5  image.

CAUTION: Always check the DockerFile of the image when using someone else's image for security issues. Its always better to build your own image and host it on Docker hub and use it.

Coming back to my example, our POC project had reference to boost threading libraries, so here is my sample bitbucket-pipelines.yml file:


# This is a sample build configuration for C++ – Make.
# Check our guides at https://confluence.atlassian.com/x/5Q4SMw for more examples.
# Only use spaces to indent your .yml configuration.
# -----
# You can specify a custom docker image from Docker Hub as your build environment.
image: ilssim/cmake-boost-qt5

pipelines:
  default:
    - step:
        script:
          - echo "This script runs on all branches that don't have any specific pipeline assigned in 'branches'."
  branches:
    master:
      - step:
          script:
          - make
          - make test
    feature/*:
      - step:
          script:
          - cd orchestrationService
          - cmake
          - make



You can read more about branch workflows here.

Thursday, March 30, 2017

Block UDP Traffic via IPTables

While testing an application to verify all code flow paths, one of the scenarios demanded that the application handle dropped UDP packet stream. Now we had a recorded UDP stream at hand which we could play at will.

To simulate UDP packet drop, we added a rule in IPTables to drop/block all UDP packets destined for our UDP destination port 10222. The following command does it for you. 

To Block all udp traffic destined for port 10222
[root@paragpc ~]# iptables -A INPUT -p udp -i eth1 --dport 10222 -j DROP
where:
  • -A is for add/Append iptable rule
  • -p is for protocol
  • -i  = --in-interface eth1  (there is similarly -o =--out-interface
  • -j = JUMP
Refer the man page of iptables for more details. 

However don't forget to remove the rule once your test case is over.
Remove iptable rule dropping udp traffic destined for port 10222 [-D is for Delete iptable rule]
[root@paragpc ~]# iptables -D INPUT -p udp -i eth1 --dport 10222 -j DROP
However if you wish to permanently save the rule run the following command(source).
service iptables save



Thursday, February 16, 2017

[Solution] Password less ssh connection not working after upgrading Git Bash in windows

After I had updated git bash with version 2.11.1 on windows my .ssh/config files was overwritten (wiped clean) along with the public-private keys entries for ssh passwordless authentication.

Good thing I had a backup of my .ssh/config and id_pub keys with me.
Simply populating the .ssh/config file, resolved the ssh hostname but still it asked my for my password. Then I added the following entries for IdentityFile at the end of .ssh-config file


Host *
    ServerAliveInterval 300
    ServerAliveCountMax 2
    Compression yes
    CompressionLevel 9
    GSSAPIAuthentication no
    ForwardAgent yes  
    IdentityFile ~/.ssh/id_dsa
    IdentityFile ~/.ssh/vm_private_key

where  id_dsa and vm_private_key were the private ssh keys for ssh authentication.
Again simply running ssh-add <Path to key did not work. It kept complaining about :   

parag@paragpc MINGW64 ~
$ ssh-add -l
Could not open a connection to your authentication agent.

So I ran the following command (source) :

parag@paragpc MINGW64 ~
$ eval $(ssh-agent -s); ssh-add /c/Users/parag/.ssh/id_dsa


Sunday, December 04, 2016

[Resolved] 'AmqpClient::AmqpLibraryException' in SimpleAmqpClient

I have an application using SimpleAmqpClient, which was working fine in our dev environment. when I moved it to a staging environment, it started failing with the following exception.


terminate called after throwing an instance of 'AmqpClient::AmqpLibraryException'
 what(): a socket error occurred
Well nothing had changed (don't we hear this all the time from devs, we changed nothing in that RabbitMQ/AMQP module), and the error looked cryptic with no details.
Well it turned out that, a (new) login account had been created for connecting to the RabbitMQ server, and it hadn't been enabled to connect from a remote host. Turned it on via the configuration file.
And viola, things started working :) . Read more about RabbitMQ configuration here

Sunday, October 16, 2016

Creating tunnels over ssh - Connecting to remote machine ports over internet

Sometimes when I am working from home, I need my application, which generates data, to be able to send it(data) from my laptop to some machine in the office. For eg. Parsing XML files and storing it to a DB on the office LAN, or talking to web-service deployed in our staging environment, etc.

You can use a VPN to connect to your office LAN. Or you can use this simple tunneling solution that comes with ssh and get going.
From one terminal connect to your office machine (paragOfcPc).

parag@paragpc:~# ssh paragOfcPc -L 10000:192.168.42.43:12871
Last login: Tue May 16 14:41:01 2016 from 172.168.XX.YY
[root@paragOfcPc ~]# ssh 192.168.42.43
Last login: Tue May 16 08:08:51 2016 from 172.168.XX.YY
[root@SQL43 ~]# nc -lkv 12871
Connection from 10.1.1.51 port 12871 [tcp/*] accepted
This forwards the local port 10000 to port 12871 on some server on my office LAN: 192.168.42.43, which is different from my office pc : paragOfcPc. The nice thing about using ssh is I get the added benefit of being on a secure encrypted connection.

From another terminal you can test it with :

parag@paragpc:~# cat hello.txt
This is a demo message for the connection
Send more messages to the remote server
parag@paragpc:~# cat hello.txt | nc localhost 10000
parag@paragpc:~#
After you execute the second command you should now see the content of the file hello.txt in nc -lkv output.

In case you want to send data only to your office pc (in my case paragOfcPc), you can substitute 192.168.42.43 with localhost while creating the tunnel with -L parameter

Sunday, September 04, 2016

C++ Identifying threads with names

In multi-threaded applications, many a times during debugging or logging, we wish to identify the threads by name[functionality ]. Simply printing the executing method name in logs does not help if the functions are common between threads doing different tasks. Also you may want to see the currently running threads from either top or ps in your application.

Here is sample program to tag your threads with proper names. Use prctl defined in <sys/prctl.h> , to get/set thread names. Keep in mind that the name can be up to 16 bytes long only. 

#include <stdio.h>
#include <pthread.h>
#include <stdio.h>
#include <iostream>
#include <unistd.h>
#include <stdlib.h>
#include <sys/prctl.h>

//compile the program with : -lpthread

using namespace std;

void * f1(void *) {

 try{
  prctl(PR_SET_NAME,"Thread_name_f1",0,0,0);
  printf("f1 : Starting sleep\n");
  // ..
  // Code goes here
  // utils();
  // g();
  // ..
 }
 catch(std::string &s){
  char thread_name[17]={0};
  prctl(PR_GET_NAME,thread_name,0,0,0);
  std::cout << "Thread:" << thread_name <<"crashed. Reason:" << s << std::endl;
 }
}

void * f2(void *) {

 try{
      prctl(PR_SET_NAME,"Thread_name_f2",0,0,0);
  printf("f2 : Starting sleep\n");
  // ..
  // Code goes here
  // a();
  // utils();
  // ..
 }
 catch(std::string &s){

  char thread_name[17]={0};
  prctl(PR_GET_NAME,thread_name,0,0,0);
  std::cout << "Thread:" << thread_name <<"crashed. Reason:" << s << std::endl;
 }
}

int main() {
 pthread_t  f1_thread, f2_thread;
 pthread_create(&f1_thread, NULL, f1, NULL);
 pthread_create(&f2_thread, NULL, f2, NULL);

 printf("Main : Starting sleep\n");
 sleep(50);
 pthread_join( f1_thread, NULL);
 pthread_join( f2_thread, NULL);
 printf("Main : Done sleep\n");
 return 0;

}

Sunday, August 28, 2016

Removing deleted remote branches from local repo

Sometimes we have a POC(Proof of concept) branches, which are forked off the main/feature branches. After a while when POC is complete, we/the dev no longer uses it.
Now over time the no of such POC branches increase and the repo doesnt look good.
So during our periodic maintenance and cleaning of GIT repo, we delete all such POC branches.

How to sync your local branch list.
Simple:

[parag@paragpc: ~/ProjectCodeName] git fetch -p
 x [deleted]  (none) ->origin/POC-webCache
 x [deleted]  (none) ->origin/POC-CDNChange
[parag@paragpc: ~/ProjectCodeName] 
where -p = --prune
                     After fetching, remove any remote-tracking branches which no longer exist on the remote. 


Similarly, if you are have created a local branch and committed some changes while the remote tracking branch is not present at origin, issue:
[parag@paragpc: ~/GitProject] git branch -d <the_dirty_poc_branch>

where -d = --delete
        Delete a branch. The branch must be fully merged in its upstream branch, or in HEAD if no upstream was set with --track or --set-upstream

PS: Before running any delete related operation on branch its better to double check and understand what you are doing!.


Using Docker Containers to build C++ Projects

With everyone moving to the docker bandwagon, why should you feel left behind. If you have your project hosted on BitBucket.org , make sure...