Some socket/application debugging tips (Linux, Python)

This is more of a note than a blog post. Recently, I was engaged in debugging a web application that used Flask (a python web framework) and Moses, a statistical machine translation program from academia. At one point the front end, a GUI built on Flask, stopped responding and I didn't want to stop the entire process.

To see what sockets were active,

sudo netstat -ltnp

From there, with the pid in hand,

sudo strace -s 3000 -f -p <PID>

Searching for socket related system calls I was able to get a sense of what was going on. If just wanted to look at network traffic then -e trace=network would have been appropriate.

In this case my application was stuck on recv(6, , ,, where a socket was established but not receiving any information. Since I was using the basic Flask webserver things were single threaded and the app could not return to serving static pages again. The solution here is to shutdown the hanging socket and let the app serve pages as normal...

The socket was shut down with gdb:

#sudo apt-get install python3-dbg gdb
sudo gdb python3 <PID>

With the symbol table loaded we can use the shutdown command with,

call shutdown(6, 2) # safely shutsdown socket, even for multithreaded applications

Connecting to your google compute instance (Google Compute, Google Cloud, SSH)

I signed up for a Google Cloud account today. I have a $300 credit or 60 days, which ever is exhausted first.

I know Amazon AWS can use Docker but I decided to go with Google Cloud because their management options for VMs seemed more complete and less shoe horned into their current offerings. In the end, I was looking for an easily manageable platform for hosting applied research backends and frontends and Google Compute looks like a great start.

So, in the process of hosting a novel Bitcoin related translation system I realized I first needed to push a custom binary to my instance. First we'll make sure ssh is set up and then we will use file-copy to push data to the instance. After some trouble shooting these are the steps I came up with:

• 0) Go to the google compute console. Create a project and an instance. Take note of the automatically generated project-ID (this is different than the project name) and is under the Overview link under the Developers Console)
• 1) Click on API -> Enabled APIs (in the right hand frame), make sure Google Compute is enabled
   
• 2) Click on Google Compute -> Metadata, (right frame) SSH keys. Remove any SSH keys that are there (unless you can already ssh in and don't need to read this entry): Edit, click the X next to the key. This forces everything to start fresh.
• 3) Open up your favorite terminal,
• 3a) Install the gcloud SDK here
• 3b) gcloud auth login # to authenticate your host
• 3c) rm ~/.ssh/google-compute*; gcloud compute instances list  # identify the instance name you want to push to
• 3d) gcloud config set project <project-ID>
• 3e) gcloud compute ssh <instance-name> # this will create a new ssh key pair (then exit the ssh session)
  
• 3f) gcloud compute copy-files <your file path> <project name>:<remote path> --zone <your zone, refer to instance list above> 

Your upload should complete and you can ssh back in to verify that the data exists.

Python (virtualenv), installing OpenCV, Ubuntu 14.04

I recently wanted to install OpenCV with Python 3.4 bindings on my Ubuntu 14.04 system.

I ran into several errors: Can not find Python.h even though python3-dev was installed. And, prior to that, cmake was not picking up the python libraries path.

Through some extensive googling I discovered that cmake 2.8.10 has a known bug with correctly finding the python includes directory. Refer to this bug report.

This issue was resolved by updating Ubuntu's version of cmake (which was ancient) and judicious use of opencv compilation flags. Specifically, I installed a newer cmake with:

wget http://www.cmake.org/files/v3.1/cmake-3.1.2.tar.gz 
tar -zxvf cmake-3.1.2.tar.gz 
cd cmake-3.1.2 mkdir _build 
cd _build 
cmake .. -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/usr
make 
sudo make install 
sudo ldconfig 

and used the following opencv flags to build opencv for my python3 virtual env:

cmake -DBUILD_TESTS=OFF -DBUILD_PERF_TESTS=OFF -DBUILD_opencv_python2=OFF -DBUILD_opencv_python3=ON -DPYTHON3_EXECUTABLE=/home/kwame/py34/bin/python -DPYTHON_INCLUDE_DIRS=/usr/include/python3.4m -DPYTHON3_LIBRARY=/home/kwame/py34/lib/python3.4/config-3.4m-x86_64-linux-gnu/libpython3.4m.so -DPYTHON3_NUMPY_INCLUDE_DIRS=build/numpy-1.9.1/numpy/core/include/ -DPYTHON3_PACKAGES_PATH=/home/kwame/py34/lib/python3.4/site-packages -DWITH_TBB=ON .. 
make -j8 
sudo make install 
sudo ldconfig

I leave off the tests because they are statically linked and bloat the library sizes. I only build opencv for python 3.

The opencv installation was verified as working within Python3 by the following:

ipython
import cv