My first steps into Ruby and OpenCV(on OSX)

I was writing some robotic code that will run on a mac mini, so I tryied on my macbook for the first time, ruby binding to opencv. It was a weird experience lol.

First, to install OpenCV, brew do the job:

$ brew tap homebrew/science
$ brew install opencv

after a while we see that instalation goes flawlessly:
/usr/local/Cellar/opencv/2.4.13.2: 278 files, 35.6MB
and we have current version: 2.4.13.2. That said, now install ruby gem:

$ gem install ruby-opencv -- --with-opencv-lib=/usr/local/Cellar/opencv/2.4.13.2/lib \
                             --with-opencv-include=/usr/local/Cellar/opencv/2.4.13.2/include/opencv \
                             --with-opencv-include=/usr/local/Cellar/opencv/2.4.13.2/include/opencv2

Fetching: ruby-opencv-0.0.18.gem (100%)
Building native extensions with: '--with-opencv-lib=/usr/local/Cellar/opencv/2.4.13.2/lib --with-opencv-include=/usr/local/Cellar/opencv/2.4.13.2/include/opencv --with-opencv-include=/usr/local/Cellar/opencv/2.4.13.2/include/opencv2'
This could take a while...
Successfully installed ruby-opencv-0.0.18
Parsing documentation for ruby-opencv-0.0.18
Installing ri documentation for ruby-opencv-0.0.18
Done installing documentation for ruby-opencv after 7 seconds
1 gem installed

after that, time to code.

My first try, obviously, is to use HAAR cascade classifier to look for faces.

require "rubygems"
require "opencv"
include OpenCV

window = GUI::Window.new("grab da face!")
camera = CvCapture.open
detector = CvHaarClassifierCascade::load('./haarcascade_frontalface_alt.xml')
loop {
  image = camera.query
  detector.detect_objects(image).each { |rect|
    image.rectangle! rect.top_left, rect.bottom_right, :color => CvColor::Blue
  }
  window.show image
  break if GUI::wait_key(100)
}

It didn’t worked as ruby-opencv currently supports only older type format of trained data xml. To solve that, I grabed older version of haarcascade_frontalface_alt.xml from https://raw.githubusercontent.com/Itseez/opencv/2.4.10.4/data/haarcascades/haarcascade_frontalface_alt.xml and it worked as expected.
Sometimes, OSX Facetime camera stop working, but is easy to fix, just run

sudo killall VDCAssistant

wait a little and voilá.

In the end, was a different experience, it worked but was way too slow. I’ll try to tweak a little but python version are way faster and I believe I’ll keep using it.

Accessing Go compiled applications through FFI

I finally started to create some useful Go code and, just like some previous posts(in pt-BR, ) where I integrated python, lua and C, I would love to use them with old python code instead of C/C++. For that, I had to generate a shared library. The process is easy as expected, requiring only the import of library “C” and a comment before function definition, exporting the function name, as following examples.

package main

import "C"

//export ModXY
func ModXY(x int) int {
        return x * 2
}

func main() {}

Now, let’s compile it to generate a shared lib:

$ go build -o libmod.so -buildmode=c-shared

This results on two files, libmod.so and libmod.h where the first one is a shared library itself and the second one is the headers to include Go types into a C application. More on this soon. First let’s check libmod.so

$ file libmod.so
libmod.so: Mach-O 64-bit dynamically linked shared library x86_64

Nice, as expected on OSX. Now let’s try to import it with python ctypes!

Python 2.7.12 (default, Sep 28 2016, 18:41:32)
[GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.38)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import ctypes
>>> lib = ctypes.CDLL('libmod.so')
>>> lib.ModXY(13,2)
1

It worked like a charm! It is, actually, pretty easy to deal with. Now let’s find out the raison d’etre of this .h file. If, for some crazy reason, you need to embbed Go code inside a C file, simply include the .h and use the functions normally, as the following example:

#include <stdio.h>
#include "libmod.h"

int main(void){
  printf("the rest of division of 13 per 2 is %d", (int)ModXY(13,2));
  return 0;
}

then, compile it with:

$ gcc -o purexecutable testlib.c libmod.so 

and

$ file purexecutable
purexecutable: Mach-O 64-bit executable x86_64
$ ./purexecutable
the rest of division of 13 per 2 is 1

Yey! Again, simple and efficient. The design documentation of this feature may be found here( http://golang.org/s/execmodes ). That’s all

Let it go? Let it go!

Lately, I added Go as one of my favourite programming languages and I’ve tried to rewrite e create new projects in Go. Basically, the reasons Go became one of my favourite languages is that, although the language/compiler/tools is opensource, different from a majority of opensource projects (JS mainly), it is backed up by a huge corporation (Google) and is easy to avoid major mess (hello JS, my old friend). Also, Go is fast (really, really fast), simple (like C, not like python), concurrent and cross-platform by design, what makes Go a great general pourpose language. If all these previous reasons don’t catch your heart, Go is designed by Rob Pike and Ken Thompson, the same guys who designed UNIX and C, so there is no way this would be bad (fanboy attack?). Another very specific issue I had with c++ and libboost: take 3-8 hours compiling libraries whose Go version took 5 seconds, made me embrace Go as a friend.

I saw Go, from very begining, as a “pythonic C” and, for a python guy’s perspective, I loved it. And, being able to handle binary data just like C but structured data just like python, made me fall in love.

In the end, I started to use Go as a main language, I hope to gain more wisdom on it and become a better Gopher.

A study in software modification and automation by integrating Paterva’s CaseFile and Maltego

Maltego is an excellent intelligence and data visualization tool, but the need of being online(and the requirement of an registered account) make it pretty much useless for more sophisticated usages(like crime investigation or any other private and offline usage). For this, CaseFile was created, which is, basically, Maltego offline without the transforms. But transforms(offline and over closed-source data) would make the life of analysts easier in checking/validating information. To have the best of two worlds, I’ve extracted the transform engine from Maltego and inserted on CaseFile, also, removing the need of login and information leakage that the tool usually allow, making trustable and usable even by the government.

First, Maltego is a great tool and everyone who likes it should support Paterva and buy licenses. This is a study in software modification, automation and should not be used to cause harm to Paterva’s copyrights/busines model by any means. That’s why I’m using old versions of Maltego and Casefile.

The versions used was:

  • Maltego 3.1: maltego-3.1.1_CE-2012-04-11.zip md5 400b427652ca3e8ed60a6d6b7a457e81
  • CaseFile 1.0: maltego-CF.1.0.1_community-2012-03-14.zip md5 8d009eae5c899d74458712fe0e1458e1

They were originally downloaded from:

The base tools used was:

  • Jasmin ( http://jasmin.sourceforge.net/ ), an assembler for the Java Virtual Machine. It takes ASCII descriptions of Java classes, written in a simple assembler-like syntax using the Java Virtual Machine instruction set. It converts them into binary Java class files, suitable for loading by a Java runtime system.
  • ClassFileAnalyzer ( http://classfileanalyzer.javaseiten.de/ ), an analyzer and disassembler (Jasmin syntax 2) for Java class files.

First, I removed the annoying background image at maltego/modules/locale/com-paterva-maltego-ui-graph_maltego.jar.

Then, I’ve copied some files related to transform from Maltego to CaseFile:

  from maltego-ui/
    com-paterva-maltego-transforms-standard
    com-paterva-maltego-transform-protocol-v2
    com-paterva-maltego-transform-manager
    com-paterva-maltego-transform-finder     # needed by com-paterva-maltego-transform-manager
    com-paterva-maltego-transform-discovery  # needed by com-paterva-maltego-transform-protocol-v2
    com-paterva-maltego-transform-runner     # needed by com-paterva-maltego-transform-protocol-v2
  from maltego-core-platform/
    com-paterva-maltego-typing               # for com.paterva.maltego.typing.TypeNameValidator

The next step was define a series of modifications and fine tunning on CaseFile:

  • remove savetoserver and fake transform
  • remove startpage website
  • remove server discovery from manage transforms toolbar
  • always use trivialurldisplay
  • make showURL a stub in trivialurldisplay
  • remove google-me and wikipedia-me actions
  • remove all discover transforms actions
  • remove lots of webbrowser actions
  • enable transform limit toolbar

For that, I’ve put all modifications in .diff, .jdiff or .java where .diff is applied by standard POSIX patch utility, .jdiff is recompiled with jasmine and .java is compiled with jdk’s javac and integrated into target application.

Instructions:

  • Run cfpatch:
    ./cfpatch maltego-3.1.1_CE-2012-04-11.zip maltego-CF.1.0.1_community-2012-03-14.zip CF-custom.zip
  • Extact CF-custom.zip to the place where you want to install the custom CaseFile.

Extra
Create an .java file with the header:

//target-contains: com/paterva/maltego/graph/MaltegoGraph.class
//filename: org/hopto/im/Test.java

to integrate a new piece of software on Casefile. Example:

//target-contains: com/paterva/maltego/graph/MaltegoGraph.class
//filename: org/hopto/im/Test.java

// the above lines are to indicate that this class will be added
// in the same JAR module that file com/paterva/maltego/graph/MaltegoGraph.class
// and that the original name of this file is org/hopto/im/Test.java

package org.hopto.im;
public class Test {
    public static void main(String[] args) {
        System.out.println("Hello");
    }
}

All the working scripts and tools can be found at: https://github.com/paoloo/casefile-extender