Saturday, January 30, 2010

Thin-plate spline example

Hello again,
Since there have been some questions about an example, I will show you how to use the CThinPlateSpline-Class. First of all we have to load an image from our disk. The image can be of any size, depth or color format. Just call


// load a nice picture
cv::Mat img = cv::imread("C:\\lena512color.jpg");




Now that we have an image we should set some reference points on which the spline algorithm will evaluate the distortion. Normally you would use a interest point detector, but it was easier to just add some generic points to the dataset.
 
// generate some generic points
// usually you would use a interest point detector such as SURF or SIFT
 
std::vector iP, iiP;

// push some points into the vector for the source image
iP.push_back(cv::Point(50,50));
iP.push_back(cv::Point(400,50));
iP.push_back(cv::Point(50,400));
iP.push_back(cv::Point(400,400));
iP.push_back(cv::Point(256,256));
iP.push_back(cv::Point(150,256));

// push some point into the vector for the dst image
iiP.push_back(cv::Point(70,70));
iiP.push_back(cv::Point(430,60));
iiP.push_back(cv::Point(60,410));
iiP.push_back(cv::Point(430,420));
iiP.push_back(cv::Point(220,280));
iiP.push_back(cv::Point(180,240));



Now we have already done the tedious part, lets create a CThinPlateSpline object that will do all the work for you.

 // create thin plate spline object and put the vectors into the constructor

CThinPlateSpline tps(iP,iiP);

The only thing we have to do now is to call the internal warping function and set the correct parameters.

// warp the image to dst
Mat dst;
tps.warpImage(img,dst,0.01,INTER_CUBIC,BACK_WARP);


To see the results just call the imshow function out of opencv

// show images
cv::imshow("original",img);
cv::imshow("distorted",dst);
cv::waitKey(0);


Because it's quite a tradition I used our good old gal "lena" to show you the advantages of the spline algorithm. Just take a look. 

9 comments:

  1. Excellent work Matthias. I havent had an oppotunity to test the software yet (still running opencv 1.1) but hope to get round to it next week. What is the speed / performance of the software like? Would be interested on testng it with large images..

    ReplyDelete
    Replies
    1. Generally speaking, the computation cost of initially solving the inverse matrix for the thin plate spline is O(n^3) where n is the number of points used in the thin plate spline mapping.

      Delete
  2. How it can be done for Android devices?

    ReplyDelete
  3. Thanks a lot, this makes image warping quite easy!

    This code might not compile with some versions of OpenCV, as reported below, due to a bug in lsh_table.h:

    http://code.opencv.org/issues/2179

    The bug is already fixed, though.

    ReplyDelete
  4. Great work Matthias!
    Just a question; your code could help me in order to overlap 2 faces using they landmark points for mouth, nose and eyes?

    ReplyDelete
  5. Hi, Firstly thank you for sharing, I tried this example and i've got following unresolved external error

    LNK2001: unresolved external symbol public __cdecl CThinPlateSpline::warpImage(class cv::Mat const &,class cv::Mat &,float,int,enum TPS_INTERPOLATION)

    I think its about non-included additional library but i added all opencv libraries and i built as Release. Can you send me your all additional libraries names or do you have another solution about this ?

    Cheers..

    ReplyDelete
  6. Hello, may I have question, does this TPS has wrapper into Python? Thanks

    ReplyDelete
  7. Hi, i have the same question, if any python wrapper available for the mentioned c++ libraries?

    ReplyDelete
  8. CThinPlateSpline tps(iP,iiP);
    Not working on Xcode.

    Undefined symbol: CThinPlateSpline::CThinPlateSpline(std::__1::vector, std::__1::allocator > > const&, std::__1::vector, std::__1::allocator > > const&)

    Is there a way to fix this, can you tell another library to warp images.

    ReplyDelete