XYZ: Communicating with the XCarve

The first step was adding a few simple shapes on Easel and then exporting the G-Code just to get an idea of what kind of commands are sent to the XCarve machine.

After looking up the G-Code commands referenced in the script (G0, G1, G20, G90), I opened a Python REPL and, using the pySerial library, started testing them out:

I proceeded to write a Python interface for XCarve to start experimenting with drawing:

Then I started thinking about how to integrate the control of the bot with face recognition. We mounted a webcam on top of the XCarve and then I used OpenCV to do eye and mouth detection.

I needed a robust way to convert pixel coordinates from the webcam to drawing coordinates for XCarve, so after considering a few options, I decided to go with ArUco markers, since they’re a tested and tried solution for plane detection. Python turned out to be quite slow when it had to do both AR marker detection + face recognition, and its options for GUI are quite poor, so I decided to try migrating my code to OpenFrameworks, which I had very little prior experience with. It turned out to be more straightforward than I thought, mainly because of the prolific ecosystem around it: I used ofxFaceTracker to track the face in the photo, ofxAruco to detect the AR marker, and ofSerial to communicate with the machine. Additionally, I used ofxGui to create a simple GUI which which we could send commands to the bot in real time.

Once we got the bot to draw an X on the subject’s eyes, we were ready to make the bot actually draw various patterns on the face features. Carlie used Processing to create various patterns that the machine would draw then sent me the sketch so I can translate it into openFrameworks and G-Code.

To make the translation from Processing easier, I wrote a Processing-esque interface for the drawing bot whihc allowed one to use shape functions like rect(), ellipse(), as well as matrix transformation functions like pushMatrix() and translate(), to draw using the XCarve in a way similar to the way they would draw on the screen.