The final step was to hack the desktop software that controls the MakerBot – ReplicatorG.

What is ReplicatorG?

From the ReplicatorG website:

[ReplicatorG] is the software that will drive your CupCake CNC, RepRap machine, or generic CNC machine. You can give it a GCode or STL file to process, and it takes it from there. Its cross platform, easily installed, and is based on the familiar Arduino / Processing environments.

For my purposes, ReplicatorG provides two things. First, RepG is a user interface for controlling the MakerBot hardware:

Second, RepG reads G-code files describing how to build an object, and transmits them to the MakerBot over the USB:

Of course, ReplicatorG is open source, and the code is available on GitHub! So, it was simple to clone their repository and start hacking on it myself.

Camera Control via ReplicatorG

While it was relatively simple to update the extruder controller firmware to make it camera-aware, ReplicatorG is a bit more complicated. My first goal was to hack a new “Camera” checkbox into the control panel. Whenever the box was checked, the camera would take pictures. Whenever the box was unchecked, the camera would be idle.

You can find the code required for these changes in this commit on GitHub, but I will try to briefly break them down here:

• Define a new machine. In the machines.xml.dist file, I defined an experimental MakerBot configuration named “EXPERIMENTAL – Cupcake CNC w/HBP and remote camera”. It is essentially a copy of the typical MakerBot configuration with a heated build platform, but in the <tool> definition, I also added a camera="true" attribute.
• Update the tool model. In ToolModel.java, I added code to represent whether the tool has an attached camera, whether the camera is activated, and how to parse the camera attribute out of machines.xml.
• Update the machine driver model. In Driver.java, DriverBaseImplementation.java, and Sanguino3GDriver.java, I added the definitions and implementations to triggerCamera() and stopTriggeringCamera(). This is the code that actually sends the TOGGLE_CAMERA serial command to the extruder controller, which I also defined in ToolCommandCode.java.
• Update the control panel interface. In ExtruderPanel.java, I added the code to draw a new label and checkbox named “Camera”, if the machine is configured for a camera, and to respond to check/uncheck events by calling triggerCamera() or stopTriggeringCamera().

Compiling and Running the new ReplicatorG

Compiling ReplicatorG is pretty simple, so long as you have a reasonable JDK environment and have Ant on your path. There are basically two steps:

1. Copy machines.xml.dist to machines.xml.
2. Run the proper dist-linux.sh, dist-mac.sh, or dist-windows.sh.

ReplicatorG will be compiled and packaged up into the dist/ directory in two forms: an installable package for the chosen platform, and an unpacked version that you can run directly.

Opening up my modified version of ReplicatorG, I selected the “EXPERIMENTAL – Cupcake CNC w/HBP and remote camera” profile from the Machine -> Driver menu, opened up the control panel, and was happy to see this:

After hooking up my camera to the extruder controller’s D9 port, and starting the Remote Button script on the camera, I was able to take pictures by quickly toggling the camera checkbox on and off. I could also leave the checkbox activated to make the camera take pictures continuously.

Automatic Triggering with G-codes

Being able to trigger the camera by hand is all well and good, but my goal was to take pictures automatically at the end of every layer. To do this, I needed to be able to embed camera trigger commands in the G-code for building each individual object.

Looking at the ReplicatorG G-code docs, and the (machine-specific) M-code docs, I chose two codes for working with the camera:

• M150 – Trigger Camera
• M151 – Stop Triggering Camera

I may have to change these in the future, as the main ReplicatorG development team claim G- and M-codes for other features, but these work for now.

Modifying ReplicatorG to accept these M-codes (GitHub commit here) was straightforward: update GCodeParser.java to recognize the codes, and call the appropriate triggerCamera() and stopTriggeringCamera() methods.

I could now construct a G-code file which, when “built” in ReplicatorG, would take a picture on demand:

M150     (trigger the camera)
G4 P700  (wait 0.7 seconds for the camera to activate)
M151     (stop triggering)
G4 P1300 (wait 1.3 seconds for the camera to finish)


Finally, it was time to edit up the G-code for the models I want to photograph.

Typically, G-code is generated by taking a 3D object in STL format and running it through the Skeinforge tool. Skeinforge is a set of Python scripts, which means it is not too difficult to insert your own code.

For now, however, I decided to make a simple hack using a Perl script I wrote called add_camera_events.pl. It works by looking for (</layer>) comments, which signal the end of a layer of printing, and inserts lines to:

1. Move to a standard pose (X=0, Y=-45),
2. Trigger the camera and wait for it to finish, and
3. Move back to the original position

You can find the source for this script in the add_camera_events.pl gist. The source for all of my changes to ReplicatorG are on GitHub, in the “schmarty-camera” branch of my fork of ReplicatorG.

And with that, the computer aspect of this system was finally done!

Wrap Up

Phew! So far I’ve hacked a camera, wired it to the MakerBot, updated the MakerBot firmware to trigger it, updated ReplicatorG to trigger it, and written a script to update G-code files with camera triggers at the end of each layer.

So… does it work? You bet! Stay tuned for more examples and a breakdown video of this whole project in the final post in this series!

• Part 1: Remote control camera with CHDK
• Part 2: Wiring it up!
• Part 3: Updating the MakerBot Firmware
• Part 4: Updating ReplicatorG [YOU ARE HERE]
• Part 5: Printing with the Camera!

While I know I should be finishing my MakerBot time-lapse camera series, I took some time for another project to play with some Processing. The above image was rendered in Processing, in real time in just couple of minutes!

Basically, I wanted to take a simple shape, defined by an SVG path, and fill it with images of 3D objects loaded from STL files. Specifically, many wonderful MakerBot-printable objects from Thingiverse!

After some Googling around, I found out that this problem is basically a space-filling problem, similar to an excellent Processing sketch named Scattered Letters by Algirdas Rascius, but with a twist.

The basic algorithm is:

• Load an SVG and render it to an off-screen buffer
• Set curr_size, the size that STLs should be rendered, to max_size
• Choose a random STL model, give it a random orientation, and render it at the current size to an off-screen buffer
• Try several times to place this model by giving it a random x,y position and checking it for a good fit:
  • Each non-background pixel of the model’s off-screen image should fit within the non-background pixels of the SVG’s off-screen image.
  • Each non-background pixel of the model’s off-screen image should NOT overlap with any non-background pixel of the main display.
• If a fitting position is found, render the model to the display.
• Otherwise, shrink curr_size by a step and choose a new model.
• If we drop below min_size, we should stop.

You can find the code for my sketch, which I call ThingiverseCollage, on GitHub. To make it work, you’ll need to follow the installation instructions in the README to install my (very slightly) modified version of the unlekkerLib for STL loading and rendering. I modified it to allow rendering to a PGraphics object, since it originally only allowed rendering to the main PApplet.

A note on STL files: unlekkerLib only loads STL files in the binary format. It chokes dramatically on ASCII STL files, such as those exported from OpenSCAD. I was able to use Zaggo’s excellent Pleasant3D to load ASCII STLs and re-save them, which converts them to binary STLs. As a bonus, Pleasant3D also allows you to orient objects in a way that will make them look most interesting when they are rendered down to 2D in the final image.

An example M.svg, as well as several objects from Thingiverse are included with the code to get started. To use your own SVGs, I have had good luck using Inkscape to draw or import shapes, and save them as the native “Inkscape SVG” or “Plain SVG” formats. Some files might require hand-tweaking; for example, if the width and height header values are something like "100%" instead of a pixel value.

There is also some simple configuration in the sketch to allow the export of PDF files. This is nice because the resulting PDF has full vector data, making it easily rescaled to any size you wish. Unfortunately, the current PDF renderer for Processing renders each triangle of each STL model as a separate path, generating very complicated vector output, which tends to bring Inkscape to its knees. I have had some luck with importing those files, rastering them out to PNG at a high resolution (e.g. 600 dpi), and using Inkscape’s “Trace Bitmap” functionality to re-vectorize them, though this requires some cleanup by hand.

Anyway, this has been a fun little diversion for me for the last couple of days. I hope that you folks find it useful! Post your awesome pictures in the comments, here!

The next step was to update the software on the extruder controller so that it could activate (and deactivate) the camera, in response to commands from the motherboard.

An aside on MakerBot communications

The MakerBot electronics ecosystem is comprised of 3 parts: your computer, the MakerBot’s motherboard, and the extruder controller board. Your computer talks to the motherboard via a USB<->TTL interface (such as this FTDI cable from SparkFun). In turn, the motherboard communicates with the extruder using another serial protocol, RS-485, over an ethernet cable. Finally, the extruder triggers the camera via the custom cable I made in the previous post.

The software for all three components is available on the indomitable GitHub. The software for your computer is called ReplicatorG, and the source can be found in the MakerBot ReplicatorG GitHub repository. I’ll talk more about ReplicatorG in the next post in this series. For now, we want to focus on the MakerBot G3Firmware GitHub repository, which contains the code for the motherboard (in the SanguinoMaster subdirectory), and for the extruder (in the ArduinoSlaveExtruder directory).

Browsing through the code, we see that these components use their serial interfaces to send packets, where each command is represented by a number. The commands for the motherboard can be found in the SanguinoMaster/Commands.h, and those for the extruder can be found in ArduinoSlaveExtruder/PacketProcessor.cpp.

To send a message to the extruder – in this case, to activate or deactivate the camera – we must create a packet for the motherboard. The HOST_CMD_TOOL_QUERY code allows us to send the motherboard a packet which it will then pass along to the extruder controller.

That’s great, because it means the motherboard part of this software hack is done!

In fact, we’ve already hacked the camera, as well, so we’re halfway there!

Hacking a camera into the extruder controller

Since the motherboard already does everything we need (passes along packets from the computer to the extruder controller), we only need to update the ArduinoSlaveExtruder code.

To get this to work, I ended up changing the following files:
ArduinoSlaveExtruder/Configuration.h.dist – added in configuration options for enabling the camera and setting the pin on which to activate it.
ArduinoSlaveExtruder/Extruder.h – added function definitions for turning on/off the camera.
ArduinoSlaveExtruder/Extruder.cpp – actually implemented turning on/off the camera.
ArduinoSlaveExtruder/PacketProcessor.cpp – implemented the serial command to toggle camera.

Building and uploading

If you followed the 4 links above, you’ll notice that they go to my own G3Firmware GitHub repository. You can download it yourself to play along by cloning the repository and checking out the ECv2.3rc0-camera branch.

To build the firmware and upload it to the extruder controller, we need some common development tools (make, in this case), and the Arduino development environment. With those things installed, we can compile everything by setting the ARDUINO_HOME environment variable to the path to our Arduino install’s java directory (e.g. on OS X this would be /Applications/Arduino.app/Contents/Resources/Java/), and simply run make.

Once the firmware has been compiled, we can upload it to the extruder controller by using the USB<->TTL cable that usually connects the motherboard to our computer. Plug the cable into the extruder controller, and run the make upload command. You’ll need to make sure that ARDUINO_HOME is set, and you will probably need to alter the Makefile to specify the correct serial port, and maybe to update the call to avrdude to include the path to the Arduino avrdude config file. You can see an example of that in this commit.

Once the firmware is uploaded to the extruder controller, the MakerBot is all set to take pictures!

… Of course, we still have no way to tell the MakerBot to take a picture, so stay tuned for that information in the next update:

• Part 1: Remote control camera with CHDK
• Part 2: Wiring it up!
• Part 3: Updating the MakerBot Firmware [YOU ARE HERE]
• Part 4: Updating ReplicatorG
• Part 5: Printing with the Camera!

On Thursday, March 25th, I spoke about desktop fabrication and the MakerBot Cupcake CNC 3D printer at Dorkbot Pittsburgh. After some slides, I gave a printing demo with my Cupcake, Makerbot #131. You can find my slides above. I’ll post the video when it becomes available.

CHDK’s remote USB trigger functionality works by detecting when it receives power over USB.  This happens when two wires inside the USB mini-B cable are connected to power: the red wire gets 5 volts, and the black wire gets connected to ground.

So, I chose to find somewhere on the MakerBot’s electronics to hook up these power and ground wires such that the MakerBot could control when the red wire receives 5V.  The MakerBot uses RepRap Generation 3 electronics, and let me tell you, the gen 3 electronics documentation is fantastic!  Unfortunately, the docs for the main motherboard reveal that there are some free I2C headers for connecting serial devices, but no free general I/O pins.

Luckily, the extruder controller docs show two free digital pins, conveniently broken out with 5V and ground connections next to them.  These are digital pins D9 and D10.  According to the docs, they are intended for hooking up servo motors, but they would absolutely work for my purposes!

The layout for pins D9 and D10 goes (from left to right): I/O pin, 5V, ground.  Since I wanted the data pin itself to provide the 5V, I chose to make a cable using a 3-pin piece of female header, soldering the red wire connecting to the I/O pin (on the left) and the black wire connecting to the ground pin (on the right).  The center pin has no connection.  You can see my “super fancy” cable on the left.

I know this post isn’t particularly about code, so stay tuned for the next parts of this series:

• Part 1: Remote control camera with CHDK
• Part 2: Wiring it up! [YOU ARE HERE]
• Part 3: Updating the MakerBot Firmware
• Part 4: Updating ReplicatorG
• Part 5: Updating Skeinforge

It recently occurred to me that, since the MakerBot is such a hackable platform, I could probably make nice time-lapse videos by taking a picture of each layer.  The idea is to have the MakerBot pose the object after each layer, and trigger a camera to take a snapshot.

When discussing this idea with fellow HackPittsburgh member Matt Mets, he recommended I try out a Canon camera with the Canon Hack Development Kit (CHDK).  Making a quick stop on eBay, I soon had an SD300 in my hands, ready to be hacked!

I started by figuring out which firmware my SD300 was running, which told me which version of CHDK to download for my camera’s particular model/firmware combo.  Using an SD card reader, I copied the contents of the CHDK zip file onto my camera’s SD card and followed the instructions to make CHDK auto-load when the camera starts up.  I also changed my CHDK firmware settings to set “Disable LCD off” to “[Script]“, so the camera wouldn’t shut down on its own.

Once CHDK was loaded and configured, I followed the USB Remote Cable instructions from the CHDK wiki.  The basic idea is to set Enable Remote to on, and load a script that is ready to handle USB remote events.  The one on the wiki page didn’t work as-is for me, presumably because my camera has half-shoot (i.e. focus and charge flash) and full-shoot (take picture) settings.  Here is the result that worked for me:

As the script says, I now turn the camera on in record mode, disable the flash, and start the script.  After that, the camera will automatically take a photo whenever I plug in the USB to my computer.

More from this series:

• Part 1: Remote control camera with CHDK [YOU ARE HERE]
• Part 2: Wiring it up!
• Part 3: Updating the MakerBot Firmware
• Part 4: Updating ReplicatorG
• Part 5: Printing with the Camera!
]]> http://creatingwithcode.com/howto/makerbot-cam-1-chdk/feed/ 2 http://creatingwithcode.com/howto/face-detection-in-static-images-with-python/ http://creatingwithcode.com/howto/face-detection-in-static-images-with-python/#comments Fri, 20 Feb 2009 21:31:22 +0000 Robert McGuire http://creatingwithcode.com/?p=16 One of the things I’ve been longing to do with my mobile photo-sharing site Camura is to offer image annotations, like objects and faces.  Over the last couple of years I have been increasingly frustrated by the appearance of face tagging on services like Facebook, and the recent addition of face recognition to iPhoto has brought this frustration to the surface once again.  I don’t even want to do something as complex as face recognition – I just want to find faces in an image.

Googling for things like “open source face detector” doesn’t come up with much.  The landscape seems to be comprised of mostly expensive for-pay libraries written for Windows, abandoned research projects, and lots of research papers full of equations — but no code that I could get to run.

To make a long post short, it turns out that Intel’s OpenCV computer vision library comes with a face detector example that should work out of the box.  Better yet, there are now some decent Python bindings for OpenCV that come pre-packaged with OpenCV for Ubuntu and Debian.  You can install them with:

$ sudo apt-get install python-opencv


Now, it seems that most OpenCV face detector examples are meant to be run “live”, usually taking the image from a webcam and highlighting faces with a red box in real-time.  However, I have a large database of static images that I want to consider individually, and I simply want to save the face coordinates for later use, rather than altering the picture.

So, with a bit more Googling, I found a Python script that I could chop up and use for this purpose, and here is what I came up with:

An example run of the script looks something like this:

$ python face_detect.py marty_mcguire.jpg
[(50,36) -> (115,101)]


You can overlay that rectangle on an output image with ImageMagick’s “convert”:

$ convert marty_mcguire.jpg -stroke red -fill none -draw "rectangle 50,36 115,101" output.jpg


And the output might look something like this:



Pretty fun stuff!

]]> http://creatingwithcode.com/howto/face-detection-in-static-images-with-python/feed/ 11 http://creatingwithcode.com/howto/getting-the-boarduino-working-with-os-x-leopard/ http://creatingwithcode.com/howto/getting-the-boarduino-working-with-os-x-leopard/#comments Thu, 12 Jun 2008 23:57:01 +0000 Robert McGuire http://creatingwithcode.com/?p=13 I’ve been into hardware hacking on and off for much of my life, but I’ve never really had the time and confidence to design and build my own hardware. In recent years, projects like the open source Arduino have been slowly convincing me that I just might be able to do this stuff.

When a recent Make blog article showed nearly step-by-step instructions for building a breadboard-friendly Arduino clone called the Boarduino, I felt compelled to order one and try to get it working with my new MacBook.

I ordered the DC Boarduino kit from LadyAda’s website, along with the USB TTL-232 cable that would connect to my laptop’s USB port. I also ordered the 9V power supply on the site to power the Boarduino.

I found building the Boarduino to be pretty easy (=ahem= with only one screwup on my part) thanks to LadyAda’s detailed instructions. It was the first soldering project I had done in awhile, so I am happy that it went so well.

I next looked to the Arduino OS X guide for the software download and installation instructions. I ended up grabbing version 11 of the Arduino software. After unpacking the .zip file from the site, I installed the FTDI driver and rebooted, then plugged in the Boarduino’s power and connected it to the laptop via USB.

Next, I double-clicked the Arduino application to start it, and nothing happened. After messing around with it on the command line for awhile, I determined that it didn’t like the version of Java I’m using (I have the 64-bit version of Java 6 as my default).

So, to run the Arduino software, I had to temporarily set my preferred Java version to 5.0 via the Java Preferences panel in /Applications/Utilities/Java/. Once I had changed my Java version, the Arduino app started right up with a double-click. Once I was done with the app, I could set my Java preferences back to Java 6.

In the Arduino app, I set it up to communicate with the Boarduino by selecting “Arduino NG or older w/ ATmega168″ under the Tools | Board menu. I then opened up the Blink test program under File | Sketchbook | Examples | Digital | Blink. To load the program onto the Boarduino, I pressed its reset button, then quickly clicked the “Upload to I/O Board” button on the interface.

The red LED on the Boarduino blinked rapidly as data was received, then it started blinking slowly as it ran the Blink program! Hooray!

I’m excited that it’s so easy to get something as powerful and versatile as the Boarduino up and running in just a couple of hours. I’m going to try and think of a couple of projects for it. Whatever I do, I’ll be sure to post about it here.

]]> http://creatingwithcode.com/howto/getting-the-boarduino-working-with-os-x-leopard/feed/ 2 http://creatingwithcode.com/howto/a-couple-of-leopard-configuration-tricks/ http://creatingwithcode.com/howto/a-couple-of-leopard-configuration-tricks/#comments Thu, 12 Jun 2008 23:28:30 +0000 Robert McGuire http://creatingwithcode.com/?p=14 I find Google to be a more and more valuable resource as time goes on, especially when seeking knowledge about how to make this new MacBook act like I want it to.

One thing I noticed early on was that pressing Tab to move around various interfaces, from Finder dialogs to web pages, would only tab to text entry fields. Coming from a primarily Windows background, I am used to tabbing my way to checkboxes, dropdowns, and buttons in my interfaces, so this crippled Tab navigation quickly became annoying.

Thanks to this blog post, I found the option to make Tab move through “All Controls” in System Preferences | Keyboard & Mouse | Keyboard Shortcuts

Another big issue I came up against is the weird Java support that comes with OS X. Because Apple releases their own versions of Java, we OS X users are kind of at their mercy with respect to what we can use and how we can configure it. You can install Java 6 from Apple’s download page, but Java 1.5 will still run by default.

Apple provides a “Java Preferences” configuration utility in /Applications/Utilities/Java/ which lets you change the default version of Java that will be used in browsers and when double-clicking to launch Java apps. However, this utility doesn’t change which version of Java will be found by command-line apps in the terminal, such as Apache Ant.

Another blog post to the rescue! It turns out that you can change your default version of Java for command line apps freely by changing the CurrentJDK and Current symlinks in /System/Library/Frameworks/JavaVM.framework/Versions/. For example:

    cd /System/Library/Frameworks/JavaVM.framework/Versions
    sudo ln -fhsv 1.6 CurrentJDK
    sudo ln -fhsv 1.6 Current

Hopefully this post will be able to help others (or at least my future self).

]]> http://creatingwithcode.com/howto/a-couple-of-leopard-configuration-tricks/feed/ 1 http://creatingwithcode.com/howto/setting-up-rails-and-mysql-on-os-x-leopard/ http://creatingwithcode.com/howto/setting-up-rails-and-mysql-on-os-x-leopard/#comments Thu, 12 Jun 2008 23:08:07 +0000 Robert McGuire http://creatingwithcode.com/?p=12 I recently purchased a MacBook to use as my primary development system. One of my first plans was to get up and running with Rails on this new machine. Given that Leopard (OS X 10.5) comes with Ruby version 1.8.6 and RubyGems 1.0.1, I thought I would be in good shape, but there were a couple of snags. Thanks to the power of Google and other people with similar problems, I was able to get it sorted out. Here’s what I did:
1. Updated rubygems to 1.1.1 with sudo gem update --system
2. Installed the 32-bit (x86, NOT x86_64) version of MySQL (community edition, of course) via the installer on the MySQL download page.
3. Installed the native MySQL driver for Ruby (this was tricky, see below)
4. Finished by installing Rails and its friends via gem install rails

Getting the speedier native MySQL driver for Ruby installed was tricky for two reasons. It seems that by default the installer attempts to compile for 4 architectures, so you have to set an environment variable and pass along a parameter to point to the installed copy of MySQL when calling gem:

sudo env ARCHFLAGS=”-arch i386” \
   gem install mysql -- \
   --with-mysql-config=/usr/local/mysql/bin/mysql_config

If you’ve never seen it before, the ‘–’ after ‘gem install mysql’ means that gem should pass along the next arguments to the programs it uses to build the driver.

The reason for installing the 32-bit version of MySQL (on your fancy 64-bit OS and machine!) is that the version of Ruby that ships with Leopard is apparently 32-bit only. Yikes.

I’ve noticed several little annoying details like this in getting other things to work “like I like” on OS X. I’ll probably end up posting more about them in the next couple of posts.

I know that “how to” posts of the form “I wanted to do X, but Y happened. Finally I fixed it by Z,” are pretty boring. Still, I hope they are useful for preserving the knowledge of how to work around these problems, both for myself and for others.

]]> http://creatingwithcode.com/howto/setting-up-rails-and-mysql-on-os-x-leopard/feed/ 0