Painting with a Robotic Arm for the RobotArt 2017 Competition The project kicked off in January, 2017 with my friend, Simon. We both had an interest in robotics and decided to take part in the RobotArt 2017 competition (Team Starfish Lab). The main aim was to take this chance to create artwork with robotics and share it with others.
Design Concept
The design direction was to create something where the Eastern culture meets with the Western culture. This means using the Chinese water and ink painting style to express western painting, plus the use of a robotic arm.
Build Details
Hardware
With the limited time, we had decided to buy an existing 5 degree of freedom robotic arm. The servo electronics board was kindly supplied by my friend, Simon. It can support many servo motors, while we are just using 4. The pen mount uses a magnet for easy pen change and crash protection. We also created a curved surface to sharpen the brush tip after dipping into the ink each time. The photo below should explain it all as well.
Robotics Software
The servo electronics board supports Mini SSC protocol for moving the motors. Therefore, all inverse kinematics are done on the computer instead in firmware. As we are both familiar with the 3D printer GCode, we decided to reuse the GCode file format and also the Pronterface software. The software flow is shown in the diagram below.
One of the problems we encountered was linking up Pronterface with the Python Servo software we wrote. In linux, you can easily use virtual COM port. However, on Windows (which we are using), all the solutions are propriety. Therefore, we simply used 2 USB to RS232 converters and hooked them up.
Image Processing and Path Generation
The path generation was done with the help of OpenCV. The major function used was the findContour function in order to reconstruct painting paths from the original painting. Commonly used image processing techniques like Gaussian blur and threshold are also used. The path generated for the Abstract Picasso Inspired Portrait can be seen in the picture below.
Drawings
Abstract Picasso Inspired Portrait
One of the painting created was with reference to a Picasso Inspired Portrait. The idea was to mix between the western Cubism and the Chinese water and ink painting style. The painting is able to convey the original details, while bringing out a different style. \
Sine Cosine
‘This is a mathematically created artwork. Although the sine cosine equations are very accurate, the actual painting produced gives a loosely drawn feel. The painting looks like a butterfly or a fidget spinner and it’s up to your imagination. Equations and the painting are shown below.
x = cos(t) + cos(3*t) + 0.4 * cos(11 * t) y = sin(t) + sin(3*t) + 0.4 * sin(11 * t)
Future Improvements
This project has a hard deadline for uploading paintins and there are tons of things that can done. Too many ideas for too short of time. See some of the ideas below.
Drawing Improvements
Typical Chinese painting uses a mixture of water and ink to create different grey scale levels and complex shapes. This project is currently limited to the use of ink only, it will be interesting the use of both in the future. In addition, we had tried to explore autonomous paintings by the robot itself without much success. If the robot is able to digest different styles from famous paintings, then make its own creation. Then the robot will be offered a creative ability to create its own paintings.
Engineer Improvements
As you might notice, the robot itself is not very rigid. We made use of this point in this project to create paintings that look loosely drawn by human. It will be better if we had more control over precise positions and then to program the uncertainties into the motion sequences. We had to write some of the software for controlling the robot. It is fun, but it will be nice to reuse some existing platforms like YARP and ROS. Also, we had not fully explored the image processing as we were limited to single colour and external contours. It will interesting to explore different filters like Gabor filters for extracting paint strokes.
Video
Hope you enjoyed the project! Here’s the exciting painting video! :)
Comments