The Challenges Of A Laundry Folding Robot

“This is the year of the general purpose home robot!” “2016 is going to be for robots like 1976 was for the home computer!” The problem with statements like those is the fact that we’ve been hearing them since the 1970’s. General purpose home robots still have a long way to go. Sure, we’ve got Roomba, we’ve even got self-driving cars. But we don’t have Rosie from the Jetsons. And while I don’t think we’re going to get to Rosie for a while, there are some simple challenges that can spur development in that direction. One need look no further than one’s own laundry room.

Using machines to wash and dry laundry isn’t a new concept. Washers and dryers have become commonplace enough that we don’t think of them as robots. Hamilton Smith patented the rotary washing machine in 1858. Maytag has had home machines available for nearly 100 years. Many of the early machines were powered by gasoline engines, as electricity wasn’t common in rural farmhouses. Things have improved quite a bit since then! From the dryer we transfer our laundry to a basket, where it has to be folded. It is this final step that cries out for a homemaking automaton to take this chore out of Everyman’s hands.

As one can imagine, folding laundry is one of those tasks that is easy for humans, but hard for robots. However, it’s not impossible. The idea of this article is to show what has been done, and get people talking. A project like this would take a person or group of people with skills in mechanics, electronics, machine vision, and software. It would also be sure to place well in the 2016 Hackaday Prize.

Commercial folding machines exist. Hotels, hospitals, and other industrial environments have machines that handle high volumes of sheets, towels, and other pieces of laundry. These giant machines iron while they fold. Even with modern process control technology, they still require human operators to hand-load each item. On the hacker side, the Foldimate 5000 is a novel approach to mechanical folding, though it can’t fold everything.

Seven Dreamers Laboratories from Japan has been advertising their Laundroid robot at trade shows, including CES. However, every demonstration I’ve seen to date consists of an unfolded piece of laundry being put in the machine. The doors close, and a few minutes later the article of clothing magically appears folded. Until I see proof that there isn’t a person back there folding the laundry, I’m going to call shenanigans on this one.

The most promising work on laundry folding robots came from [Pieter Abbeel and Jeremy Maitin-Shepard] at Berkeley. Way back in 2010, the pair and their team used a $400,000 USD PR2 robot from Willow Garage. While the results were promising, they were operating on a limited set of laundry – towels, and towels only. Their final research paper details their results. (PDF warning)

A basket of clean laundry is an unstructured environment. Whites, colors, and prints all piled together. Pattern recognition is right out the window. Garments aren’t necessarily a solid color either. What is a robot to do?

One method would be to grab something – anything out of the basket, shake it out, and go from there. This is a simplified view of how the Berkeley system worked. Grasp, re-grasp, and attempt to determine the corners of the piece of laundry. Granted, the Berkeley robot only had to deal with towels. It also had the help of a green screen to view what it was working on. Real world laundry is a lot more than towels. Some articles of clothing – T-shirts for example, don’t have well-defined 90-degree angles at every corner. T-shirts can also end up inside out, a major problem for a laundry folding robot.

Berkeley had a very expensive PR2 robot doing their folding. Since most readers here on Hackaday don’t have that kind of budget, we’ll have to find a cheaper way. Thankfully there are several robot arm projects over on Hackaday.io which are up to the task.

Smaller, weaker robots aren’t out of the running either. Humans come in all shapes and sizes – and so does their laundry. Have a small robot arm? Start with baby clothes. Any new parent you know would be overjoyed to have help with the laundry – though you may have to promise to refold anything your robot fails to complete.

A robot which can fold laundry will need a lot of software. Thankfully there are some open source libraries to build upon. Input will come from cameras. That means we need a serious machine vision package to process the images. This already exists in OpenCV. OpenCV is an open source (BSD license) computer vision package which has been around since 1999. It has been continuously improved since then. Ports exist for Mac, Windows, Linux, as well as Android and iOS. It also runs great on the Raspberry Pi.

Output will come in the form of arm movements. In robotics terms, deciding where you want a robot arm to move is motion planning. Actually calculating how much to move each joint, and when to move them in order to get there is Inverse kinematics. Reading about motion planning and inverse kinematics is a great way to lose an afternoon – or seven years in graduate school. There’s plenty of research out there. A good portion of it lies behind paywalls though, so if you’re not at a major university, you’re out of luck. All is not lost though. You can easily find free software to help with motion planning and inverse kinematic equations. A great example is openRave. Another option is to dig through the ROS (Robot Operating System) documentation pages, ROS is the Linux based system which forms the brain of the PR2 robot.

With input and output out of the way, all that’s left is control. This is the software that will handle the data coming out of the OpenCV and decide where to move the arms. Folding laundry has a huge number of variables. Trying to come up with a simple algorithm that covers every possible piece of laundry would be impossible (though I would love to be proven wrong on that). The best bet for teaching a robot to fold laundry is to do just that. I’m talking of course about machine learning, chiefly with neural networks. There is an incredible amount of work being done on machine learning right now. One of the giants is Google, who just recently released TensorFlow to the open source world. TensorFlow is a package to try out machine learning algorithms. It’s also used in real world environments like Google Voice Search and Google Images.

So there you have it. A quick lowdown of a very challenging problem, and a few tools to get started. These are just my ideas though. What would you do differently? Join the discussion down in the comments, then head over to Hackaday.io and get your laundry folding robot project started!