Muodov's Honey Jar

Meccano Rubik's Shrine

Fri, 12 Aug 2016 00:00:00 +0000

Update:

We’ve received some feedback from people who’d like to get their own replica of this and other models. That’s very cool! Have a look at the project page to get more details about this project.

References

Project page on Meccano Kinematics website
Solving algorithm implementation: GitHub
Other Wilbert’s machines: YouTube channel
More about Meccano and FAC system: Meccano Kinematics

Video

Although our FAC Solver has gathered 120K+ views on YouTube, we felt that this was only the beginning. By the time we finished it, we had a bunch of ideas in mind that we were eager to realize. While I was trying to figure out more efficient Arduino-Raspberry setup, Wilbert’s tireless genius came up with another 3-gripper machine, this time assembled with old-school (yet well-known) Meccano construction set.

While FAC Solver was more of an experimental creation, this time we had lots of experience, so we could address fundamental weaknesses of the FAC Solver, at the same time focusing on the look and feel of the machine, making it more friendly and appealing. We didn’t want to make yet another hacky DIY machine, but something that would combine technology with design, something nice to build, to see, and to own.

Mechanical Design: Less Wiring, More Art

Having several decades of Meccano experience, Wilbert managed to design the machine that is much more compact than the FAC solver. To achieve the cleanest look possible, all the wiring and unnecessary details are hidden inside the metal body. If necessary, the PCB board and other electronics are easily accessible through the door on the back side. Moreover, the whole machine is assembled using normal Meccano parts that are available on the internet.

The Rubik’s Shrine in transport box

Wilbert was inspired by the Dark Tower from the Lord of The Rings, so the look of the Shrine is meant to be quite pompous :)

Being very careful about the details, Wilbert made sure all parts matched each other: all original rounded shiny Meccano strips, with 3-layered custom airbrush finish on the plates. Brass gears put the final touch to the appearance of the machine. The fine-grained assembly was also very important: the distance between some parts was just a couple of millimeters.

Wilbert tells more about the mechanical part on his website.

Internals: What can we do better?

We were very satisfied with our previous project, the FAC Solver, but we also knew we could do better. We learned a lot about hardware parts like stepper motors and drivers, microcontroller boards, and faced all kinds of unexpected issues on the way. This experience helped us a lot when we were building this Meccano Rubik’s Shrine.

Microcontrollers: Use the right tool for the job

In our previous project, the FAC Solver, (almost) everything was controlled by Raspberry Pi, including solving algorithm, color recognition, movement scheduling, and pulse generation for the stepper motors. We used Arduino only for reading the analog signal from LDRs in scanner device.

While it did work quite well eventually, it turned out to be tricky when it comes to controlling the motors. On the lower level, you can’t precisely control the time that processor gives to your process on RPi. Due to the presence of other processes, and, actually, the operating system itself, your program might be given different chunks of CPU time, which makes the generated pulse train unstable. For lower speeds, it is not a problem, but if the motor speed is high enough, it can be killing for performance: motors start hesitating. To overcome this problem, we decided to use an Arduino board for the actual motor control.

We’ve come a long way with the PCB board. The first fersion looked something like this (using Arduino Uno):

The first version of PCB board

We chose Arduino Pro Mini because it is probably the most compact one. We made the first working prototype with it, and later replaced it with the Teensy 3.2, which looks and feels almost identical, but has better performance. We also used DRV8825 for driving the stepper motors. So eventually, the PCB became this (the red thing on the left is a voltage level trigger needed for the LEDs):

PCB board behind the door on the back side

Arduino gave us much more stable pulse train, which allowed to reach higher speeds. We also used the great AccelStepper library that made movements smoother and more reliable. As before, I used the MIN protocol for RPi-Arduino communication.

Firmware: Optimize everything!

Since software part was my main responsibility in this project, the firmware was the very first thing I cared about. Pure RPi/Python implementation of FAC Solver proved to have a number of nasty caveats, so I wanted to do it right this time. I braced myself and prepared for the challenge of embedded systems programming, ready to implement things from scratch for saving a few bytes of RAM.

Movements

Probably the most difficult part was the movement optimization. The solving algorithm produces a solution sequence on Raspberry Pi, which then needs to be translated into a sequence of gripper movements. Then this data has to be transferred to the Arduino board because actual electrical pulses are generated there. I wanted to make the algorithm generic so that I could use it in other machine configurations (with 4 or even 6 grippers). This added to the complexity of the problem.

Minimizing the overall solving time is not obvious. For example, in a 3-gripper setup like this, to rotate the Front side of the cube we need to rotate the whole cube first. We can rotate it around the vertical or horizontal axis. Whichever is better (leads to faster solution) depends on the current state of the grippers and on subsequent moves.

Synchronization

Generally, all grippers can move simultaneously. To do this, we need to have multiple parallel execution flows. Since Arduino is single-threaded, it has to be done with coroutine-like mechanism. On the other hand, due to the mechanical construction, some movements require the other grippers to be in specific positions, which means that parallel workers need to be able to block each other. Average solving process requires about 200 blocking operations.

Movement scheduling for 4-gripper setup

Memory management

Arduino Pro Mini, which we used initially, has only 2KB of RAM. After initializing objects for serial communication and motor control, there are approximately 200 bytes (!) left for local variables in actual business logic. With such limited resources, I could afford to store only absolutely vital data. Everything else had to be buffered in Raspberry Pi and pushed to Arduino via serial connection during the run. Pure serial communication is quite expensive, so it was causing an additional slowdown of the solving process.

Arduino Pro Mini compilation result

It is (kinda) possible to write C++ code for Arduino, but the dialect is very limited: for the sake of memory it doesn’t have the C++ standard library, exceptions, etc., and in most cases it is a bad idea to use things like dynamic allocation. For spoiled developers like me, it is really tough to refrain from using abstract classes with a bunch of virtual methods and dynamically allocated arrays. It is really hard (I mean freaking hard) to debug because you never see the exceptions: it never fails explicitly, it just starts working weird at some point, and you can only guess what went wrong, especially if you have memory-related problems. I ended up with quite hacky virtual method implementation based on switch statements and static variables, which was lame, but straightforward.

Later, we changed the Arduino Pro Mini with Teensy 3.2 which was a great improvement. Not only it is way faster (thanks to ARM processor), and has 64KB of RAM, but it has exactly the same form-factor as Arduino Pro Mini. We didn’t even need to change the PCB connectors!

Arduino Pro Mini vs Teensy 3

Well, 64K ought to be enough for anybody, at least in Arduino world :) With all this memory available, I could easily store information about the whole movement sequence at once. No additional serial communication + increased speed = faster and smoother solving.

Camera: Come on, it’s the 21st century!

Apart from the motor controlling, the biggest technical change in Rubik’s Shrine, compared to the FAC Solver, was the scanning part. While it was fun to play with the low-end scanner made from LDRs and LEDs, it was quite a challenge to deal with ambient light and do the color balancing. It did fit well the brutal nature of the machine but obviously was not the most modern approach.

Camera with flashlight on

This time, we went fancy and used the Raspberry Pi Camera for color scanning the cube. Since it automatically adapts to the lighting conditions, it solved pretty much all of those color recognition problems (we had to use bright LED flashlight though). All I needed to do was to carefully pick the average colors from specific regions on the photos.

The photo of the cube with marked regions for color recognition

As a bonus, each run is recorded on video and available on the Archive page in the touch interface. It was also quite handy during debugging.

Every run is captured on video

Pi Display: the Final Touch

I just couldn’t resist. Not so long before we started this project, Pi Foundation announced an awesome 7-inch touch screen for Raspberry Pi that perfectly fitted our model. I was pleasantly surprised that it just worked: after a couple of nights I had a nice-looking touch UI written in Kivy.

Having a touch screen, I was able to add some extra functionality to the machine. In addition to a nice presentation of the scanning and solving results, it is possible to make some pretty patterns.

I also made a special debug screen for manual control that practically made it unnecessary to use a laptop for testing.

Public demonstrations and final words

When the Rubik’s Shrine was almost finished, we presented it on SkegEx 2016, the annual Meccano exhibition in Skegness, England. We even managed to win the 5th prize, which is quite something considering that SkegEx is generally more about traditional mechanics rather than robotics and electronics.

But the biggest prize for us were the gleaming eyes of kids visiting the exposition. It was wonderful to see them pulling their parents to our table over and over, just to scramble the cube and start the machine once again.

All in all, we are very satisfied with the result. Apart from all the hands-on experience, for me, as a software developer, it was very fun to work on something physical, something that I could see and touch. And I like to think that, thanks to Wilbert, we could push engineering a little bit towards art. Because this is what it is all about.

3rd-party cookies in practice

Wed, 21 Oct 2015 00:00:00 +0000

Recent days I was building a cross-site communication tool. So I was looking into CORS, window.postMessage and other relevant stuff.

While these mechanisms seem to be well-documented and have simple APIs, handling 3rd-party cookies is a little bit vague. In particular, I was interested in What exactly does it mean when user disables 3rd-party cookies?.

It turned out not so complicated, but you don’t know this until you actually have to use it, so hopefully this will help someone in future.

Questions

First of all, what is a 3rd-party cookie anyway? Common sense tells us this must be cookies in scope of domains other than the current page. But this is quite vague. Does it mean we only cannot set new cookies, should we be able to read the old ones? How should browser handle cookies set from Javascript? Does this rule affect requests triggered by images and other resources? How does it affect CORS requests? What about localStorage and sessionStorage? And how do we detect that user has disabled 3rd-party cookies?

Answers

I’ve done a quick testing trying to answer those questions and here is what I found. Everything was checked on Firefox 41.0.2, Google Chrome 46.0.2490.71, and Safari 9.0 under Mac OS El Capitan.

First of all, when 3rd-party cookies are disabled, browsers do not save, nor send cookies for domains other than the top-level window (current page). This affects all cross-domain requests including resources (e.g. <img> tags), iframes, and XMLHttpRequests (including CORS). This is something that you would expect, actually.
If there is a cookie that was set previously as a 1st-party, it won’t be used.
Javascript in cross-origin iframe cannot set cookies as well. No exceptions will be thrown, but document.cookie will always return an empty string, even if you set it to something.
No cookies in CORS requests too, even if you use .withCredentials parameter. Cookie header from server is just ignored.
3rd-level subdomains and sibling 3rd-level subdomains are not considered 3rd-party: foo.example.com can load an iframe pointing to bar.example.com and it will be allowed to set cookies.

localStorage/sessionStorage is a bit more tricky:

in Google Chrome any attempt to access localStorage will result in DOMException
in Safari, localStorage.getItem() and localStorage itself don’t raise errors, but if you try to use localStorage.setItem(), it will give you a QuotaExceededError
in Firefox, localStorage and sessionStorage are still working inside iframes. So you can save stuff in localStorage, and it will stay there. However, this will change in upcoming Firefox 43.

Safari pitfall

The term “3rd-party cookie” (actually “3rd-party origin”), seems to have different meanings across browsers. For example, Safari by default does send cookies to third-party domains, if it was visited before. There is a very nice story about that by Alex Volkov. Worth reading.

Detection

If you need to detect if 3rd-party cookies are enabled, one option is to take advantage of always-empty document.cookie. Just create an iframe pointing to a page on another domain like this:

<html>
<head>
<script>
var randStr = function(){
  return Math.random().toString(36).substring(7);
};

var thirdPartyCookiesEnabled;
var k = randStr(),
    v = randStr();
document.cookie = k + '=' + v;
if(document.cookie.indexOf(k + '=' + v) == -1) {
  thirdPartyCookiesEnabled = false;
} else {
  thirdPartyCookiesEnabled = true;
}
alert(thirdPartyCookiesEnabled);
</script>
</head>
<body></body>
</html>

Workaround(s)

If you really need to access cookies on another domain, there are some ways to do so.

The most bullet-proof one is just make sure that cookies are 1st-party. Essentially, it means redirects. So you can redirect user to a page on another domain, set the cookie there, and then immediately redirect him back. This is not the best user experience, but it works.

In some cases it is possible to use a subdomain for setting cookies. If you have access to example.com DNS, you can create a CNAME record foo.example.com, pointing to a server that needs to set cookies. Provided that cookies are not host-only (that is, they are set with domain attribute specified), you can share cookies between those domains.

Conclusion

What I have described above, is just a top of the iceberg. It is sufficient to understand the immediate side-effects of blocked cookies, but if you are really going to address this issue, you will most likely come across many caveats. And I believe there is no ideal solution to this problem.

On the other hand, the 3rd-party cookies problem has been there for a while, and now you can find lots of nice articles and posts with all kinds of reusable ideas. Check out this post by Alex Volkov for example.

FAC system Rubik's Cube solver

Tue, 18 Aug 2015 00:00:00 +0000

Update 2:

Make sure you check out the successor of this project - Meccano Rubik’s Shrine, it’s even more awesome!

Update:

Since this post was published, we upgraded the scanner on the machine, and now the scanning process takes way less time. Check out the Hardware section.

TL;DR

For those who don’t feel like reading any further, here are some links:

Project page on Meccano Kinematics website
Solving algorithm implementation: GitHub
Final video: YouTube
Other Wilbert’s machines: YouTube channel
More about Meccano and FAC system: Meccano Kinematics

Introduction

About three months ago I was introduced to Wilbert Swinkels. Normally you don’t meet people with such an amazing blend of skills and passion for creating awesomeness, so this was a great luck for me when he offered to help him with the Rubik’s cube solving machine.

The machine looked absolutely marvelous. Just like all other Wilbert’s creations, it is a piece of art by itself. Besides, color cubes had already carried me away once. With no doubt, I accepted his offer.

By the time I met Wilbert, he had been working on the machine for almost five years. It is quite an interesting story, with a number of great successes, disappointing failures, and invaluable experience. When I started working on it, most of the hardware-related problems had already been solved. There was even a proof-of-concept program for Arduino that could control grippers and scanner. However, the software part essentially did not exist.

Hardware

Main board

First of all, we decided to replace Arduino board with Raspberry Pi. This was for a number of reasons: it seemed that we didn’t really need low-level capabilities of Arduino, and I didn’t feel like writing everything in the limited dialect of C++ used by Arduino. Moreover, sophisticated cube solving algorithms required quite a lot of memory, that Arduino cannot provide.

The standard version of Raspberry Pi did not have enough GPIO pins, so we got a Compute Module Development Kit. That one is really cool: not only it has a way more convenient pinout with 45 controllable pins, but it also has two camera slots. We were so happy with it that we bought two more boards for future projects :)

Capacitor-based scanner

Of course, this change from Arduino to Raspberry required some changes in circuits as well. One of the problems was that you can only get 3.3V from Raspberry’s GPIO pin whereas Arduino gives you 5V. Fortunately, our motor drivers still worked under lower voltage, so we only needed to change resistors in scanner circuit.

Much more annoying problem was that Raspberry didn’t have analog input. This was crucial for us because our color scanner was based on LDRs. On Arduino, it was very easy to read the value from LDR, but to do the same in Raspberry one has to use external devices or make some hacks. After all, we took this approach. Basically, instead of reading the voltage, we are counting the time needed to charge the capacitor. This time is proportional to LDR resistance, so in the end we can estimate its value.

Of course, it was not reliable. Mainly, because there are a lot of background processes running alongside your program on Raspberry, so you always get random deviations in readings. This is not so important if you just need to detect the light with LDR, but can have nasty consequences if you want to compare the amounts of light. To circumvent this effect, we had to do multiple readings every time, detect outliers, and calculate an average between the remaining values.

Here is the video of a machine with capacitor-based scanner:

Arduino-controlled scanner

Though we made the capacitor-based scheme working, it was ridiculously slow. It would take about two minutes to scan the whole cube. So we decided to get a small Arduino board specifically to control the scanner.

It turned out to be very easy to connect Arduino to Raspberry and make them work together. We only needed two wires, this tiny little level converter in between - and voila: we have a serial connection between the boards. I also used cool Min protocol on top of serial to make it more convenient.

This change dramatically decreased the time required for scanning the cube. Since Arduino can read the voltage directly from the LDR, we don’t need to wait until capacitor is charged, and we don’t need to do it multiple times anymore!

Solving algorithm

The most important part of the solving program is a solving algorithm. There is quite some serious mathematical work already done on Rubik’s cube. If you are into algebra and combinatorics, it might be pretty interesting. For instance, I was very surprised that the God number (the exact lower bound for a number of moves needed to solve arbitrary cube) was only found in 2010!

While we did care about the speed of solving, we had to take into account that Raspberry Pi has limited computational powers. Therefore, we could not use the optimal solver as it would require a lot of time for bruteforcing. After all, we ended up with a great two-phase algorithm by Herbert Kociemba.

The original implementation of the two-phase algorithm is only available in Java. First thing I did was translating it to Python. It turned out to be quite easy and straightforward. However, in the beginning I didn’t realize that solving the Rubik’s cube requires a lot of resources. When I first started Python solver, it took about one minute (!) to find a solution on my MacBook.

PyPy showed much better results, it reduced the time to one second on the laptop, but it was still about a minute on Raspberry. After a day trying different ways to speed up Python code, I decided to go for a pure C version. This solved all problems: now it takes about one second to find a solution, even on Raspberry Pi. This was good enough for this project.

I have published both Python and C versions of the solver on GitHub.

Moving the assembly

Next step was to build a program that would actually control the machine. This implied moving the motors, carefully handling gear ratios, keeping in mind the limitations forced by the assembly (for example, side grippers can only be rotated when the bottom gripper is in certain position). Some additional code was required to handle user input (machine has a multi-purpose controlling button), and to optimize gripper movements. All in all, there was nothing special in this part, just a routine software development.

I also made a handy interactive shell for debugging purposes. It made testing very convenient and I could easily debug parts of the program. To debug the scanning, I rendered nice images to visualize the readings.

Scanning and color recognition (a.k.a. devil in details)

Update: This section describes the previous version of the scanner (capacitor-based). New Arduino-controlled scanner made the readings much more reliable and fast. However, color clustering algorithm remained the same.

Up to this point, things were exciting, but not difficult. And we were ready to open champagne to celebrate our great success. The only thing left was to recognize the colors using an LDR-based scanner. The idea was as simple as it could be: we had 3 sets of LEDs (red, green, and blue) which we turned on one by one and captured reflected light with LDRs. Essentially, this gave us RGB values which we could use for color recognition. We already had a proof-of-concept program which seemed to work well on Arduino, so we didn’t expect surprises here.

However, the scanning process turned out to be the most challenging part of the whole project. It took almost 2 months to accomplish this.

Problems

First of all, we had to fundamentally change the circuit of the scanner, as described above.

Due to the hack with capacitors, it introduced random errors in our readings. So the first thing was to do several readings in a row and get rid of outliers. Once we did that, we could get something like following (these are results of scanning a solved cube in a dark room):

As you can see, this is far from ideal. First, results for the same color in different positions significantly differ due to the fact that every LDR and every LED is pointing in a slightly different direction. Second, sometimes different colors can have overlapping value ranges (for example, red and orange often appear the same). Finally, all the readings are highly sensitive to the ambient light.

Due to the random deviation in readings and ambient light, dispersion is quite extensive as seen on the following (clickable) plot (by the way, you should check out plot.ly, it is awesome):

UPD: Arduino-based scanner showed much better results:

Clustering color values

Given these results, I came up with two fundamentally different approaches to color recognition:

1. Color ranging

Find the ranges (a cubical region in RGB color space) for each color and just look for a suitable range every time we scan a facelet. Each combination of facelet position, LDR, and LED color should be handled separately. While this approach is quite straightforward and easy to implement, we have to stick to particular cube colors. And in the end it turned out to be not usable under different ambient light conditions since some ranges overlap and, therefore, become ambiguous.

2. True clustering

Make artificial tweaks on raw values to make the same colors in different positions look similar. If we can do this, we can apply some clustering algorithm to distinguish colors. The most difficult part here was to figure out proper tweaking coefficients that would work well under different circumstances and ambient conditions. Unfortunately, this approach was not usable as well, due to probability nature of clustering algorithms: they are mostly intended to give a “decent” solution, but not the exact one.

Compromise

Each of the approaches above has its pros and cons. So after all we used a sort of a mixed algorithm:

we do some artificial calibrations based on gathered statistics to normalize the readings
convert values from RGB to HSV
find a white cluster, by taking out 9 facelets with the least Saturation
artificially increase saturation for all other facelets
do a simple clustering of remaining (non-white) colors by comparing to center facelets

Ambient light elimination

Even with a decent clustering algorithm, scanning was still tricky due to the ambient light. What seemed to work perfectly in a dark room, failed miserably under a bright sun, and vice versa. So the final step was to get rid of the ambient light. Wilbert had to do quite some work on the scanner module to isolate LDRs from ambient light. It took 3 iterations: each time we thought it was ready, but each time we would find another hole through which ambient light could reach the LDRs:

Summary and promises

This project was fun. It was very exciting to bring it to life step by step, and such a pleasure to see it working after we put so much effort in it. But this is nothing compared to how much we learned thanks to it. What seemed to be pretty easy from the beginning, turned out to be very tricky in details. I could not imagine I would have to learn statistical methods and clustering algorithms, dust off my school notes on electronics (though we only need the basics here), and discover a number of useful tools and services along the way. This is why I am so happy I had a chance to work on this.

That said, this machine was only a prototype. We didn’t contemplate to challenge record-breaking robots in terms of solving speed, and we didn’t really make the most of the hardware parts. But we will definitely go for it in the next generation of this machine.