FabricFlowNet: Bimanual Cloth Manipulation with a Flow-based Policy

Cloth manipulation is a complicated job for robotic manipulation as materials do not change rigidly when manipulated.

A current paper introduces FabricFlowNet, a aim-conditioned coverage for bimanual cloth manipulation that employs optical circulation to enhance coverage functionality.

Image credit score: Pxfuel, Free of charge licence

An optical circulation-kind network is made use of to estimate the romance between the recent observation and a sub-aim. The technique is realized with supervised studying, relying on random steps devoid of any professional demonstrations. The realized coverage can execute bimanual manipulation and switches quickly between twin and single-arm steps, based on what is most suited for the sought after aim.

Experiments on a twin-arm robotic technique and in simulation exhibit that FabricFlowNet outperforms condition-of-the-artwork model-based and model-free of charge baselines. It also generalizes with no more teaching to other cloth designs and hues.

We tackle the difficulty of aim-directed cloth manipulation, a complicated job because of to the deformability of cloth. Our perception is that optical circulation, a system typically made use of for movement estimation in online video, can also deliver an helpful representation for corresponding cloth poses across observation and aim pictures. We introduce FabricFlowNet (FFN), a cloth manipulation coverage that leverages circulation as equally an enter and as an action representation to enhance functionality. FabricFlowNet also elegantly switches between bimanual and single-arm steps based on the sought after aim. We exhibit that FabricFlowNet appreciably outperforms condition-of-the-artwork model-free of charge and model-based cloth manipulation policies that choose picture enter. We also current serious-globe experiments on a bimanual technique, demonstrating helpful sim-to-serious transfer. Ultimately, we exhibit that our technique generalizes when qualified on a single sq. cloth to other cloth designs, this sort of as T-shirts and rectangular cloths. Video clip and other supplementary supplies are readily available at: this https URL.

Research paper: Weng, T., Bajracharya, S., Wang, Y., Agrawal, K., and Held, D., “FabricFlowNet: Bimanual Cloth Manipulation with a Movement-based Policy”, 2021. Link: https://arxiv.org/stomach muscles/2111.05623