A robot dog walking on a pile of plum blossom .. self-balancing bikes

On November 20, Tencent of China announced its new progress in mobile robotics research: the Jamoca four-legged portable robot and the self-balancing wheeled mobile robot debuted.

Jamoca is the first four-legged robot in China.

It can complete the complex challenge of walking on plum blossom piles, and the relevant research results of portable self-balancing wheeled robots were selected as the best IROS 2020 conference in the robotics industry.

These two developments come from Tencent Robotics X Lab, which are technological breakthroughs in the direction of movement and are also one of the most fundamental capabilities of robots.

Established in 2018, the Tencent Robotics X Lab is dedicated to advancing the search for next-generation robots through human-machine collaboration, establishing telecom companies and connectors from the virtual world to the real world.

At present, the laboratory focuses on the research and application of three general robot technologies, namely, portable devices, skilled operation and intelligent body, and it focuses on the research and development of multimedia mobile robots.

Jamoca mobile four-legged robot: can walk, run, jump and challenge the complex terrain of plum blossom mounds

“He studied Shaolin for dozens of winters and practiced real skills on plum blossom piles. If you keep practicing, you will definitely become famous and build the best.”

The plum blossom pile is one of the important training tools of Chinese kung fu, and the practice of the plum blossom pile is also the basis for light kung fu practice in martial arts novels.

Now, a new “practitioner” has been inserted – a Jamoca mobile four-legged robot on the Plum Blossom pile.


Jamoca body has been modified based on an external hardware platform, it weighs about 70 kg, 1 m long, 0.5 m wide and 0.75 m high.

Four-legged portable robot, its most common name is dog robot.

Based on externally presented ontology and based on self-developed robot control technology, the Tencent Robotics X Lab has created an intelligent brain for Jamoca that can handle complex environments, this brain allows Jamoca to walk, trott and jump, giving him the ability to center and avoid obstacles.

The challenge Jamoca completed this time consisted of two parts: a step 60 cm high and a flat angle 20 ° and a plum pile with a maximum height difference of 16 cm between adjacent pedestals.

According to a researcher from Tencent Robotics X Lab, compared to other four-legged robots in the world, Jamoca’s plum pile has a smaller landing area, higher elevation, and range of steps. It is more difficult to achieve.

Among them, there are two main difficulties in testing a robot’s ability to move:

  • Understand the arrangement of the plum blossom piles (including position and height).
  • Choosing foot points and paths, walking steadily and accurately (feet to the center of the plum blossom pile).
The plum blossom pile
The plum blossom pile

This corresponds to the two core technology modules the lab studied in mobile technology:
Perception, movement planning and control; Meanwhile, the two achieved systematization through another core technical unit of the laboratory: the design and construction of the whole machine system of integrated integration.

Building on the results of the Jamoca Challenge, the Tencent Robotics X Lab has reached very high technical standards:

  • Identify perceptual sites within 1 cm.
  • And trajectory mapping at 10 ms level depending on the environment.
  • Real-time dynamic torque control 1kHz.
  • The plum blossom is footsteps within 1 cm of the center of the substrate and has a high degree of coordination of the entire system.

In terms of accurate environmental perception, Jamoca innovatively achieves powerful eye and foot calibration and uses RGB-D cameras to perceive the surrounding environment in real time.

Through feature point matching, Jamuka’s motion path is tracked online, and vision-based positioning information is combined with kinematic-based odometer information and IMU (Inertial Measurement Unit) data to improve positioning accuracy and frequency of tracking.

At the same time, algorithms are used to define and extract the surface area range and position of the focal point of the steps and the plum blossom mounds, and the positioning and recognition data are combined to completely reconstruct the 3D motion scene.


In terms of optimal motion path mapping and real-time motion control, Jamoca realizes online midpoint path generation and foothold mapping based on position information of the robot body and plum blossom stacks that are sensed in real time by the sensor system, and based on midpoint dynamics.

While ensuring that the four legs of the robot can safely tread on the next plum blossom pile, it improves the movement path with the shortest body movement length and the least overall effort, and the above mentioned motion planning can be done continuously online.

Meanwhile, based on real-time body condition estimation, Jamoca can combine the mass dynamics center model to build optimization problems, solve the ground reaction force required by the robotic foot in real time, and combine feedback control to achieve precise and robust control of Real-time, which can complete walking, motion control for trot and slash jump.

Unlike industrial robots that perform repetitive movements after pre-designed rules, the Robotics X laboratory pays more attention to studying the autonomous properties of robots with awareness and judgment, the purpose of which is to perceive robots in a dynamic environment with a great deal of uncertainty. Autonomy, independent decision-making and independent task accomplishment.


At this point Jamoca is mainly used for in-vitro scientific research experiments.

Online environment awareness, optimized movement planning and real-time motion control capabilities will help other Tencent robot products to better adapt to the complex real-world environment in the future.

The plum blossom pile1

Wheeled robot that can independently maintain balance during rest and movement

A self-balancing wheeled moving robot, also called a self-balancing bike, was also unveiled for the first time, and is the first self-developed robot in the Tencent Robotics X Lab.

On the basis of traditional mobile wheeled robots, it adds momentum wheel and drive system

self-balancing bikes
self-balancing bikes

The hardware platform for this fully autonomous wheeled mobile robot was developed by Tencent Robotics X.

It weighs about 15 kg and is 1.15 meters long and 0.52 meters high.

Its controllability can be further divided into dynamic self-balancing on movement and static self-balancing on stop.

Dynamic self-balancing during travel is achieved by rotating the front handle and driving the car body rotation, while static self-balancing is stopped when traveling with momentum wheel torque balancing technology and the principle of preserving angular momentum to achieve self-balancing control.

Relying on the balancing algorithm, Tencent’s self-balancing wheeled mobile robot can still advance steadily while maintaining its self-balancing state even if it is subject to certain external disturbances.

self-balancing bikes1

In various road conditions, such as when facing slopes up and down, the robot also shows good exercise ability and balanced performance. Even when there are restrictions

self-balancing bikes2

In the first paper titled “Nonlinear Balance Control of Portable Self-Balancing Wheel Robots: Design and Experiment,” the Tencent Robotics X Lab and New York University collaborated on the stability of dynamic and static self-balancing control of mobile wheeled robots. .

On the basis of conventional mobile wheeled robots, Tencent Robotics X Lab has added momentum wheels and drive motor systems, using the passive control method (bonding-based, damping and damping control, IDA-PBC) to configure damping and damping. And he applied Lyapunov’s theory to theoretically prove the stability of the closed-loop system of a self-balancing wheeled moving robot.

In addition, the laboratory also demonstrated that the new control unit performs well on its self-built platform, which allows the robot to maintain self-balance and has a certain ability to resist external interference.

In another study, “Planning Controller Design for Self Balancing of Wheeled Mobile Robots”, which differs from the traditional method for characterizing static and dynamic balance, the research team concluded that the dynamic and static characteristics of wheeled mobile robots can be described at the same time. A unified mathematical model for the properties.

Based on the same model, dynamic balance and static balance depend on different driving inputs to different degrees, and the same control parameters cannot be used.
Therefore, the research team uses a gain-scheduling control method. The control method can maintain the balance of a wheeled robot in both cases, which has been proven in theory and experience.

The Tencent Robotics X Lab focuses on researching and applying three basic general-purpose technologies: robot mobility, dexterous operation and intelligent body.

Among them, movement is one of the most fundamental capabilities of robots, and the two developments released this time belong to the direction of movement.

Movement direction can be divided into four technical units: mechanical design, perception, motion planning and control, and overall system design and construction. In layman’s terms, the first three are the robot’s body, eyes, and brain, and the last three is the ability to coordinate its “organs”.

Jamoca, a self-developed algorithm, combines results from Tencent’s research in perception, motion mapping, and control. The self-balancing wheeled moving robot developed by the whole machine can be considered as a landmark project of Tencent in the mechanical design of the robot and the design and construction of the whole machine system.

Zhang Zhengyou, Director of Tencent Robotics X and Tencent AI Lab said: “The ability to move or exercise is one of the basic and fundamental capabilities of a robot. It defines the sights it can go in, what to do, and what will happen in the future. Imagination. We are very happy to see. These two developments will continue to explore the global capabilities of robots in depth to build a strong bridge from the virtual world to the real world. “

Related Stories

Stay on op - Ge the daily news in your inbox

Recent Articles