The Three Barriers to Implementing Robots

Faster and in greater quantities — industrial robots were once introduced as a means to simply enhance productivity. However, with labor shortages becoming a critical issue in many developed countries, robots are now being implemented out of necessity, to compensate for the shortfall of human workers. And yet, with the diversification of consumer demands, manufacturing trends are shifting toward multi-product, variable-volume production. Industry is being pushed to adopt flexible production systems that can manufacture different products on a single production line, and robots that can only perform simple tasks are no longer effective enough. As a result, automation through the implementation of robots is not as easy as it used to be.

Nevertheless, the implementation of high-performance, high-functionality robots comes at a high cost. On top of that, the complexity of the operations and settings also significantly increases. Performance, cost, and usability — only the robots that can overcome these three barriers will be able to keep pace with the rapid changes occurring across the manufacturing landscape.

Aiming to become a key technology solutions company in a global society where humans and machines co-create seamlessly, in line with its Vision 2030, Nikon has embarked on a challenging project to provide new solutions to meet the demands of such times.

Unprecedented High-Precision "Eyes"

The aim is to overcome the three barriers and usher in an era where robots can thrive in various manufacturing fields. The "robot vision system" is an approach the project team is taking to realize this greater vision.

In general, "robot vision" refers to the vision systems installed on industrial robots. Until now, many robots have only possessed a "body" that moves according to pre-programmed instructions. They cannot react to unexpected situations, meaning that they cannot effectively substitute for humans. The first requirement to overcome this is "vision" that can recognize objects and situations with high precision.

In addition, they also need a "brain" that processes the information obtained via vision. Robot vision consists of a "sensor (camera)" that correspond to the eyes and an "engine (control PC)" that acts as the brain, allowing the robot to determine how to move its body based on the information gathered by the sensor. When this is implemented in a robot's body, it enables the flexible operation required for multi-product, variable-volume production.

By leveraging the core technologies it has developed over many years, Nikon has gained a significant advantage in the market as manufacturers strive to improve performance.

Upgrading Robots with Proprietary Technologies

The robot vision project team developed the system with three key features leveraged from Nikon's proprietary technologies, setting it apart from other manufacturers.

The first is its "unmatched speed." Working with moving objects is incredibly difficult, and automation requires a robot to be perfectly in sync with the conveyor belt. As robots move according to how they are programmed, most systems can't adapt to changes in the conveyor belt's speed or object positioning, which is why many tasks still rely on manual labor. Nikon's robot vision, however, captures up to 250 frames per second to detect objects, which is far faster than the human eye. This real-time recognition of objects' movements and location allows the robot to adjust flexibly without needing complex programming.

The second feature is its "superior recognition." While many robot vision systems rely solely on 3D cameras as their "eyes" for recognizing objects in three dimensions, Nikon's project team combined a 3D camera with a 2D camera for high-speed recognition, in a hybrid approach. This superior recognition capability enhances both recognition accuracy and speed of operation, cutting response time to just two seconds^*1 between detection and movement. Additionally, conventional robot vision systems primarily used a "fixed overhead" approach. Cameras were mounted on a tower to capture objects from above and send movement commands to the robot. However, because each camera's field of view was fixed, if the object moved out of view, the system was unable to track it anymore, leading to many missed objects. Nikon's "hand vision" design mounts the "eye" directly on the "hand" and unifies them into a single system; it's as if a person has eyes on the back of their hands. Unlike fixed overhead cameras, the robot arm can move and view objects from different angles, even when they previously would have shifted out of sight.

Thirdly is its "ease of use." Conventional robot vision systems are often complex and require specialized knowledge to operate the software, making implementation of these robots an obstacle for many users. To address this, the project team developed a simple, intuitive interface with automated settings. Users can easily configure where the robot should grasp an object, and the system automatically calculates a path for the robot arm to avoid collisions. The resulting improved user experience allows basic setups to be completed in about 20 minutes. Unlike the fixed overhead systems that require complex and troublesome installations, the hand vision system can be installed by just mounting it on the robot's arm, allowing users to utilize existing equipment. This significantly reduces both the time and cost of implementation.

These beneficial features are particularly evident in two applications. With "vision tracking," the high-speed performance allows the robot and conveyor to continue working without stopping, even when the position or speed of objects on a conveyor changes. And with "bin picking," the system's high recognition accuracy enables the robot to precisely pick up objects randomly piled in a box, helping to reduce overall cycle time^*2.

The development of robot vision was not without its challenges, but Nikon's core technologies helped to successfully overcome them. In factories with heavy vibrations, a robot's arm can experience slight shaking. Here, Nikon's advanced precision technologies, honed through its semiconductor lithography systems, help correct these small movements. Naturally, Nikon's opto-electronic technologies, refined in its imaging equipment, also play a crucial role within the "eyes" of the system.

But it wasn't just about technology. Understanding the realities of the production floor was equally important in developing robot vision. The project team's goal wasn't simply to create something they were satisfied with — it had to meet the real needs of the front line. Team members regularly visited production sites, gathered real-world feedback, and incorporated it into prototypes. The process of actually having test users utilize the system was repeated numerous times to effectively overcome the barriers of performance, cost, and usability.

• *1In a test conducted within Nikon’s environment
• *2The average time it takes to finish one unit of a product.

The Future that Robot Vision Envisions

Amid great expectations among manufacturing industries, Nikon's robot vision has officially launched. In the short term, it is expected to play a vital role in the assembly of automobiles, devices, and smartphones. Eventually, it is likely to lead robots into fields that are beyond our current imagination.

Through robot vision, Nikon aims not only to alleviate labor shortages, but also to create a society in which people can express their creativity.

Next Generation Project Division
Kenta Wakimoto

I originally worked in robot manufacturing and was invited by Nikon to contribute to the development of robot vision. Combining Nikon's century of expertise with my knowledge of robotics has opened new possibilities for expanding the capabilities of robots.

Robot vision plays a crucial role in supporting tasks that robots usually find challenging. As the demand for higher performance and functionality in robots increases, so does the importance of robot vision. I aim to keep pace with changing needs and continuously deliver new value. Moving on, I plan to focus not only on improving robot vision, but also robot intelligence and the human-machine interface, and in so doing contribute to the evolution of robotics.

Next Generation Project Division
Taiga Tsunoda

As a salesperson, I have a lot of interaction with customers, and lately I've noticed that they've been placing a strong emphasis on "ease of use." Most conventional products take about two hours to set up, whereas Nikon's robot vision can be set up in about 20 minutes. I've only recently joined Nikon and had no previous experience with robots, but I find the system easy to understand and use. The faster setup time also offers the advantage of being able to use the time saved for other tasks. I want to share my personal experience with customers in a relatable way and help spread the word about this product. I hope that robot vision will become a sought-after solution in a wide range of fields.