Keeping an eye on cow calving 24 hours a day

Livestock farmers' duties range from raising the cattle and cleaning cowsheds to shipping raw milk and transporting beef cattle. Since they are dealing with live animals, it is impossible to finish or stop working for reasons of human convenience. In particular, cow calving is something that you cannot take your eyes off at all, for 24 hours a day.

Calving always involves such risks as stillbirth. For beef farmers, whose raising and selling of cows is a major source of income, it is extremely important to ensure that calves are born safely. Furthermore, because each cow can only produce milk for a certain period of time after giving birth, calving is an essential part of making a living even for dairy farmers, who sell the raw milk.

Therefore, as the calving season approaches, livestock farmers need to prepare for calving without knowing exactly when it will start around the clock. Particularly in small- to medium-sized farms run by families, members of the family have to take turns monitoring the livestock all night to prepare for any sudden calving.

Recognizing the signs of labor by utilizing video and AI analysis

In order to improve work styles within the livestock industry, Nikon took the initiative of developing a system for highly accurate prediction of calving. Combining the image analysis technology and AI technology that Nikon has been cultivating for a long time, Nikon has developed the live monitoring system "NiLIMo". This provides a solution that can monitor cattle 24 hours a day and 365 days a year on behalf of livestock farmers, in order to notify them of calving.

The basic components of NiLIMo are two cameras and an AI calculation system. Usually, the cattle that are about to give birth are moved to a cow pen one month before the scheduled calving date. Inside the pen, two installed cameras monitor the animals and the data can be transferred to a recorder in real time. AI analyzes the data and predicts the delivery.

Cows perform unique behavior about 4 to 5 hours before calving — the amount of activity increases and the number of times they stand up and lie down also increases. Three hours prior to calving, they start to lift up their tails, a behavior called "tail raising". Then, when birth is imminent, the amniotic sac appears, followed by the calf's legs. If this series of processes could be detected with high precision as early as possible, there would be no need for livestock farmers to monitor the cows day and night, leading to a welcome reduction in labor burden.

Despite such advances, it is not enough to just obtain data with quantity and quality. To be able to accurately detect calving behavior, the AI must be thoroughly trained to correctly recognize which behaviors indicate signs of imminent delivery. Of course, this was the first time for Nikon's development team to perform AI analysis of cow calving. It was a challenging start from scratch.

The development team first obtained the cooperation of livestock farmers in Kumamoto prefecture for test shooting, visited the sites many times, had them send recorded data, deepened their understanding of calving and progressed AI learning. Alongside all the cutting-edge technology, these steadfast efforts came to fruition as NiLlMo.

NiLlMo made it possible to detect the range of processes shown in yellow in the figure below with high precision, in real time. This is a significant achievement since conventional monitoring systems can only detect two or at most three of these processes. So, how has NiLlMo changed the way livestock farmers work?

A stress-free environment for both people and cattle

When a calving is detected through the AI analysis of NiLlMo, a notification is sent to a smart device app. Until now, farmers and their families or staff couldn't risk taking their eyes off their cows around the clock during the calving season, but now they can rest easy until they receive a smart device notification. The notification can also be delivered in audio via a message such as "It looks like the amniotic sac is appearing". So, if farmers keep their smart device nearby while sleeping, they won't have to worry about missing the timing. All that's needed is to check the analysis results on the app, then go to the cow pen when calving is about to begin.

The benefits of NiLlMo will not only be felt by the people involved, but also by the cows. Some of the monitoring systems currently available in the market have devices that require attachment of sensors inside and outside of the cow's body. If the device needs to be placed on the tail, the blood may become congested and the tail may rot. Also, this equipment easily becomes soiled, so takes time to clean, and there is always a risk of being kicked by a reluctant cow when attaching or removing such devices. Eliminating these drawbacks, NiLlMo doesn't require any device to be attached. Detection and analysis start simply by bringing the cow into the pen in front of the installed cameras, which measure and record the changes in the cow's movements, so there is absolutely no stress on the animal itself.

Currently, Nikon is working on further expanding the functions of NiLlMo. Currently, the number of cows that can be monitored per pen is limited to one or two animals. But we are planning to introduce this system to livestock farms with a scale of 50 to 100 cows. So, if we can increase the number of cattle able to be handled, it opens the door to introducing the system to large-scale farms. Additionally, if we can monitor not only the calving season, but also a wider range of processes from the estrus period to postnatal, it will be possible to predict the timing of fertilization and manage the health of calves during their growing process. Furthermore, Nikon is also considering its application for other animals, such as horses. As the introduction of NiLlMo progresses, the work styles of the livestock industry will change significantly.

Studying the ecology and behavior of cattle from scratch to solve problems in the livestock industry

When developing NiLlMo, we took videos of cows with the cooperation of farmers, and we trained the AI to learn the signs of calving over the course of about a year and a half while comparing a huge amount of data. The most difficult part was the finetuning of "detecting what should be detected and eliminating everything else". At first, there were errors such as the glimmering of the ground caused by the reflection of the sun being mistaken for the amniotic sac. Also, there was the issue of how to deal with individual differences among cows, such as different appearances in the color of calves' legs. In order to deal with all kinds of patterns, we watched each video carefully and studied cow ecology and behavior from scratch. Through trial and error, we worked out why identification errors happened and how to analyze them correctly, then applied this to AI.

In the videos provided by farmers, there were stillbirths that could have been prevented if NiLiMo had been used. There were many difficulties during the development process, but these were the moments when I felt a strong sense of mission. The other day, I spoke to a farmer who was considering introducing the system, and he told me that the owner and another person stay up all night, taking turns checking on the cow pens. If one person gets sick, the other has to be there all night. I became acutely aware of the reality that such livestock farms are highly dependent on human labor. Farmers who have installed our system have remarked, "Now, I can sleep at night". I would be very happy if NiLlMo becomes a new means of solving such problems in the livestock industry.