About Cromai
Cromai is a Brazilian AgTech founded in 2017 with a mission to lead the next digital revolution in agriculture, taking Brazilian producers to a new level of efficiency, production and sustainability. In order to assist workers in the agriculture industry, Cromai utilises Machine Learning and computer vision to automatically identify patterns in images collected from the field, generating diagnostic information. Access to this data allows for more accurate decision-making and precision farming, enabling agricultural producers to reach their maximum productive potential, through the use of AI, in a simple and sustainable way.
Cromai has two primary solutions on the market, both with AI and computer vision at their core. The first solution provides farmers with the geographical location of weeds growing in sugarcane plantations, enabling isolated management. The second solution increases quality control for sugarcane crops through instant alerts upon identification of impurities. These solutions increase the efficiency of agricultural producers’ work, with a reduced demand for physical labour as well as resulting in a significant reduction in the use of herbicides and pesticides.
Cromai was developed initially to assist Agricultural producers throughout Brazil, however, in recent years the AgTech has gained international recognition, and was selected by StartUs Insights as one of the world’s top 5 most promising computer vision startups impacting agriculture.
The Business Challenge
DNX Solutions was tasked with the main challenge of optimising Cromai’s Machine Learning training time for the Deep Learning models. For computer vision to produce efficient and meaningful results, a huge amount of data analysis is required; in Cromai’s weed identification solution, for example, the dataset contained over 20 million images. Whilst Deep Learning algorithms allow for Machine Learning to occur in the absence of a programmer, reliance on inefficient technology can make it a time-consuming process. In Cromai’s case, the use of a server with a single GPU aimed at training Deep Learning models took approximately 3 months per model, a delay which directly impacted the company’s core business. It was clear to DNX Solutions that more robust cluster training was crucial to Cromai’s success, and the cloud was the answer.
Benefiting from significant time reduction by training neural networks using multiple GPUs in parallel
Once the DNX team assessed Cromai’s business and technical needs, the team built a comprehensive business case and provided it to the Cromai team. The business case presented a number of potential strategies in order to achieve the desired outcome of the project, which was a reduction in training time, without significantly affecting the cost or performance metrics of the model. By bringing Cromai’s Machine Learning to AWS cloud, the team could take advantage of two main factors to achieve the project goals. Firstly, AWS allows for the use of extremely powerful training instances, equipped with several modern GPUs per instance. This change benefited Cromai simply in terms of sheer performance. Secondly, AWS makes it possible to distribute the training to more than one instance. This task is far from trivial since the training of neural networks, even when distributed, requires synchronisation between its instances and GPUs to be maintained. To perform these tasks, the DNX team selected the perfect framework: Amazon SageMaker distributed.
In addition, the DNX team chose to use Horovod, an open-source distributed training framework for deep learning algorithms. Horovod can be used in conjunction with SageMaker and is highly regarded for improving speed, scale and resource allocation during training.
From here the main task was to adapt the Cromai training script to the Amazon SageMaker environment. To do this, DNX used S3 as a training data store and added the Horovod layer into the script. In addition, the team created an easy and cost-transparent way for Cromai to choose the quantity and type of instances for each training
Training time reduced from 3 months to 6 days
The delay in training models was directly affecting the success of Cromai’s core business, making a decrease in training time critical to the scale of projects they were able to undertake. As a result of the strategy employed by DNX Solutions and the use of AWS tools, this has become an issue of the past from Cromai.
The solution delivered drastically reduced training time from 3 months to just 6 days, whilst maintaining all existing performance metrics. Furthermore, Cromai has the option to increase their training investments on a needs basis, allowing them to obtain results in as little as 3 days.
This huge decrease in time led to more frequent interaction, increased agility, and the flexibility for Cromai’s technology team to focus their time and energy on what they love: developing improved solutions that reflect the reality of rural producers’ needs.
