The Map/Reduce script type is often the most efficient way to handle large amounts of data in NetSuite. This article seeks to introduce readers to this powerful and complicated script type.
Introduction to Map/Reduce
Before diving into the technical aspects of the Map/Reduce script type, an illustration may be helpful to grasp the overarching idea.
Imagine with me that you decide to go shopping at the mall one day. Once inside, you notice a giant funnel near the door. A man walks up to it, reaches into his backpack, and pulls out a piggy bank full of coins. He proceeds to smash the piggy bank open on the rim of the funnel like an egg. The coins go flying. But within a moment, each coin is in a mesmerizing spiral trajectory, slowly winding down towards the center. As the coins near the center, they almost seem to merge together into one—until finally, they all fall into the lower regions of the funnel where they are sorted by coin type. Once they are separated by coin, a charity worker can gather and handle them. And perhaps a report will be made of how many coins were donated by such generous people!
Although the illustration certainly isn’t perfect, it does help shed some light on the nature of Map/Reduce.
Map/Reduce is a way to handle large amounts of data. The greatest benefit of Map/Reduce (as compared to a Mass Update or Scheduled Script) is its asynchronous nature. Multiple pieces of data (or coins, per the illustration) can be processed simultaneously. It is a kind of organized chaos, and it has potential to be much more efficient than the traditional serial method.
5 Map/Reduce Stages
The Map/Reduce script contains five stages. Unlike other script types, Map/Reduce involves the NetSuite system far more, making it reliant partly on the script and partly on the system.
Get-Input-Data
The first stage of M/R scripts is the Get-Input-Data stage. This is where the man grabbed the piggy bank and unleashed the coins. In scripting terms, this may simply be a Saved Search to collect the initial data. As you might imagine, M/R scripts require this function since it is the most important of the stages.
Map
The second stage is the Map stage. This corresponds to the coins entering a circular trajectory. Though scattered, the coins are still in the system. If someone wanted to, they could count the coins as they spun or even pick them up out of the spiral. In other words, this is one stage which possesses the potential to process the data in some way. It is not yet “reduced” however, which may be a pro or a con depending on your situation. The Map function represents the data as it is, without filtering or categorizing. Nevertheless, there may still be benefit to simply using stages 1 and 2 in place of a Mass Update or Scheduled script, as it is asynchronous.
Shuffle
The third stage (what NetSuite calls, “Shuffle“) is actually not something we include in our scripts. It comes in between the Map and the Reduce stages. As a result, the coins categorize by type once they enter inside the funnel. We’ll explain more of the technical details in the next part to this series about Map/Reduce scripts.
Reduce
The fourth stage is Reduce. This is our other best opportunity to handle the data. Here, we can perform calculations, update records, send emails, and do other helpful automations based on the data. Interestingly enough, the Reduce stage is also asynchronous (and also where my illustration begins to break down!). For each asynchronously sorted coin (i.e., data) that comes in, we can perform a task.
Summarize
The fifth and final stage is Summarize. This is a where a report can be made for all the processed data. If a coin came from another country, perhaps that could be noted. The number of successful coins or statistics about which coins were most common also might be helpful. In other words, important data from the Map/Reduce can be summarized. This ultimately helps us see how successful the automation was.
Conclusion
Here, we introduced the Map/Reduce script type. Because of the complexity of this script type, I have decided to divide this topic into multiple sections. In our next post, we will dive into the technical aspects of Map/Reduce scripts and provide some helpful code examples.
We hope you find this helpful! If you have any questions or comments, don’t hesitate to let us know in the comment section. And remember to subscribe to our SuiteScript email list to stay current with our latest blogs!