Taking a look at advancements in New Product Forecasting techniques

Taking a look at advancements in New Product Forecasting techniques

A recent progression in new product forecasting (NPF) is a shift from model-based approaches to data-driven analytical approaches that use data to create new models, rather than applying known models to available data. Together with automated data collection, these data-driven analytical approaches achieve greater accuracy in predicting market trends and customer demand.

Structured analogies

For example, an article by Charles Chase, a well-known author in the forecasting domain, highlighted the use of structured data in analogy forecasts. The method adds structure to data used in analogy forecasts (like products), where data mining techniques are applied effectively to identify candidate products that have similar attributes and characteristics. These candidate products are filtered to yield a surrogate series that has the most similar statistical properties to the unknown new product series. Since each process step requires both statistical and judgmental analysis supported by technology, a formal approach to combining statistics and domain expertise (i.e., judgment) can be applied to significantly enhance the structured-judgment procedure. This method tends to be more accurate and helps reduce the time needed to create a new product forecast.

Automated structured judgement and sentiment analysis

A further application is an automated structured judgment procedure using machine-learning algorithms. In this procedure, the analysis of the internet search pattern data can be used to explain a new product diffusion curve; and internet search volume helps to predict new product diffusion. Also, judgement is improved by integrating structured and unstructured data into a sentiment analysis.

In sentiment analysis, text mining techniques collect unstructured data from social media to examine what people are saying about the new product in real time. Sentiments are extracted and with the use of statistical modelling and rule-based natural language processing techniques, patterns of customer opinions are displayed, e.g. do they like a product or not, opinion on quality and level of availability. This information allows organisations to predict new product diffusion, and then new product forecasts can be adjusted and sales and marketing activities sooner in first few weeks of launch.

Neural networks in Bayesian updating process

A study by Elalem provides a framework to use quantitative methods to forecast new product sales. Bayesian updating is the process where the parameters estimated by the Bass model are revised once sufficient sales have been recorded – to provide better forecasting results. Here, the researchers explore the use of artificial intelligence neural networks in the Bayesian updating process. The study proposes that neural networks are emerging as the most popular and suitable process to deal with unstructured data, such as texts, conversations or images.

Machine Learning (ML) and Dynamic Time Warping (DTW)

Also, Elalem mentions the use of ML and DTW. In analogy forecasting, often past similar products are grouped together by means of clustering, and then the products in a cluster are used as a basis for forecasting sales of a new product. Using managerial opinions, a ready-to launch new product is placed in a particular cluster and traditional curve fitting is applied to the average sales data in a cluster to forecast an entire product lifecycle before launch.

DTW is a way to compare two data sequences that do not perfectly synchronise. It is a method to calculate the optimal matching between two data sequences. This allows the positioning of new products in clusters of similar products, rather than relying only on managerial opinions as discussed above.

Then taking a look at ML techniques for time-series forecasting, ML methods are applied in a cross-learning manner, which means that they learn from multiple time series in a cluster to forecast one individual series. The ML-based approach combines the historical sales data of previously introduced products, as well as product characteristics of existing and new products, to develop prelaunch forecasts. ML can capture complex nonlinear relationships, but it usually requires significant amounts of data. When ML is supplemented with an expert’s domain knowledge, this can help relieve the need for vast training datasets.

New Heuristic Demand Planning model

In addition, a Japanese study highlights the use of Heuristic models. Heuristics is a type of decision making that does not require much data or analytics, rather it is based on one’s implicit knowledge from experience. Heuristics does not require complex analysis of large amounts of data, rather it requires focusing only on the most important bits of information. In the absence of data, we will rely on empirical observation and domain knowledge.

In this study, a Heuristic demand planning model has been developed. This tool supports decision- making by analysing multiple alternative products and weighting their attributes. In this way, the forecaster can understand which product characteristics (demand factors) have the greatest impact on forecast accuracy, and by extension, are most likely to impact demand for our new product, i.e. the most important bit of information

This model can be useful for new to world products (revolutionary) as it helps shift uncertainty to risk quantification.

Macro-flow models and Virtual Reality (VR)

In a further study, the use of macro-flow models and VR to forecast prelaunch sales is discussed. The macro-flow models predict the flows of potential buyers from one state of the purchase journey to the next. To make these predictions before launch, a purchase journey simulation is required. Simulation incorporates the potential buyer’s behaviours, flow between behavioural states and the determinants of the flow, e.g. from Becoming Aware, Info Search to the Final Purchase Decision. Here the determinants of flow are the independent variables in the forecasting model.

Then, together with the use of VR, the flow is studied, i.e. awareness creation, free information search, and the purchase decision. The researchers tracked individual level behavioural data, the type and number of actions per product at each information source, the time per action and product at each information source, and the visits per information source and product.

Conclusion

As we look to the future, the potential for NPF models to become even more sophisticated and integrated appears inevitable. The above studies have provided insights into how new emerging technologies can help in enabling data-driven analytical approaches to NPF, reducing the time to forecast (especially when there are hundreds of thousands of new product SKU combinations), and in improving NPF accuracy, while factoring in investments in new technology setup and implementation.

These data-driven analytical approaches will continue to become more dynamic, democratized and accessible, enabling organisations to not only to stay ahead of the curve but also to reshape their industries by predicting and setting trends.

By: Sharon Brand, Department of Business Studies, Cornerstone Institute, June 2024

References

Chase, C. 2019. Advancements in New Product Forecasting. The journal of business forecasting. 38(1):21–28.

Elalem, Y.K., Maier, S. & Seifert, R.W. 2023. A machine learning-based framework for forecasting sales of new products with short life cycles using deep neural networks. International journal of forecasting. 39(4):1874–1894. DOI: 10.1016/j.ijforecast.2022.09.005.

Kim, D., Woo, J., Shin, J., Lee, J. & Kim, Y. 2019. Can search engine data improve accuracy of demand forecasting for new products? Evidence from automotive market. Industrial management + data systems. 119(5):1089–1103. DOI: 10.1108/IMDS-08-2018-0347.

van Steenbergen, R.M. & Mes, M.R.K. 2020. Forecasting demand profiles of new products. Decision Support Systems. 139(July):113401. DOI: 10.1016/j.dss.2020.113401.

Yamamura, C.L.K., Santana, J.C.C., Masiero, B.S., Quintanilha, J.A. & Berssaneti, F.T. 2022. Forecasting New Product Demand Using Domain Knowledge and Machine Learning: A proposed method uses machine learning and an expert’s domain knowledge to enhance the accuracy of new product predictions. Research Technology Management. 65(4):27–36. DOI: 10.1080/08956308.2022.2062553.