Introduction
SenTMap
Over time markets have become sentiment driven. Stock prices are heavily influenced by economic events, company/industry performance, and market sentiments. Major financial institutes are heavily investing in Sentiment based analysis and decision making for portfolio managers.
SenTMap is an advanced research tool designed to keep investors up to date with analyzed financial news – so they can focus on what matters to their holdings and make better decisions. It gives investors confidence that valuable information is being extracted from the fire hose of financial news.
SenTMap is an advanced research tool designed to keep investors up to date with analyzed financial news – so they can focus on what matters to their holdings and make better decisions. It gives investors confidence that valuable information is being extracted from the fire hose of financial news.
Processing Firehose of information
to get the right information
The biggest challenge was to develop a backend system that can churn all the data coming from thousands of News Broadcast channels. After operating on that data, the system must present meaningful insight to the investors which will help them to understand the bearish or bullish trend of the market in real-time. The system should be scalable on both backend and frontend so that real-time states, historical states, and often merging of states can be handled smoothly.
The other major challenge was to present that continuously changing data into different reports, charts, and Three-dimensional trends in real-time. Unlike prepackaged visualizations, SenTMap required custom built charts display sentiments in way that required minimum visual processing.
The other major challenge was to present that continuously changing data into different reports, charts, and Three-dimensional trends in real-time. Unlike prepackaged visualizations, SenTMap required custom built charts display sentiments in way that required minimum visual processing.
Result
Market sentiment which can be positive(bullish) or negative(bearish) is output of people’s emotions, opinions and speculation towards any particular stock or entire market.
It is as important as fundamental and technical analysis. However, too much information can leave portfolio managers swamped and raises the problem of information overload. Unlike standard knowledge management system, SenTMap helps them keep the whole picture of financial news in view, including its market impact.
We built a scalable real-time analytics engine that can perform analysis based on real-time News Data to determine the trends in sentiments and stock price movements. Portfolio managers can now stay aligned with the large volume of processed content, and they can focus on what is really important for the investment.
Dealing with Big Data
in real time
When a system is ingested with news articles from thousands of sources constantly, you need single threaded server-side tech to process and execute all the requests without fail, and scalable server instances with high processing power. As we were not storing data anywhere, it was more difficult to process them in real-time. A leak of few articles or news can lead to wrong market sentiment.
We used Azure Event hubs to continuously ingress the data from different news sources with low latency and configurable time retention. Along with that, combination of ExpressJS, NodeJS and BackboneJS helped us to efficiently process the data on server-side. Processed data is then pushed to the client side, which ultimately boosts the performance.
We used Azure Event hubs to continuously ingress the data from different news sources with low latency and configurable time retention. Along with that, combination of ExpressJS, NodeJS and BackboneJS helped us to efficiently process the data on server-side. Processed data is then pushed to the client side, which ultimately boosts the performance.
Presenting the processed data using
3D charts and Treemapping
Portfolio managers love to get quick reports as it helps them buying, holding, or selling decisions. It was important to convert raw news data into technical charts which can notify investors whether the market sentiment for any specific stock or company is bullish or bearish. Unlike prepackaged visualizations, SenTMap required custom-built charts to display sentiments in a way that required minimum visual processing.
Along with popular charts in the financial market like line, bar, and candlestick, SenTMap required 3D charting capabilities with heat map. To build highly customized 3D charts, we utilized three.js, a Javascript 3D library to create 3D graphics. More complicated implementations involved integrating geometries, lighting and data mesh. To build Treemaps, we utilized squarified treemapping algorithm which is widely accepted by a broad audience, especially in financial contexts.
Along with popular charts in the financial market like line, bar, and candlestick, SenTMap required 3D charting capabilities with heat map. To build highly customized 3D charts, we utilized three.js, a Javascript 3D library to create 3D graphics. More complicated implementations involved integrating geometries, lighting and data mesh. To build Treemaps, we utilized squarified treemapping algorithm which is widely accepted by a broad audience, especially in financial contexts.
Presenting the processed data using
Portfolio managers love to get quick reports as it helps them buying, holding, or selling decisions. It was important to convert raw news data into technical charts which can notify investors whether the market sentiment for any specific stock or company is bullish or bearish. Unlike prepackaged visualizations, SenTMap required custom-built charts to display sentiments in a way that required minimum visual processing.
Along with popular charts in the financial market like line, bar, and candlestick, SenTMap required 3D charting capabilities with heat map. To build highly customized 3D charts, we utilized three.js, a Javascript 3D library to create 3D graphics. More complicated implementations involved integrating geometries, lighting and data mesh. To build Treemaps, we utilized squarified treemapping algorithm which is widely accepted by a broad audience, especially in financial contexts.
Along with popular charts in the financial market like line, bar, and candlestick, SenTMap required 3D charting capabilities with heat map. To build highly customized 3D charts, we utilized three.js, a Javascript 3D library to create 3D graphics. More complicated implementations involved integrating geometries, lighting and data mesh. To build Treemaps, we utilized squarified treemapping algorithm which is widely accepted by a broad audience, especially in financial contexts.