Global News Analysis Platform

This project leverages an advanced API to browse and analyze content from over 80,000 blogs and news sources, ranging from major media outlets to niche blogs. At its core, the platform employs a robust pipeline architecture designed to fetch, process, and visualize data, providing users with actionable insights and a richer understanding of news patterns.

Utilizing state-of-the-art technologies such as Elasticsearch and Kibana, alongside a suite of tools including Logstash, Zookeeper, and Kafka, the platform meticulously processes data fetched from newsapi.org. The system is engineered to perform named entity recognition (NER), extracting and analyzing significant entities within the content. This allows for the visualization of key trends and the frequency of named entities, offering a dynamic exploration of global news narratives.

Key Features:

• Comprehensive Data Analysis: Browse and analyze millions of news items from a vast array of sources.
• Advanced Visualization: Utilize Elasticsearch and Kibana for sophisticated data visualization, including vertical stacked bar plots, heat maps, tag clouds, and more.
• Real-time Processing: Fetch and process news titles every 15 minutes, ensuring up-to-date analysis and insights.
• Named Entity Recognition: Extract and analyze top named entities to uncover significant trends and patterns.

By offering a granular view of global news narratives through advanced data analysis and visualization techniques, this project empowers users to uncover hidden patterns, understand trends, and gain a deeper insight into the world's stories. It serves as a valuable tool for researchers, data analysts, and anyone with a curiosity for global news dynamics.