Spark Structured Streaming with CSV files
Publish Date: 2022-10-21 18 min read
Structured Streaming is a scalable and fault-tolerant stream processing engine built on the Spark SQL engine. You can express your streaming computation the same way you would express a batch computation on static data. The Spark SQL engine will take care of running it incrementally and...
read moreThe principles of Heuristic Function and GPS coordinates
Publish Date: 2022-10-20 10 min read
In sphere we have no strait lines, so the distance between 2 points on the surface of sphere measured by geodesics which is circles whose centers coincide with the center of the sphere, and they are called great circles. Between antipodal points (directly opposite each other ), there are infinitely...
read moreImplement A* Shortest Path using Neo4j
Publish Date: 2022-10-17 9 min read
The A* (pronounced "A-Star") Shortest Path algorithm computes the shortest path between two nodes. A* is an informed search algorithm as it uses a heuristic function to guide the graph traversal. The algorithm supports weighted graphs with positive relationship weights. Today We will learn How to...
read moreRealtime Analysis using Spark Streaming and Kafka
Publish Date: 2022-10-09 8 min read
Spark Streaming is an extension of the core Spark API that enables scalable, high-throughput, fault-tolerant stream processing of live data streams. Data can be ingested from many sources like Kafka, Kinesis, or TCP sockets, and can be processed using complex algorithms expressed with high-level...
read moreApache Spark and Cassandra : Best NoSQL Big Data combination
Publish Date: 2022-10-07 9 min read
Apache Spark™ is considered as the most powerful engine for executing data engineering, data science, and machine learning on single-node machines or clusters over diverse data sources such as NoSQL databases. Apache Cassandra is an open source NoSQL distributed database trusted by thousands of...
read more