Big Data Resources

Comprehensive Guide to Data Analysis and Visualization With Pandas and Matplotlib

Covering data preparation, descriptive statistics, profit margin calculations, and various types of plots such as bar charts, pie charts, and scatter plots.

June 7, 2024

by Krishnamurty Raju Mudunuru

· 4,378 Views · 1 Like

The Role of Data Engineering in Building Scalable AI Systems

Explore the role of data engineering in the field of artificial intelligence, specifically how data engineering influences the success and scalability of AI projects.

June 7, 2024

by Karthik Rajashekaran

· 2,443 Views · 2 Likes

ChatGPT: Your Guide to SQL Query Translation Between Databases

ChatGPT offers a solution for SQL translation by accurately translating queries, handling data types, and suggesting alternatives for proprietary features.

June 7, 2024

by Dmitry Narizhnykh

CORE

· 2,408 Views · 2 Likes

Building a Tool To Generate Text With OpenAI’s GPT-4 Model

Build a text generation tool using OpenAI's GPT-4. Set up your environment, authenticate the API, make API calls, handle errors, and integrate with Flask for a web app.

June 6, 2024

by Neha Dhaliwal

· 2,329 Views · 4 Likes

Streamlining Data Integration

This article demonstrates integrating Salesforce and Oracle into Amazon Redshift using Python and stored procedures, leveraging the ELT approach.

June 6, 2024

by Rajesh Remala

· 2,741 Views · 2 Likes

Take Your First Steps for Building on LLMs With Google Gemini

Learn to build an LLM application using the Google Gemini API and deploy it to Heroku. This guide walks you through setup, code, and deployment step-by-step.

June 5, 2024

by Alvin Lee

CORE

· 4,187 Views · 2 Likes

What Is Reverse ETL? Overview, Use Cases, and Key Benefits

Looking to go beyond traditional analytics? Reverse ETL is a nuanced process for businesses aiming to leverage data warehouses and other data platforms.

June 4, 2024

by Suhas Jangoan

· 3,759 Views · 2 Likes

High-Performance Storage Solutions

Pairing PostgreSQL with fast storage solutions is crucial for optimizing performance, minimizing downtime, and ensuring seamless data access.

June 3, 2024

by Sergey Platonov

· 1,873 Views · 3 Likes

An Overview of Data Pipeline Architecture

Dive into how a data pipeline helps process enormous amounts of data, key components, various architecture options, and best practices for maximum benefits.

Updated June 3, 2024

by Sreenath Devineni

· 12,210 Views · 4 Likes

Profiling Big Datasets With Apache Spark and Deequ

Struggling to maintain large dataset quality or ensure reliable data attributes in your analyses? Integrating Deequ with Spark might be the solution you need.

May 29, 2024

by Akshay Jain

· 75,514 Views · 4 Likes

Streamlining Data Warehouse Migrations From RedShift, Snowflake, and BigQuery to Apache Doris

Migrate Redshift, Snowflake, and BigQuery to Apache Doris for high-performance OLAP using tools like Apache Nifi, Talend, and Python.

May 28, 2024

by Sameer Danave

· 519 Views · 2 Likes

Preventing and Fixing Bad Data in Event Streams (Part 1)

In this blog, we’ll take a look at how bad data may come to be, and how we can deal with it when it comes to event streams.

May 28, 2024

by Adam Bellemare

· 2,057 Views · 1 Like

ARM CPU for Cost-Effective Apache Kafka at the Edge and Cloud

Learn about ARM CPU for cost-effective and energy-efficient data streaming with Apache Kafka at the Edge and in the Cloud, including use cases.

May 27, 2024

by Kai Wähner

CORE

· 1,217 Views · 1 Like

Bridging the Gap: Exploring Innovative Paths in Data Integration

This article solely focuses on delving into understanding data integrations, associated challenges, emerging technologies, and advanced methodologies.

May 27, 2024

by Yash Mehta

· 572 Views · 1 Like

Apache Doris for Log and Time Series Data Analysis

NetEase has replaced Elasticsearch and InfluxDB with Apache Doris, achieving 11X query performance and saving 70% of resources.

May 25, 2024

by Frank Z

· 1,119 Views · 1 Like

Capturing Acknowledgement in Kafka Streaming With RecordMetadata

In this article, explore RecordMetadata, and its attributes, and discover how it can be leveraged using various use cases.

May 24, 2024

by Raja Chattopadhyay

· 1,219 Views · 2 Likes

Useful Tips and Tricks for Data Scientists

In this article, I will share some of the tricks and tools that I am using to interpret the data in a fast and precise way and get useful insights from it.

May 22, 2024

by Pavel Perfilov

· 4,844 Views · 1 Like

Cost Optimization Strategies for High-Volume Data Platforms

Managing costs in running a Big Data Platform can be very challenging. This article talks through various strategies to optimize cost at every layer of the platform.

May 22, 2024

by Mayank Singhi

· 610 Views · 2 Likes

A Beginner’s Guide to Snowflake Architecture

Snowflake is a cloud-based data warehousing solution that targets removing the nightmares associated with business data storage, management, and analytics.

May 22, 2024

by Saigurudatta Pamulaparthyvenkata

· 904 Views · 2 Likes

Using Primary, Partition, and Clustering Keys in ScyllaDB (or Cassandra)

Learn core data modeling concepts that are critical for optimizing performance in wide-column NoSQL databases.

May 20, 2024

by Guy Shtub

· 779 Views · 1 Like

The Latest Big Data Topics