Building Real-Time Magic with HTTP Streaming in NestJS and React.js 🚀

Imagine you're the CTO of a dynamic fintech company, PayStream Inc., providing real-time transaction monitoring for businesses. Your clients need to track their transactions as they happen, without waiting for batch updates or page reloads. How do you deliver seamless, real-time updates?

The answer: HTTP Streaming! 💡

In this blog, we'll explore how to implement HTTP streaming in a NestJS and React.js application. By the end, you'll be streaming data like a pro and wowing your clients! ✨

---

🎯 The Problem

Your enterprise clients are frustrated with stale dashboards that require frequent refreshes to see the latest transaction data. They demand:

• Live updates for transactions.
• A solution that is scalable and can handle thousands of clients simultaneously.
• A simple, elegant user experience.

The challenge? Most solutions like WebSockets might be overkill for lightweight streaming scenarios. Enter HTTP Streaming: the perfect middle ground! 🌈

---

🛠️ The Tech Stack

• Backend: NestJS - for building scalable APIs.
• Frontend: React.js - for a smooth user experience.
• Protocol: HTTP/1.1 with Transfer-Encoding: chunked for streaming magic.

---

🛧️ The Backend: NestJS Streaming Service

Step 1: Create a Controller for Streaming 📡

import { Controller, Get, Res } from '@nestjs/common';
import { Response } from 'express';

@Controller('transactions')
export class TransactionsController {
  @Get('stream')
  streamTransactions(@Res() res: Response) {
    res.setHeader('Content-Type', 'text/event-stream');
    res.setHeader('Cache-Control', 'no-cache');
    res.setHeader('Connection', 'keep-alive');

    let transactionCount = 0;

    const interval = setInterval(() => {
      transactionCount++;
      const transaction = {
        id: transactionCount,
        amount: Math.random() * 1000,
        timestamp: new Date().toISOString(),
      };

      res.write(`data: ${JSON.stringify(transaction)}\n\n`);

      if (transactionCount === 10) {
        clearInterval(interval);
        res.end();
      }
    }, 1000);
  }
}

Step 2: Simulating Real-World Data Sources

Here, we’ve used setInterval to simulate incoming transactions. In real-world scenarios, you might fetch data from:

• Kafka: Stream data from topics in a message queue.
• RabbitMQ: Use message brokers to handle high-throughput data streams.
• Database Changes: Monitor inserts or updates using CDC (Change Data Capture).
• External APIs: Poll APIs and push responses to the client.

Step 3: Test Your Endpoint 🔮

Use curl to test the stream:

curl -N http://localhost:3000/transactions/stream

You should see real-time JSON data chunks flowing in! 🌊

---

💻 The Frontend: React.js Streaming Client

Step 1: Fetch and Render Data 🎥

Here’s how to connect to the NestJS stream in React:

import React, { useEffect, useState } from 'react';

const TransactionsStream = () => {
  const [transactions, setTransactions] = useState([]);

  useEffect(() => {
    const eventSource = new EventSource('http://localhost:3000/transactions/stream');

    eventSource.onmessage = (event) => {
      const transaction = JSON.parse(event.data);
      setTransactions((prev) => [...prev, transaction]);
    };

    return () => {
      eventSource.close();
    };
  }, []);

  return (
    <div>
      <h1>💸 Live Transactions</h1>
      <ul>
        {transactions.map((tx, index) => (
          <li key={index}>
            <strong>ID:</strong> {tx.id} | <strong>Amount:</strong> ${tx.amount.toFixed(2)} | <strong>Time:</strong> {tx.timestamp}
          </li>
        ))}
      </ul>
    </div>
  );
};

export default TransactionsStream;

Step 2: Style It Up 🎨

Use your favorite CSS framework (e.g., Tailwind CSS) to make the transaction list pop! ✨

---

🌍 Real-World Enterprise Scenario

At PayStream Inc., you deploy this feature to provide real-time transaction tracking. Here’s how it helps:

1. Improved User Experience: Clients no longer need to refresh dashboards. They see their transactions appear in real-time. 🚀
2. Reduced First Contentful Paint (FCP): By sending data in small chunks as it becomes available, HTTP streaming allows content to appear faster, significantly reducing the FCP and enhancing the perceived performance of the application. 🏎️
3. Backend Memory Optimization: Unlike buffered responses, streaming sends data as it's generated, reducing memory usage on the server. This is especially beneficial when handling large datasets or high-concurrency environments. 📉
4. Scalability: HTTP streaming works seamlessly for thousands of simultaneous users. 🤝💻
5. Cost Efficiency: No need for complex WebSocket infrastructure for simple streaming scenarios. 💰

---

❓ Questions One Might Have

1. Is this similar to Server-Sent Events (SSE)?

Yes! The example uses SSE under the hood, where updates are pushed to the client over a single HTTP connection. SSE is perfect for unidirectional data flows like real-time dashboards or notifications.

2. Why not use WebSockets?

While WebSockets are great for bi-directional communication, they can be overkill for simple one-way streaming. SSE is simpler, uses less overhead, and works seamlessly for scenarios where only the server needs to send updates.

3. How does HTTP streaming improve FCP?

By sending data in chunks as it becomes available, users see content faster. This reduces the perceived load time and enhances user satisfaction.

4. Does HTTP streaming reduce server memory usage?

Absolutely! Streaming sends data incrementally rather than buffering the entire response. This approach minimizes memory usage and helps the server handle more concurrent users efficiently.

5. What about data sources like Kafka or RabbitMQ?

In production, setInterval is replaced by real-time data sources such as:

• Kafka: Listen to messages from Kafka topics and stream them to the client.
• RabbitMQ: Push messages from a queue to the client as they arrive.
• Database Triggers: Monitor database changes in real-time and push updates.
• Third-Party APIs: Poll external APIs or use webhooks to fetch real-time data.

---

🔥 Key Takeaways

1. HTTP streaming is a lightweight yet powerful solution for real-time updates. 🌟
2. Combining NestJS and React.js enables you to build scalable and engaging applications. 🛠️
3. Sending data in chunks improves FCP and reduces server memory usage, ensuring a faster and smoother user experience. ✨
4. Always test for performance under load to ensure enterprise readiness. 🏈

---

💬 What’s Next?

Try adding advanced features like:

• Authentication to secure the stream. 🔒
• Filters to customize data streams for each client. 🔍
• Analytics to monitor stream performance. 📊

Have you implemented HTTP streaming in your projects? Share your experiences in the comments! 🚀

---

Happy Coding! 😄