Auction System Documentation

This document describes how our auction system works. The system is split into two main parts: the Auction Distributor (which is responsible for handling and maintaining the socket connection. It Enables communication b/w Client and Auction Processor; Any kind of communication b/w auction processor and client is done via distributor) and the Auction Processor (which is responsible for generating & updating the auction entries, updating the prices and also handling the bids). Below, you'll find details about the high-level design, challenges faced, and a detailed flow of the entire process.

---

Demo

---

High Level Design

The auction system is designed for high concurrency, real-time updates, and fault tolerance. It leverages a microservices architecture to separate connection handling from business logic.

Key Components

• Client: The end-user interface that connects via WebSockets to receive real-time updates (price changes, bids, sold/unsold events).
• API Gateway: Acts as the entry point for HTTP requests and potentially WebSocket connections, routing traffic to the appropriate services.
• Auction Distributor (Socket Conn):
  • Manages persistent WebSocket connections with clients.
  • Broadcasts events (price updates, sold items) to specific rooms.
  • Forwards user actions (like placeBid) to the message broker.
  • Scalable horizontally to handle thousands of concurrent connections.
• Message Broker (Redis Pub/Sub):
  • Decouples the Distributor and Processor.
  • Facilitates real-time event propagation (e.g., when the Engine updates a price, it publishes to Redis, which the Distributor consumes and sends to clients or when client places a bid, it publishes to Redis, which the Engine consumes and make product sold and make order).
• Auction Engine (Processor):
  • The core business logic unit.
  • Auction Engine: Calculates price drops, manages auction state (QUEUED, RUNNING, SOLD, UNSOLD), and handles transitions.
  • Fault Tolerance (Recovery Manager): Ensures the system can recover from crashes by restoring state from the database and Redis.
  • Internal Queue: Manages local processing of bids and unsold item handling. Each of Unsold or PlaceBid is placed in Queued to prevent any last second bid race condition where unsold and sold transition is having race condition.
• Cache (Redis):
  • Stores ephemeral state for high-speed access:
    • liveAuctionMeta_<roomId>: Current price, status, product ID.
    • auctionId: List of auction items.
  • Used for distributed locking (Redlock) to prevent race conditions during bidding.
• Database (MongoDB):
  • Persists permanent data: Auctions, AuctionItems, Products, and Orders.
  • Source of truth for system recovery.
• Scheduler Service (QStash):
  • Triggers the start of scheduled auctions.
  • Ensures auctions start precisely at their designated times.

---

Challenges Faced and How We Solved It

1. Handling Concurrent Bids (Race Conditions)

Challenge: Multiple users might try to bid on the same item at the exact same millisecond. Without control, two users could "win" the same item, or the price could update incorrectly. Solution: Implemented Distributed Locking using Redlock (Redis) and a Internal Queueing of State transition. When a bid is placed it is first placed in internal queue to prevent any last second race condition of product state changes then the system acquires a lock on the specific auction item. Only the process holding the lock can validate and accept the bid. Other concurrent requests are rejected or queued, ensuring data integrity.

2. Real-time State Synchronization

Challenge: Keeping thousands of clients in sync with the rapidly changing auction price (decrementing every few seconds) and status without overloading the server. Solution:

• Decoupled Architecture: Separated the Auction Distributor (socket handling) from the Auction Processor (logic).
• Redis Pub/Sub: The Processor publishes state changes (price updates, sold events) to Redis. The Distributor subscribes to these channels and broadcasts updates only to the relevant rooms. This allows the heavy logic to run independently of the connection load.

3. Fault Tolerance and System Recovery

Challenge: If the Auction Processor crashes mid-auction, the active auction state (current item, current price, timer) could be lost, ruining the user experience. Solution:

• State Persistence: Critical state is constantly updated in Redis (Cache) and MongoDB (Persistent DB).
• Recovery Manager: On startup, the system checks for "orphaned" active auctions in the database. It reconstructs the in-memory state from Redis/DB, recalculates the correct current price based on the time elapsed, and resumes the auction seamlessly.

4. Scalability

Challenge: A monolithic app would struggle to handle both heavy computation (price logic) and thousands of open socket connections. Solution: Split into Distributor and Processor services. The Distributor can be scaled horizontally (adding more instances) to handle more users, while the Processor can be scaled independently based on the number of active auctions.

---

Overall Flow of the Auction Process

1. Auction Start Trigger When the auction start endpoint is hit (e.g., GET /start/:auctionId), the following happens:

   • Router calls:
     • schedulerService.unscheduleAuction(auctionId) to ensure there are no existing schedules. for the given auctionId.
     • auctionEngine.startAuctionInRoom(auctionId, roomId) (Typically, roomId is the same as auctionId.)

2. Starting the Auction (AuctionEngine.startAuctionInRoom)

   • AuctionEngine.startAuctionInRoom is invoked by the router. Inside this method:
     • It calls auctionService.getOngoingAuctions() to check if the auction is already running.
     • If not running:
       • It calls auctionService.insertOngoingAuction(auctionId) to mark it as active.
       • It calls auctionService.queueInAuctionItemsInDBAndCache(auctionId) to fetch auction items from the database, set their product status to QUEUED, and cache them.
       • It calls auctionMetadataService.insertAuctionMetaForDay(auctionId) to create initial auction metadata.
     • Then it retrieves auction configuration data via auctionService.getAuctionInfoById(auctionId).
     • It publishes an "auction start" event with redisEventHandler.publishAuctionStart(roomId) which ultimately publishes the event that to the client that auction is about to start.
     • Finally, it schedules the first product for the auction by calling prepareNextProductInRoomIdWithDelay(roomId, auctionId, delayOptions).

3. Preparing the Next Product

   • AuctionEngine.prepareNextProductInRoomIdWithDelay
     • After a short fixed delay, it publishes a "preparing next" event using redisEventHandler.publishPreparingNext(roomId).
     • Then it calls prepareNextProductInRoomId(roomId, auctionId, delayOptions) to actually load the next auction item.
   • AuctionEngine.prepareNextProductInRoomId
     • Clears any existing price update intervals.
     • Retrieves the current auction metadata with auctionMetadataService.getCurrentAuctionMeta(roomId).
     • Gets all auction item IDs via auctionService.getAuctionItemIds(auctionId).
     • Calls checkAndHandleIfAuctionEnd to see if there are more items queued for auction. If not, it triggers triggerAuctionEnd.
     • If more items exist, it:
       • Determines the next product index.
       • Fetches the next auction item with auctionService.getAuctionItem.
       • Retrieves auction configuration data via auctionService.getAuctionInfoById.
       • Updates the starting price if needed (including adjustments based on the previous item’s sale status) by calling auctionService.updateAuctionItemPrice.
       • Constructs new auction metadata and updates it using auctionMetadataService.updateAuctionMeta.
       • Waits for a delay (based on either product or section intervals) and then starts the price update cycle.
     • It initializes the price update cycle by:
       • Calling the function returned by priceUpdateHandler once (to immediately show the starting price).
       • Setting an interval that repeatedly calls the update handler at intervals defined by the auction configuration.

4. Bidding Process

   • Handling a Bid (AuctionEngine.handlePlaceBid)
     • When a bid event is received (via a subscription from redisEventHandler), the callback handlePlaceBid is invoked.
     • This method:
       • Acquires a distributed lock (using Redlock) to avoid race conditions(Preventing multiple users to place bid once the bid has been placed by the first user.)
       • Fetches auction listings by calling auctionService.getAuctionItems(auctionId).
       • Validates the bid against the current auction metadata (retrieved via auctionMetadataService.getCurrentAuctionMeta(roomId)).
       • If the bid is valid (i.e. the bid is on the currently queued listing):
         • It updates the bid price in the metadata.
         • It calls markCurrentListingSoldAndNotifyUsers to mark the listing as sold.
   • Marking the Listing as Sold (AuctionEngine.markCurrentListingSoldAndNotifyUsers)
     • This function:
       • Clears the current price decrement interval.
       • Calls auctionService.markAuctionItemSoldAndPlaceOrder to update the database (mark the item as sold, create an order, etc.).
       • Calls auctionService.updateAuctionItemStatus to update the item’s status in the cache.
       • Publishes a "product sold" event via redisEventHandler.publishProductSold to notify all connected clients.
       • If the sale is successful, it calls prepareNextProductInRoomIdWithDelay to load the next auction item.

5. Price Updates and Unsold Items

   • Price Update Handler (AuctionEngine.priceUpdateHandler)
     • Runs at regular intervals (set by an interval timer).
     • Retrieves the current auction metadata.
     • Calculates a new price by decrementing from the original price.
     • If the price falls below the minimum without a bid:
       • Updates the metadata to indicate no sale.
       • Clears the update interval.
       • Calls markCurrentListingUnsold to mark the listing as unsold.
       • Then calls prepareNextProductInRoomIdWithDelay to move to the next product.
     • If the new price is still above the minimum, it updates the round number and publishes the new price via redisEventHandler.publishPriceUpdate, then updates the metadata.

6. Ending the Auction

   • AuctionEngine.checkAndHandleIfAuctionEnd is called in the product preparation process to verify if there are any remaining items.
   • If no items remain, it calls AuctionEngine.triggerAuctionEnd, which:
     • Clears all intervals.
     • Publishes an auction end (or cancel) event.
     • Cleans up auction items from the cache and database by calling functions in the AuctionService (e.g., deleteAuctionItemsFromCacheAndDB, removeOngoingAuction).
     • Updates the final auction status in the database (COMPLETED or CANCELLED).
     • Updates sold and unsold counts via auctionService.updateAuctionProductIds and auctionService.updateAuctionProductCounts.

---

Summary of Function Calls and Flow

• Endpoint Trigger (/start/:auctionId)

  • Router calls:
    • schedulerService.unscheduleAuction(auctionId)
    • auctionEngine.startAuctionInRoom(auctionId, roomId)

• AuctionEngine.startAuctionInRoom

  • Checks ongoing auctions via auctionService.getOngoingAuctions
  • If not ongoing:
    • Calls auctionService.insertOngoingAuction(auctionId)
    • Calls auctionService.queueInAuctionItemsInDBAndCache(auctionId)
    • Calls auctionMetadataService.insertAuctionMetaForDay(auctionId)
  • Retrieves auction configuration with auctionService.getAuctionInfoById(auctionId)
  • Publishes auction start using redisEventHandler.publishAuctionStart(roomId)
  • Schedules first product with prepareNextProductInRoomIdWithDelay(roomId, auctionId, delayOptions)

• prepareNextProductInRoomIdWithDelay

  • Waits a short delay then:
    • Publishes "preparing next" event via redisEventHandler.publishPreparingNext(roomId)
    • Calls prepareNextProductInRoomId(roomId, auctionId, delayOptions)

• prepareNextProductInRoomId

  • Clears previous intervals.
  • Retrieves current metadata (auctionMetadataService.getCurrentAuctionMeta(roomId))
  • Gets auction item IDs via auctionService.getAuctionItemIds(auctionId)
  • Checks if auction is over with checkAndHandleIfAuctionEnd
  • Fetches next auction item with auctionService.getAuctionItem
  • Updates auction item price using auctionService.updateAuctionItemPrice
  • Creates and updates new auction metadata with auctionMetadataService.updateAuctionMeta
  • Waits for a configured delay and initializes the price update cycle via priceUpdateHandler

• Price Update Cycle (priceUpdateHandler)

  • Decrements the current price at regular intervals.
  • Publishes price updates via redisEventHandler.publishPriceUpdate
  • If price falls below minimum, calls markCurrentListingUnsold and then prepares the next product.

• Bid Handling

  • When a bid event is received, handlePlaceBid(roomId, bidderMeta) is called by the Redis event listener.
  • This function:
    • Acquires a lock.
    • Validates the bid against current auction metadata.
    • Calls markCurrentListingSoldAndNotifyUsers to process the bid.
  • markCurrentListingSoldAndNotifyUsers
    • Clears the current update interval.
    • Calls auctionService.markAuctionItemSoldAndPlaceOrder to update DB and create an order.
    • Calls auctionService.updateAuctionItemStatus to update the cache.
    • Publishes a "product sold" event via redisEventHandler.publishProductSold
    • Calls prepareNextProductInRoomIdWithDelay if successful.

• Auction End

  • When no more products remain, checkAndHandleIfAuctionEnd triggers triggerAuctionEnd.
  • triggerAuctionEnd
    • Clears intervals.
    • Publishes an auction end (or cancel) event.
    • Cleans up auction data from cache and database.
    • Updates auction status and product counts.

---