Personal Expense Assistant with Gemini 2.5, Firestore and Cloud Run

A multimodal personal expense management assistant powered by Google's Agent Development Kit (ADK) and Gemini 2.5. This application helps users track expenses, analyze receipts, and manage financial records through a chat interface.

Features

• Upload and store receipt images with automatic data extraction
• Search receipts by metadata (date range, amount) or natural language queries
• Expense analysis and breakdown reports
• File retrieval of stored receipts
• Multimodal interaction (text + images)

Architecture

The application consists of:

• Frontend: Gradio-based chat interface
• Backend: FastAPI server with ADK agent
• Database: Firestore (NoSQL + vector search)
• Storage: Cloud Storage for receipt images
• Deployment: Cloud Run

Prerequisites

• Google Cloud account with billing enabled
• Python 3.12
• Basic understanding of Python and full-stack development

Setup Instructions

1. Google Cloud Setup

1. Create or select a Google Cloud project
2. Enable the following APIs:

   gcloud services enable aiplatform.googleapis.com \
                      firestore.googleapis.com \
                      run.googleapis.com \
                      cloudbuild.googleapis.com \
                      cloudresourcemanager.googleapis.com

2. Database Setup

1. Create Firestore database in Native mode
2. Create required indexes:

   # For compound queries
   gcloud firestore indexes composite create \
           --collection-group=personal-expense-assistant-receipts \
           --field-config field-path=total_amount,order=ASCENDING \
           --field-config field-path=transaction_time,order=ASCENDING \
           --field-config field-path=__name__,order=ASCENDING \
           --database="(default)"
   
   # For vector search
   gcloud firestore indexes composite create \
           --collection-group="personal-expense-assistant-receipts" \
           --query-scope=COLLECTION \
           --field-config field-path="embedding",vector-config='{"dimension":"768", "flat": "{}"}' \
           --database="(default)"

3. Storage Setup

Create a Cloud Storage bucket:

gsutil mb -l us-central1 gs://personal-expense-assistant-receipts

4. Configuration

Create settings.yaml with:

GCLOUD_LOCATION: "us-central1"
GCLOUD_PROJECT_ID: "your_gcloud_project_id"
BACKEND_URL: "http://localhost:8081/chat"
STORAGE_BUCKET_NAME: "personal-expense-assistant-receipts"
DB_COLLECTION_NAME: "personal-expense-assistant-receipts"

Development

Install Dependencies

Using uv Python package manager:

uv sync --frozen

Run Locally

1. Start backend service:

   uv run backend.py

2. Start frontend service (in another terminal):

   uv run frontend.py

Access the application at http://localhost:8080

Deployment

Deploy to Cloud Run:

gcloud run deploy personal-expense-assistant \
                  --source . \
                  --port=8080 \
                  --allow-unauthenticated \
                  --env-vars-file=settings.yaml \
                  --memory 1024Mi \
                  --region us-central1

Live Demo

The application is deployed and available at: https://personal-expense-assistant-242583023975.us-central1.run.app/

Key Components

Agent Development

• Agent Tools: Receipt storage, metadata search, vector search
• Callbacks: Image data optimization in conversation history
• Prompt Engineering: Structured response format and behavior guidance

Services

• Frontend: Gradio chat interface with file upload
• Backend: FastAPI server with ADK integration
• Session Management: In-memory session service
• Artifact Storage: Cloud Storage for receipt images