Upgraded to DipDup 7.5.10

[ashutoshpw]

Overview

This PR upgrades the Homebase Indexer from DipDup v6.1.2 to v7.5.10, modernizes the configuration, improves handler implementations, and adds new features for historical DAO discovery. Resolves #42

Key Improvements

Major Version Upgrade

• DipDup upgrade: Migrated from v6.1.2 to v7.5.10
• Spec version: Updated dipdup.yml from spec_version 1.2 to 2.0
• Native WebSocket support: Removed TzKT proxy service as DipDup 7.x has native WebSocket support for TzKT 1.16+

Configuration Modernization

• Datasource URLs: Updated to use direct TzKT API endpoints (https://api.mainnet.tzkt.io and https://api.ghostnet.tzkt.io)
• Index configuration: Updated index kinds from operation to tezos.tzkt.operations
• Contract configuration: Added kind: tezos to contract definitions

Infrastructure Updates

• Added Rust toolchain installation for build dependencies
• Updated Postgres from default to version 16
• Made Hasura admin secret configurable via environment variable
• Removed services: Eliminated tzkt-proxy Nginx service and related configuration files (nginx-tzkt.conf, nginx.conf)
• Volume naming: Standardized volume name to v3-indexer-db

Handler Refactoring

• Model imports: Updated all handlers to use DipDup 7.x models:
• TzktTransaction instead of Transaction
• TzktOrigination instead of Origination
• TzktOperationData for operation data
• Storage access: Migrated from dictionary access (storage['key']) to attribute access (storage.key)
• Parameter access: Updated to use parameter.__root__ instead of data.parameter_json
• Type safety: Improved type handling with proper casting and Union types

New Features

• Historical DAO discovery: Added on_startup.py hook that automatically discovers and indexes historical DAOs by:
• Querying TzKT for all originations with matching code hash
• Verifying required entrypoints before indexing
• Creating indexes dynamically for discovered DAOs
• Handling race conditions and duplicate detection
• Entrypoint verification: Added validation to ensure DAOs have required entrypoints (propose, vote, flush, freeze, unfreeze, drop_proposal) before indexing
• Enhanced synchronization: Updated on_synchronized hook to trigger historical DAO discovery when insufficient DAOs are found
• Improved factory origination handler: Enhanced with entrypoint verification, better error handling, and support for both TzktOrigination and TzktOperationData types

Bug Fixes & Patches

• Reward field patch: Added fix_reward_field.py script to handle missing 'reward' field in TzKT API responses (patches DipDup's TzktBlockData.from_json method)
• Rollback hook: Updated to use string-based reindex reason instead of enum
• Error handling: Improved error logging with tracebacks and context information
• Proposal diff handling: Simplified proposal diff extraction in on_propose handler

📦 Dependencies

• Requirements cleanup: Removed explicit dependency list (now managed by DipDup base image)
• Build tools: Added build-essential, curl, python3-dev, and Rust toolchain to Dockerfile

Documentation Updates

• README updates:
• Updated version references from v6.1 to v7.x
• Removed TzKT proxy documentation
• Added note about native WebSocket support in DipDup 7.x

Migration Notes

• Database schema remains compatible (no breaking changes to models)
• Existing indexes will continue to work after upgrade
• Historical DAOs will be automatically discovered on first sync after upgrade
• Environment variables for database connection should be updated to use new component-based format

Testing Recommendations

• Verify all handlers work correctly with new model types
• Test historical DAO discovery on both mainnet and ghostnet
• Confirm WebSocket connections work without proxy
• Validate entrypoint verification logic
• Test rollback scenarios