diff --git a/CLAUDE.md b/CLAUDE.md new file mode 100644 index 0000000..3a0a07b --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1,191 @@ +# CLAUDE.md + +This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository. + +## Project Overview + +The Open Compute Framework (OCF) is a decentralized computing platform that combines LibP2P-based peer-to-peer networking with Web3 blockchain integration. The platform enables distributed compute services with token-based incentives and modern web interfaces. + +## Architecture Overview + +### Core Components + +1. **Backend (Go)** - `src/`: LibP2P-based decentralized compute fabric + - Implements CRDT-backed service registry and peer discovery + - HTTP gateway (port 8092) and P2P HTTP over LibP2P + - Multi-modal operation: standalone, local, and networked modes + - Entry points in `src/entry/cmd/` with wallet management capabilities + +2. **Frontend (Next.js)** - `apps/web/`: Modern React web application + - Web3 wallet integration with wagmi and ethers + - Responsive UI with Radix UI and Tailwind CSS + - TypeScript with strict configuration + +3. **Proxy Service (Python)** - `apps/proxy/`: FastAPI-based OpenAI-compatible proxy + - Async request handling with timeout management + - Routes to OpenAI-compatible endpoints (/v1/chat/completions, etc.) + - Docker containerization support + +4. **Blockchain (Solana)** - `tokens/`: Token economics and incentives + - Anchor framework for Solana smart contracts + - Rust-based token implementation + +5. **Documentation (Astro)** - `docs/`: Comprehensive project documentation + - Starlight theme with professional styling + +## Development Commands + +### Backend (Go) + +From the `src/` directory: + +```bash +# Build and development +make build # Build all applications +make build-debug # Build with debugging capabilities +make build-release # Build release binaries (no debug info) +make run # Build and execute all applications + +# Testing and quality +make test # Run tests with coverage +make lint # Run linters (golangci-lint) +make check # Run both tests and linters + +# Release management +make patch # Release new patch version +make minor # Release new minor version +make major # Release new major version +``` + +### Frontend (Next.js) + +From the `apps/web/` directory: + +```bash +npm run dev # Start development server +npm run build # Build for production +npm run start # Start production server +npm run lint # Run ESLint +``` + +### Blockchain (Solana/Anchor) + +From the `tokens/` directory: + +```bash +npm run lint # Check code formatting +npm run lint:fix # Fix code formatting +yarn run ts-mocha # Run TypeScript tests +``` + +### Proxy Service (Python) + +From the `apps/proxy/` directory: + +```bash +# Local development +pip install -r requirements.txt +python main.py + +# Docker build and run +./scripts/build_docker.sh +docker run -p 8000:8000 researchcomputer/ocf-proxy + +# Environment variables +TARGET_SERVICE_URL=http://localhost:8000/v1 # Target service URL +TIMEOUT=30.0 # Request timeout in seconds +``` + +### Documentation (Astro) + +From the `docs/` directory: + +```bash +npm run dev # Start development server +npm run build # Build documentation +npm run preview # Preview built site +``` + +## Key Technical Details + +### Backend Architecture + +- **LibP2P Networking**: TCP, WebSocket, and QUIC transports +- **CRDT Registry**: Conflict-free replicated data types for distributed state +- **Service Discovery**: Identity-based service registration and routing +- **Multi-Transport**: HTTP gateway and P2P HTTP over LibP2P + +### Routing Models + +1. **Direct Service**: Forward to locally registered services +2. **Global Service**: Route to matching providers by identity groups +3. **P2P Forwarding**: Direct peer-to-peer communication + +### Frontend Integration + +- **Web3 Stack**: wagmi v2.16.9, ethers v6.15.0, web3modal +- **State Management**: TanStack Query for data fetching +- **UI Components**: Radix UI with Tailwind CSS +- **Type Safety**: Strict TypeScript configuration + +### Blockchain Integration + +- **Solana Program ID**: `xTRCFBHAfjepfKNStvWQ7xmHwFS7aJ85oufa1BoXedL` +- **Anchor Framework**: Rust smart contract development +- **Token Economics**: Platform incentive mechanisms + +## Important Build Information + +### Go Backend + +- **Version**: Go 1.23.0 required +- **Build Output**: `build/` directory +- **Entry Points**: `src/entry/cmd/` with CLI commands (start, config, wallet, etc.) +- **Linting**: golangci-lint v1.61.0 +- **Testing**: gotestsum v0.4.2 with coverage reports +- **Wallet Management**: Integrated wallet functionality for node owner identification + +### Multi-Architecture Support + +- **AMD64**: Default build target +- **ARM64**: Available via `make arm` target +- **Release Builds**: Stripped binaries in `build/release/` + +### Environment Variables + +Key build-time environment variables: +- `AUTH_URL`, `AUTH_CLIENT_ID`, `AUTH_CLIENT_SECRET` +- `SENTRY_DSN` for error tracking +- `VERBOSE=1` for verbose build output + +## Development Workflow + +1. **Backend Changes**: Work in `src/`, use `make build && make run` for testing +2. **Frontend Changes**: Work in `apps/web/`, use `npm run dev` for hot reload +3. **Proxy Service**: Work in `apps/proxy/`, use `python main.py` for local testing +4. **Blockchain Changes**: Work in `tokens/`, use Anchor tooling for deployment +5. **Documentation**: Work in `docs/`, use Astro dev server for preview + +## Code Style and Conventions + +### Go Backend +- Follow standard Go formatting and conventions +- Use golangci-lint for code quality +- Comprehensive test coverage with gotestsum + +### Frontend +- TypeScript strict mode enabled +- ESLint configuration for code quality +- Tailwind CSS utility classes for styling +- Radix UI components for accessibility + +### Proxy Service +- FastAPI with async request handling +- Python type hints with pydantic validation +- Docker containerization for deployment +- Environment-based configuration + +### Blockchain +- Rust smart contracts with Anchor framework +- TypeScript tests for contract verification +- Prettier formatting for JavaScript/TypeScript files \ No newline at end of file diff --git a/TODO.md b/TODO.md new file mode 100644 index 0000000..8b368fa --- /dev/null +++ b/TODO.md @@ -0,0 +1,117 @@ +# OpenComputeFramework (OCF) - Future Development Roadmap + +## Core Architecture & Protocol + +- [ ] **CRDT State Synchronization Improvements** + - Implement conflict resolution policies for service registration (e.g., last-write-wins vs. merge policies) + - Add version vectors or vector clocks to detect and resolve concurrent updates to peer records + - [x] Introduce tombstone compaction for deleted peers to prevent datastore bloat + - Add CRDT health monitoring and automatic repair mechanism (e.g., periodic state sync with bootstrap nodes) + +- [ ] **P2P Network Resilience** + - Implement peer scoring and reputation system to prioritize connections with reliable nodes + - [x] Add automatic reconnection logic with exponential backoff for transient network failures + - Implement connection rate limiting to prevent DDoS or misbehaving peers + - Add support for rendezvous servers for NAT traversal in restrictive networks + +- [ ] **Bootstrap Discovery Enhancements** + - Implement DNS-based bootstrap discovery (e.g., `_ocf-bootstrap._tcp.example.com`) + - [x] Add support for multiple bootstrap sources (HTTP, DNS, static list) with fallback logic + - Implement bootstrap node validation to prevent malicious bootstrap lists + - Add bootstrap node rotation and load balancing for high availability + +## Service Registry & Routing + +- [ ] **Service Discovery & Registration** + - Implement service health checks (HTTP/TCP) for registered services to auto-remove unhealthy providers + - Add service metadata (e.g., latency, cost, region) to enable intelligent routing decisions + - Implement service versioning and compatibility checks for API endpoints + - Add support for dynamic service deregistration (e.g., on process exit) + +- [ ] **Intelligent Request Routing** + - Implement load balancing algorithms (round-robin, least-connections, weighted) for global service routing + - Add geographic routing based on peer location (IP geolocation or explicit location metadata) + - Implement cost-aware routing (e.g., prioritize free or low-cost providers) + - Add request prioritization and QoS support (e.g., high-priority requests bypass queues) + +- [ ] **Identity & Access Control** + - Implement JWT-based authentication for service access (e.g., `model=resnet50` requires valid token) + - Add role-based access control (RBAC) for service registration and routing + - Implement peer identity verification using public key signatures + - Add audit logging for all service access attempts + +## Server & API Layer + +- [ ] **HTTP API Enhancements** + - Add OpenAPI/Swagger documentation for all API endpoints + - Implement rate limiting and request throttling per peer/IP + - Add pagination and filtering for `/v1/dnt/table` and `/v1/dnt/peers` endpoints + - Implement WebSocket support for real-time event streaming (e.g., peer join/leave) + +- [ ] **Observability & Monitoring** + - Add Prometheus metrics endpoint (`/metrics`) for key metrics (peers, requests, latency, errors) + - Implement structured logging with trace IDs for end-to-end request tracing + - Add support for OpenTelemetry integration (beyond Axiom) + - Implement alerting for critical events (e.g., no active peers, high error rate) + +- [ ] **Security Hardening** + - Restrict CORS origins to trusted domains instead of `*` + - Implement mutual TLS (mTLS) for P2P communication + - Add request signature validation for all forwarded requests + - Implement input sanitization and validation for all API endpoints + +## Entry Point & Configuration + +- [ ] **CLI & Configuration Management** + - Add config validation and schema checking (e.g., using JSON Schema) + - Implement config hot-reload without restarting the node + - Add config export/import functionality for backup and migration + - Implement environment variable overrides for all config options + +- [ ] **Deployment & Orchestration** + - Create Helm chart for Kubernetes deployment + - Add Docker Compose templates for local development and testing + - Implement systemd service files for production deployment + - Add health check endpoints for container orchestration (e.g., `/healthz`) + +## Documentation & Developer Experience + +- [ ] **Documentation Updates** + - Update architecture.md with latest routing logic and CRDT details + - Add sequence diagrams for service registration and request routing + - Document all CLI flags and config options in a single reference page + - Add troubleshooting guide for common issues (e.g., connection failures, bootstrap errors) + +- [ ] **Developer Tooling** + - Create a CLI tool for debugging and inspecting the local node state + - Add unit test coverage for all core packages (protocol, server, common) + - Implement integration tests for P2P communication and service routing + - Add a local development mode with mock services for testing + +## Performance & Scalability + +- [ ] **Performance Optimization** + - Profile and optimize CRDT serialization/deserialization performance + - Implement connection pooling for HTTP proxying to reduce overhead + - Add memory usage monitoring and limits to prevent OOM crashes + - Optimize DHT lookups and reduce latency for global service routing + +- [ ] **Scalability Improvements** + - Implement sharding of the node table for large networks (>10k peers) + - Add support for hierarchical routing (e.g., regional clusters) + - Implement caching of frequently accessed service records + - Add support for federated networks (multiple independent OCF networks) + +## Integration & Ecosystem + +- [ ] **Third-Party Integrations** + - Add support for integrating with existing ML platforms (e.g., vLLM, TGI) + - Implement OpenAI-compatible API endpoint for seamless client integration + - Add support for Kubernetes CSI for dynamic resource provisioning + - Integrate with blockchain for decentralized payment and incentive mechanisms + +- [ ] **Community & Ecosystem** + - Create a public registry of known bootstrap nodes + - Implement a plugin system for custom service types + - Add a web-based dashboard for monitoring the network + - Create a developer SDK for Go, Python, and JavaScript diff --git a/apps/README.md b/apps/README.md new file mode 100644 index 0000000..e69de29 diff --git a/apps/package-lock.json b/apps/package-lock.json new file mode 100644 index 0000000..35ee3dd --- /dev/null +++ b/apps/package-lock.json @@ -0,0 +1,6 @@ +{ + "name": "apps", + "lockfileVersion": 3, + "requires": true, + "packages": {} +} diff --git a/apps/web/README-CONSOLE.md b/apps/web/README-CONSOLE.md new file mode 100644 index 0000000..d52c475 --- /dev/null +++ b/apps/web/README-CONSOLE.md @@ -0,0 +1,146 @@ +# OCF Web Console + +A decentralized computing web console built on Next.js 15 with wallet authentication and machine management capabilities. + +## Features + +- **Wallet Authentication**: Connect with MetaMask, WalletConnect, and other Web3 wallets +- **Machine Management**: View connected computing resources and their status +- **API Key Management**: Generate and manage API keys for programmatic access +- **Real-time Status**: Monitor machine health, resource usage, and served models +- **Multi-chain Support**: Works with Ethereum, Polygon, Arbitrum, and Optimism +- **Responsive Design**: Built with Tailwind CSS and shadcn/ui components + +## Setup Instructions + +### 1. Install Dependencies + +```bash +npm install +``` + +### 2. Environment Configuration + +Create a `.env.local` file in the root directory: + +```env +NEXT_PUBLIC_WALLETCONNECT_PROJECT_ID=your_walletconnect_project_id_here +NEXT_PUBLIC_API_BASE_URL=http://localhost:3000/api +``` + +To get a WalletConnect Project ID: +1. Visit [WalletConnect Cloud](https://cloud.walletconnect.com/) +2. Create a new project +3. Copy the Project ID + +### 3. Run the Development Server + +```bash +npm run dev +``` + +The application will be available at `http://localhost:3001` + +## How It Works + +### Authentication Flow +1. User clicks "Connect Wallet" in the navbar +2. Web3Modal opens with wallet options +3. User connects their wallet (MetaMask, WalletConnect, etc.) +4. Application switches to dashboard view + +### Dashboard Features +- **User Profile**: Shows wallet address, ENS name (if available), and balance +- **Statistics**: Overview of total machines, active models, API keys, and requests +- **Machine Status**: Real-time status of connected computing resources +- **API Management**: Generate and manage API keys for accessing the OCF network + +### API Endpoints + +#### Get Machines +``` +GET /api/machines?walletAddress=0x... +``` + +#### Get API Keys +``` +GET /api/api-keys?walletAddress=0x... +``` + +#### Generate API Key +``` +POST /api/api-keys +{ + "walletAddress": "0x...", + "name": "API Key Name" +} +``` + +## Project Structure + +``` +apps/web/ +├── app/ +│ ├── api/ +│ │ ├── machines/ +│ │ └── api-keys/ +│ ├── layout.tsx +│ └── page.tsx +├── components/ +│ ├── dashboard.tsx +│ ├── wallet-connect.tsx +│ ├── ui/ +│ └── navbar/ +├── lib/ +│ ├── web3-provider.tsx +│ └── utils.ts +└── package.json +``` + +## Tech Stack + +- **Frontend**: Next.js 15, React 19, TypeScript +- **Styling**: Tailwind CSS, shadcn/ui +- **Web3**: Wagmi, Ethers.js, Web3Modal +- **State Management**: React Query +- **Authentication**: WalletConnect, MetaMask + +## Development + +### Available Scripts + +```bash +npm run dev # Start development server +npm run build # Build for production +npm run start # Start production server +npm run lint # Run ESLint +``` + +### Building for Production + +```bash +npm run build +npm start +``` + +## Future Enhancements + +- Real-time machine status updates with WebSockets +- Machine registration and management +- Detailed usage analytics and billing +- Model deployment and management +- Advanced API key permissions and rate limiting +- Mobile app integration +- Multi-user support with role-based access control + +## Contributing + +1. Fork the repository +2. Create a feature branch +3. Make your changes +4. Add tests if applicable +5. Submit a pull request + +## License + +This project is part of the Open Compute Framework and is licensed under the MIT License. \ No newline at end of file diff --git a/apps/web/app/layout.tsx b/apps/web/app/layout.tsx index 63eebdd..2c28597 100644 --- a/apps/web/app/layout.tsx +++ b/apps/web/app/layout.tsx @@ -2,6 +2,7 @@ import type { Metadata } from "next"; import { Geist } from "next/font/google"; import "./globals.css"; import { ThemeProvider } from "next-themes"; +import { Web3Provider } from "@/lib/web3-provider"; const geistSans = Geist({ variable: "--font-geist-sans", @@ -101,7 +102,9 @@ export default function RootLayout({ - {children} + + {children} + diff --git a/apps/web/app/page.tsx b/apps/web/app/page.tsx index 96cb452..85c6d03 100644 --- a/apps/web/app/page.tsx +++ b/apps/web/app/page.tsx @@ -1,21 +1,19 @@ -import FAQ from "@/components/faq"; -import Features from "@/components/features"; -import Footer from "@/components/footer"; -import Hero from "@/components/hero"; -import { Navbar } from "@/components/navbar"; -import Testimonial from "@/components/testimonial"; -import CodeExample from "@/components/code-example"; +'use client' + +import { useAccount } from 'wagmi' +import Hero from '@/components/hero' +import { Dashboard } from '@/components/dashboard' export default function Home() { + const { isConnected } = useAccount() + + if (isConnected) { + return + } + return ( <> - - - - - -