Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
191 changes: 191 additions & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,191 @@
# CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

## Project Overview

The Open Compute Framework (OCF) is a decentralized computing platform that combines LibP2P-based peer-to-peer networking with Web3 blockchain integration. The platform enables distributed compute services with token-based incentives and modern web interfaces.

## Architecture Overview

### Core Components

1. **Backend (Go)** - `src/`: LibP2P-based decentralized compute fabric
- Implements CRDT-backed service registry and peer discovery
- HTTP gateway (port 8092) and P2P HTTP over LibP2P
- Multi-modal operation: standalone, local, and networked modes
- Entry points in `src/entry/cmd/` with wallet management capabilities

2. **Frontend (Next.js)** - `apps/web/`: Modern React web application
- Web3 wallet integration with wagmi and ethers
- Responsive UI with Radix UI and Tailwind CSS
- TypeScript with strict configuration

3. **Proxy Service (Python)** - `apps/proxy/`: FastAPI-based OpenAI-compatible proxy
- Async request handling with timeout management
- Routes to OpenAI-compatible endpoints (/v1/chat/completions, etc.)
- Docker containerization support

4. **Blockchain (Solana)** - `tokens/`: Token economics and incentives
- Anchor framework for Solana smart contracts
- Rust-based token implementation

5. **Documentation (Astro)** - `docs/`: Comprehensive project documentation
- Starlight theme with professional styling

## Development Commands

### Backend (Go)

From the `src/` directory:

```bash
# Build and development
make build # Build all applications
make build-debug # Build with debugging capabilities
make build-release # Build release binaries (no debug info)
make run # Build and execute all applications

# Testing and quality
make test # Run tests with coverage
make lint # Run linters (golangci-lint)
make check # Run both tests and linters

# Release management
make patch # Release new patch version
make minor # Release new minor version
make major # Release new major version
```

### Frontend (Next.js)

From the `apps/web/` directory:

```bash
npm run dev # Start development server
npm run build # Build for production
npm run start # Start production server
npm run lint # Run ESLint
```

### Blockchain (Solana/Anchor)

From the `tokens/` directory:

```bash
npm run lint # Check code formatting
npm run lint:fix # Fix code formatting
yarn run ts-mocha # Run TypeScript tests
```

### Proxy Service (Python)

From the `apps/proxy/` directory:

```bash
# Local development
pip install -r requirements.txt
python main.py

# Docker build and run
./scripts/build_docker.sh
docker run -p 8000:8000 researchcomputer/ocf-proxy

# Environment variables
TARGET_SERVICE_URL=http://localhost:8000/v1 # Target service URL
TIMEOUT=30.0 # Request timeout in seconds
```

### Documentation (Astro)

From the `docs/` directory:

```bash
npm run dev # Start development server
npm run build # Build documentation
npm run preview # Preview built site
```

## Key Technical Details

### Backend Architecture

- **LibP2P Networking**: TCP, WebSocket, and QUIC transports
- **CRDT Registry**: Conflict-free replicated data types for distributed state
- **Service Discovery**: Identity-based service registration and routing
- **Multi-Transport**: HTTP gateway and P2P HTTP over LibP2P

### Routing Models

1. **Direct Service**: Forward to locally registered services
2. **Global Service**: Route to matching providers by identity groups
3. **P2P Forwarding**: Direct peer-to-peer communication

### Frontend Integration

- **Web3 Stack**: wagmi v2.16.9, ethers v6.15.0, web3modal
- **State Management**: TanStack Query for data fetching
- **UI Components**: Radix UI with Tailwind CSS
- **Type Safety**: Strict TypeScript configuration

### Blockchain Integration

- **Solana Program ID**: `xTRCFBHAfjepfKNStvWQ7xmHwFS7aJ85oufa1BoXedL`
- **Anchor Framework**: Rust smart contract development
- **Token Economics**: Platform incentive mechanisms

## Important Build Information

### Go Backend

- **Version**: Go 1.23.0 required
- **Build Output**: `build/` directory
- **Entry Points**: `src/entry/cmd/` with CLI commands (start, config, wallet, etc.)
- **Linting**: golangci-lint v1.61.0
- **Testing**: gotestsum v0.4.2 with coverage reports
- **Wallet Management**: Integrated wallet functionality for node owner identification

### Multi-Architecture Support

- **AMD64**: Default build target
- **ARM64**: Available via `make arm` target
- **Release Builds**: Stripped binaries in `build/release/`

### Environment Variables

Key build-time environment variables:
- `AUTH_URL`, `AUTH_CLIENT_ID`, `AUTH_CLIENT_SECRET`
- `SENTRY_DSN` for error tracking
- `VERBOSE=1` for verbose build output

## Development Workflow

1. **Backend Changes**: Work in `src/`, use `make build && make run` for testing
2. **Frontend Changes**: Work in `apps/web/`, use `npm run dev` for hot reload
3. **Proxy Service**: Work in `apps/proxy/`, use `python main.py` for local testing
4. **Blockchain Changes**: Work in `tokens/`, use Anchor tooling for deployment
5. **Documentation**: Work in `docs/`, use Astro dev server for preview

## Code Style and Conventions

### Go Backend
- Follow standard Go formatting and conventions
- Use golangci-lint for code quality
- Comprehensive test coverage with gotestsum

### Frontend
- TypeScript strict mode enabled
- ESLint configuration for code quality
- Tailwind CSS utility classes for styling
- Radix UI components for accessibility

### Proxy Service
- FastAPI with async request handling
- Python type hints with pydantic validation
- Docker containerization for deployment
- Environment-based configuration

### Blockchain
- Rust smart contracts with Anchor framework
- TypeScript tests for contract verification
- Prettier formatting for JavaScript/TypeScript files
117 changes: 117 additions & 0 deletions TODO.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,117 @@
# OpenComputeFramework (OCF) - Future Development Roadmap

## Core Architecture & Protocol

- [ ] **CRDT State Synchronization Improvements**
- Implement conflict resolution policies for service registration (e.g., last-write-wins vs. merge policies)
- Add version vectors or vector clocks to detect and resolve concurrent updates to peer records
- [x] Introduce tombstone compaction for deleted peers to prevent datastore bloat
- Add CRDT health monitoring and automatic repair mechanism (e.g., periodic state sync with bootstrap nodes)

- [ ] **P2P Network Resilience**
- Implement peer scoring and reputation system to prioritize connections with reliable nodes
- [x] Add automatic reconnection logic with exponential backoff for transient network failures
- Implement connection rate limiting to prevent DDoS or misbehaving peers
- Add support for rendezvous servers for NAT traversal in restrictive networks

- [ ] **Bootstrap Discovery Enhancements**
- Implement DNS-based bootstrap discovery (e.g., `_ocf-bootstrap._tcp.example.com`)
- [x] Add support for multiple bootstrap sources (HTTP, DNS, static list) with fallback logic
- Implement bootstrap node validation to prevent malicious bootstrap lists
- Add bootstrap node rotation and load balancing for high availability

## Service Registry & Routing

- [ ] **Service Discovery & Registration**
- Implement service health checks (HTTP/TCP) for registered services to auto-remove unhealthy providers
- Add service metadata (e.g., latency, cost, region) to enable intelligent routing decisions
- Implement service versioning and compatibility checks for API endpoints
- Add support for dynamic service deregistration (e.g., on process exit)

- [ ] **Intelligent Request Routing**
- Implement load balancing algorithms (round-robin, least-connections, weighted) for global service routing
- Add geographic routing based on peer location (IP geolocation or explicit location metadata)
- Implement cost-aware routing (e.g., prioritize free or low-cost providers)
- Add request prioritization and QoS support (e.g., high-priority requests bypass queues)

- [ ] **Identity & Access Control**
- Implement JWT-based authentication for service access (e.g., `model=resnet50` requires valid token)
- Add role-based access control (RBAC) for service registration and routing
- Implement peer identity verification using public key signatures
- Add audit logging for all service access attempts

## Server & API Layer

- [ ] **HTTP API Enhancements**
- Add OpenAPI/Swagger documentation for all API endpoints
- Implement rate limiting and request throttling per peer/IP
- Add pagination and filtering for `/v1/dnt/table` and `/v1/dnt/peers` endpoints
- Implement WebSocket support for real-time event streaming (e.g., peer join/leave)

- [ ] **Observability & Monitoring**
- Add Prometheus metrics endpoint (`/metrics`) for key metrics (peers, requests, latency, errors)
- Implement structured logging with trace IDs for end-to-end request tracing
- Add support for OpenTelemetry integration (beyond Axiom)
- Implement alerting for critical events (e.g., no active peers, high error rate)

- [ ] **Security Hardening**
- Restrict CORS origins to trusted domains instead of `*`
- Implement mutual TLS (mTLS) for P2P communication
- Add request signature validation for all forwarded requests
- Implement input sanitization and validation for all API endpoints

## Entry Point & Configuration

- [ ] **CLI & Configuration Management**
- Add config validation and schema checking (e.g., using JSON Schema)
- Implement config hot-reload without restarting the node
- Add config export/import functionality for backup and migration
- Implement environment variable overrides for all config options

- [ ] **Deployment & Orchestration**
- Create Helm chart for Kubernetes deployment
- Add Docker Compose templates for local development and testing
- Implement systemd service files for production deployment
- Add health check endpoints for container orchestration (e.g., `/healthz`)

## Documentation & Developer Experience

- [ ] **Documentation Updates**
- Update architecture.md with latest routing logic and CRDT details
- Add sequence diagrams for service registration and request routing
- Document all CLI flags and config options in a single reference page
- Add troubleshooting guide for common issues (e.g., connection failures, bootstrap errors)

- [ ] **Developer Tooling**
- Create a CLI tool for debugging and inspecting the local node state
- Add unit test coverage for all core packages (protocol, server, common)
- Implement integration tests for P2P communication and service routing
- Add a local development mode with mock services for testing

## Performance & Scalability

- [ ] **Performance Optimization**
- Profile and optimize CRDT serialization/deserialization performance
- Implement connection pooling for HTTP proxying to reduce overhead
- Add memory usage monitoring and limits to prevent OOM crashes
- Optimize DHT lookups and reduce latency for global service routing

- [ ] **Scalability Improvements**
- Implement sharding of the node table for large networks (>10k peers)
- Add support for hierarchical routing (e.g., regional clusters)
- Implement caching of frequently accessed service records
- Add support for federated networks (multiple independent OCF networks)

## Integration & Ecosystem

- [ ] **Third-Party Integrations**
- Add support for integrating with existing ML platforms (e.g., vLLM, TGI)
- Implement OpenAI-compatible API endpoint for seamless client integration
- Add support for Kubernetes CSI for dynamic resource provisioning
- Integrate with blockchain for decentralized payment and incentive mechanisms

- [ ] **Community & Ecosystem**
- Create a public registry of known bootstrap nodes
- Implement a plugin system for custom service types
- Add a web-based dashboard for monitoring the network
- Create a developer SDK for Go, Python, and JavaScript
Empty file added apps/README.md
Empty file.
6 changes: 6 additions & 0 deletions apps/package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading
Loading