Release v0.3.11

🚀 Production Release v0.3.11

⚠️ Currently using testnet configuration

Platform:

• Linux x86_64

Components:

• Worker
• Validator
• Orchestrator
• Discovery service

SHA256 checksums are provided for each binary.

Installation

Download the appropriate binary for your system and verify the checksum.

# Verify checksum (example for worker)
sha256sum -c worker-linux-x86_64.checksum

Changes

See CHANGELOG.md for detailed changes in this release.

What's Changed

• Add checks and base cargo file by @JannikSt in #2
• [pri-850] Setup basic system checks in miner by @JannikSt in #1
• Ability to control Docker runtime pri-852 by @JannikSt in #3
• Add check for nvidia container toolkit pri-894 by @JannikSt in #4
• [PRI-904, PRI-875, PRI-921, PRI-876, PRI-851] Setup Miner, Validator, Orchestrator, Discovery Service by @JannikSt in #7
• solve #12 - support docker start command by @mattdf in #40
• attempt to fix tests in toolchain by @JannikSt in #41
• solve #39 - properly handle invalid commands + restarts for containers by @mattdf in #43
• add extra function in loop to sync node state to chain by @mattdf in #45
• #32 Feature/dynamic address support middleware by @JannikSt in #42
• fix alloy-rs overloaded functions error by @mattdf in #49
• Rust Discovery Service by @JannikSt in #47
• implement file based auto-recovery by @JannikSt in #53
• Fix/discovery overwrite by @JannikSt in #56
• add dead -> discovered state transition and test by @mattdf in #52
• Add EP to fetch node status from discovery svc by subkey by @manveerxyz in #57
• Feature/gpu support by @JannikSt in #59
• Fix/include abi in bin by @JannikSt in #69
• adjust miner and validator cli cmds by @JannikSt in #70
• fix: change orchestrator host ip by @JannikSt in #71
• Feature/e2e scripts by @JannikSt in #44
• support latest release folder in s3 bucket by @JannikSt in #74
• Improvement/general improvements by @JannikSt in #73
• fix foundryup command readme by @samsja in #77
• also allow compute manager to access pool nodes by @JannikSt in #79
• add debug logs for discovery error by @JannikSt in #80
• activate persistence on chain by @JannikSt in #78
• Feature/taskbridge by @JannikSt in #75
• Storage auto detection & docker storage mounts by @JannikSt in #81
• add error handling and logs to orchestrator discovery sync by @JannikSt in #82
• Improvement/check storage writable by @JannikSt in #83
• improvement: allow running other redis instances in parallel by @JannikSt in #87
• Fix/task bridge metrics parsing by @JannikSt in #85
• use bash as default shell for make by @samsja in #88
• add instruction to the readme by @samsja in #89
• add redis server install to readme by @samsja in #90
• Improvement/orchestrator invite failure handling by @JannikSt in #91
• ensure docker service does not delete task before heartbeat is sent by @JannikSt in #92
• add missing api_key_mw by @JannikSt in #93
• Feature/container utils by @JannikSt in #94
• automatically build discovery packages by @JannikSt in #95
• Improvement/allow setting orchestrator url by @JannikSt in #97
• Feature/validator health api by @JannikSt in #98
• Improvement/prod release setup by @JannikSt in #100
• Release v.0.1 by @JannikSt in #101
• release v.0.1.1 by @JannikSt in #103
• Feature/orchestrator metrics api by @JannikSt in #104
• hardcoded h200 compute units by @JannikSt in #105
• Release v.0.1.2 by @JannikSt in #106
• Release v.0.1.3 with shm size adjustment by @JannikSt in #109
• Release v.0.1.3 by @JannikSt in #110
• release v.0.1.4 by @JannikSt in #113
• Release v.1.0.5 by @JannikSt in #115
• Release 0.1.6 by @JannikSt in #121
• Release v.0.1.7 by @JannikSt in #126
• Release worker v. 0.2.0 by @JannikSt in #210
• Release v.0.2.1 by @JannikSt in #264
• Release v. 0.2.2 by @JannikSt in #268
• Release issue template by @JannikSt in #271
• bump version by @JannikSt in #274
• Release: v0.2.4 by @JannikSt in #279
• Add software check for miner port by @outwrit in #240
• Refactor: use shared dependencies across crates by @gsegatti in #153
• improvement: exp backoff for container on failure by @spaghettic0de in #189
• Add compute node deregistration and stake reclamation by @marcogiglio in #255
• Release: v.0.2.5 by @JannikSt in #285
• release v.0.2.5 by @JannikSt in #286
• add check if pool is active by @JannikSt in #287
• Chore: move code into crates by @JannikSt in #288
• temp adjust discovery restrictions with new gpu setup by @JannikSt in #289
• release v.0.2.6 by @JannikSt in #290
• increase gas allowance to handle invalid estimate in submit_work by @mattdf in #291
• bump version by @JannikSt in #292
• release v.0.2.7 by @JannikSt in #293
• Fix/temp health disable by @JannikSt in #296
• Release/v0.2.8 by @JannikSt in #300
• remove discovered nodes after 1 hour by @JannikSt in #301
• Release v.0.2.9 by @JannikSt in #302
• Fix: Discovery re-upload when active by @JannikSt in #305
• bump version to 0.2.10 by @JannikSt in #306
• Feature: discovery updater by @JannikSt in #308
• Chore: adjust installation urls by @JannikSt in #309
• Improvement: concurrent node invitation by @gsegatti in #294
• Feature: stun based ip detection by @JannikSt in #317
• Chore: Bump alloy version by @JannikSt in #320
• Fix: underpriced tx handling & tx retries by @JannikSt in #318
• Orchestrator: Keep old tasks in store by @JannikSt in #321
• introduce server modes to improve scalability of svc by @JannikSt in #322
• Feature: orchestrator scheduler by @JannikSt in #323
• Feature: status update plugins by @JannikSt in #324
• Feature: Node topology plugin by @JannikSt in #325
• align smart contracts by @JannikSt in #326
• Fix: multiple group assignments by @JannikSt in #327
• rebuild groups when one node dies by @JannikSt in #328
• add group id to variable substitution by @JannikSt in #331
• Release: 0.2.11 by @JannikSt in #310
• Improvement: Ensure a group is only working on a single task by @JannikSt in #333
• allow to access resource specs directly from orchestrator by @JannikSt in #334
• Feature: Node group plugin with configurable topologies by @JannikSt in #335
• Feature: Task Scheduling configuration during task creation by @JannikSt in #337
• Improvement/plugin folder structure by @JannikSt in #339
• Feature: task storage configuration by @JannikSt in #341
• Support multiple toploc servers in synthetic-data validation by @JannikSt in #342
• Feature: Group size in storage variables by @JannikSt in #343
• adjust network config in docker-compose by @JannikSt in #344
• Improvement: adjust node config loading by @JannikSt in #346
• Fix: compute requirement parsing creates multiple subojects in gpu array by @JannikSt in #347
• Worker p2p seed replacement in task args by @JannikSt in #349
• fix for showing tmuxinator pane titles in newest version by @JohannesHa in #348
• introduce a simple filecounter based on upload requests by @JannikSt in #351
• Fix: k8s deployment files by @JannikSt in #350
• Chore: align interfaces by @JannikSt in #352
• Toploc Integration for p2p workers by @JannikSt in #353
• fix validator docker image adding boolean values to args by @JannikSt in #355
• add more debug logging for toploc routing by @JannikSt in #356
• add current upload count to task env var by @JannikSt in #357
• fix: wrong invalidation flag by @JannikSt in #358
• introduce file number variable by @JannikSt in #359
• add gpu:memory_min, gpu:memory_max to filter for gpu memory ranges by @JannikSt in #360
• Refactor storage & scheduling variables by @JannikSt in #361
• Improvement: Dynamic node group creation based on task requirements by @JannikSt in #362
• Fix/k8s helm health by @JannikSt in #363
• only create tags and not releases during dev flow by @JannikSt in #364
• move setup to tmuxinator project start command to remove sleeps by @JohannesHa in #365
• replace anvil with reth devchain by @mattdf in #332
• Feature: Validator prometheus metrics by @JannikSt in #366
• Feature: export node task metrics to prometheus by @JannikSt in #367
• chore: adjust taskbridge to simple new logfmt by @JannikSt in #368
• Feature: Advanced Webhooks by @JannikSt in #369
• add nonces and timeouts to pool invite signatures by @mattdf in #354
• activate work unit submission by @JannikSt in #370
• disalbe webhook calls when mode is api only by @JannikSt in #371
• improvmeent: exit on webhook or node configs parsing error by @JannikSt in #372
• upload assets to s3 to ensure we do not spam users with release infos by @JannikSt in #375
• Fix/webhook env parsing by @JannikSt in #374
• adjust helm deployments to make monitoring with prometheus easier by @JannikSt in #378
• align toploc api with recent adjustments by @JannikSt in #377
• Send pool id with metrics updated webhook by @manveerxyz in #373
• fix: better metrics changed comaprison by @JannikSt in #379
• measure amount of keys we have to process in time interval by @JannikSt in #380
• exclude dead nodes in /nodes api by @JannikSt in #381
• fix: Docker status code handling by @JannikSt in #383
• Fix/toploc routing w similar model by @JannikSt in #386
• set appropriate permissions for docker shared folders by @JannikSt in #388
• add groups api with group log functionality by @JannikSt in #387
• add test to workkey logging on validator by @JannikSt in #382
• ability to filter by total gpu memory on machine by @JannikSt in #389
• Adjust socket path permissions for system wide writes by @JannikSt in #390
• Fix/gpu requirement count by @JannikSt in #391
• Imp/add toploc name to prometheus by @JannikSt in #392
• remove print statements by @JannikSt in #394
• add batch trigger size to validator by @JannikSt in #395
• improve stale metrics handling by @JannikSt in #393
• fix validator status check aborts too early by @JannikSt in #396
• fix validator group formation by @JannikSt in #397
• add log when toploc returns unknown status by @JannikSt in #398
• major orchestrator async refactor by @JannikSt in #399
• Group Work Unit checks by @JannikSt in #401
• adjust file submission and retry logic by @JannikSt in #403
• add issue template for improvement / feature by @JannikSt in #406
• improve validator e2e tests by @JannikSt in #405
• refactor: use generic provider for contracts; separate read and write functionality by @noot in #404
• fix: potential panics in routes, avoid keys cmd by @JannikSt in #423
• Fix: stale metrics in distributed deployment of orchestrator by @JannikSt in #425
• add additional prometheus operator metrics, add taskname to node metrics by @JannikSt in #428
• fix metrics api not available on processor by @JannikSt in #429
• fix: only validate work unit submissions on toploc accept by @JannikSt in #430
• chore: add service to processor in helm chart by @JannikSt in #431
• feat(orchestrator): add first_seen timestamp to track node discovery … by @JohannesHa in #427
• fix: syncing nodes with same ip and port by @JannikSt in #432
• terminate worker when no validator addresses are added to middleware by @JannikSt in #433
• add missing toploc timeout by @JannikSt in #434
• add additional logging to understand why hardware challenge fails by @JannikSt in #435
• fix(orchestrator): high latency of groups api by @JannikSt in #437
• make discovery service async, optimize redis performance by @JannikSt in #440
• fix: always pull Docker images with :latest and :main tags by @JohannesHa in #439
• P2P communication layer by @JannikSt in #442
• Allow to add metadata to task & various optimizations by @JannikSt in #445
• fix install script on dev by @JannikSt in #447
• Fix/dev install script by @JannikSt in #448
• add discovery to p2p setup, fix README by @JannikSt in #450
• add explicit logging when fw blocks discovery upload by @JannikSt in #451
• adjust prime socket path naming by @JannikSt in #456
• imp(orchestrator): replace default variables node_address, task_id on orchestrator by @JannikSt in #457
• imp(orchestrator): atomic operation on node group removal, ability to delete groups manually by @JannikSt in #454
• imp(orchestrator): improve task creation api by @JannikSt in #459
• feat(orchestrator): customize volume mounts via task by @JannikSt in #458
• fix tests after docker cmd change by @JannikSt in #462
• Include exact version tag in dev-release by @JannikSt in #466
• fix dev release sequence by @JannikSt in #468
• introduce task container, cleanup task volume handling by @JannikSt in #465
• feat(worker): ability to manually set storage path by @JannikSt in #460
• add ability to customize the socket path env var by @JannikSt in #467
• fix(worker): seed replacement accidentally removed by @JannikSt in #471
• sort groups in groups api by @JannikSt in #472
• imp(orchestrator): p2p relay setup by @JannikSt in #474
• feat(validator): group timeout when work submissions missing by @JannikSt in #473
• align smart contracts repo by @JannikSt in #476
• imp(worker): log heartbeat error response by @JannikSt in #477
• chore(worker): cleanup log message content around contract balance by @JannikSt in #478
• fix(bug): fix staking console log by @JannikSt in #480
• imp(orchestrator): automatically try to increase group size from single nodes by @JannikSt in #479
• imp(orchestrator): Avoid task with same name, sort task API return by @JannikSt in #482
• imp(orchestrator): log worker task states to prometheus by @JannikSt in #484
• fix(worker): avoid worker cleanup on failure restart, remove exp. backoff by @JannikSt in #483
• imp(worker): implement p2p service auto recover on failure by @JannikSt in #487
• fix race condition when reading large amounts of data in iroh by @JannikSt in #488
• imp(validator) validator redis performance by @JannikSt in #490
• fix(worker): circular docker volume dependency when restarting worker by @JannikSt in #492
• imp(validator): handle filename resolution failure on validator by @JannikSt in #493
• fix(validator): filter out work keys where group resolution failed by @JannikSt in #495
• allow to set threshold of same ips per discovery node by @JannikSt in #497
• imp: add nonce to auth middleware by @JannikSt in #489
• feat: allow loading data from multiple disc svc by @JannikSt in #491
• Feat(validator): rejections api & toploc reason loading by @JannikSt in #486
• fix(discovery): url in dockerfiles by @JannikSt in #499
• fix(worker): upload request missing nonce by @JannikSt in #500
• fix service name in helm chart by @JannikSt in #505
• adjust validator chart to simplify toploc mgmt by @JannikSt in #506
• fix(worker): do not submit work when work units are 0 by @JannikSt in #504
• feat: support multiple discovery pods by @JannikSt in #508
• chore(validator): disable old validator migration by @JannikSt in #509
• imp(worker) container restart on failure by @JannikSt in #510
• fix(discovery): enable missing discovery hb endpoint by @JannikSt in #511
• update contract addresses and ABIs by @mattdf in #513
• fix(worker): Use correct version for prime worker --version on develop builds by @lgingerich in #512
• imp(orchestrator): webhook resiliency by @JannikSt in #515
• imp: Orchestrator Details about docker status on machine by @JannikSt in #514
• feat(orchestrator): optimize redis get nodes performance by @JannikSt in #516
• chore(orchestrator): disable orchestrator nodes migration by @JannikSt in #517
• bump version by @JannikSt in #518
• chore(deployment): ability to customize redis in helm charts by @JannikSt in #519
• feat(orchestrator): ability to delete all tasks from orchestrator by @JannikSt in #522
• fix(validator): Missing reason on work units mismatch by @JannikSt in #521
• feat(orchestrator): ability to sync location via discovery, location based group building by @JannikSt in #523
• imp(orchestrator): sync locations from discovery by @JannikSt in #524
• fix(auth): use subtle for constant time key comparison, fix lower case key issue by @JannikSt in #526
• imp(orchestrator): optimize storage upload routes by @JannikSt in #528
• imp(worker): add log why provider registration fails by @JannikSt in #532
• feat(orchestrator): swagger docs by @JannikSt in #531
• imp(worker): use build based default rpc for prod build by @JannikSt in #533
• bump version by @JannikSt in #534
• refactor helm charts by @pokgak in #527
• fix helm chart by @pokgak in #537
• charts: add quotes by @pokgak in #538
• charts/validator: fix conditions for redis by @pokgak in #539
• fix validator redis chart by @pokgak in #540
• bump version by @JannikSt in #541
• chore: remove deprecated e2e folder by @JannikSt in #543
• fix: validator dockerfile not triggering invalidations by @JannikSt in #536
• imp(validator): ability to pick invalidation type for rejection by @JannikSt in #542
• imp(orchestrator): optimize heartbeat & metrics storage by @JannikSt in #545
• imp(orchestrator): optimize orchestrator nodes / groups performance by @JannikSt in #546
• bump version by @JannikSt in #548
• bump version by @JannikSt in #550
• fix(validator): add missing toploc invalidation type by @JannikSt in #551
• imp(orchestrator): restrict redis migrations to processor by @JannikSt in #552
• improve node metrics migration by @JannikSt in #553
• validator: fix fullname template by @pokgak in #554
• chore: remove migrations by @JannikSt in #555
• bump version to 0.3.3 by @JannikSt in #556
• bump version to 0.3.3 by @JannikSt in #559
• bump release by @JannikSt in #558
• imp(orchestrator): geolocation plugin also on group merges by @JannikSt in #561
• imp(orchestrator): cleanup mutex handling to proper async by @JannikSt in #530
• charts: add pdb to api by @pokgak in #560
• imp(orchestrator): node store by @JannikSt in #562
• imp(orchestrator): node store migration by @JannikSt in #563
• fix node store counting by @JannikSt in #565
• bump version by @JannikSt in #567
• chore(orchestrator): disable node store migration by @JannikSt in #568
• bump version by @JannikSt in #570
• imp(orchestrator): add additional status update loop metrics by @JannikSt in #575
• imp(orchestrator): increase heartbeat timeout by @JannikSt in #574
• imp(validator): switch to sequential validation on valdiator instance by @JannikSt in #576
• imp(worker): host nw mode with ability to switch networking config by @JannikSt in #577
• imp(general): Orchestrator Discovery Sync, Worker Timeouts by @JannikSt in #578
• fix(orchestrator): task name list not cleared by @JannikSt in #582
• release 0.3.7 by @JannikSt in #580
• fix(validator): check if work info returned by contract is zero by @JannikSt in #581
• release 0.3.7 by @JannikSt in #584
• fix: autofix clippy uninlined_format_args by @noot in #572
• fix(worker): use dynamic nvml lib init instead of hardcoding path by @mattdf in #586
• bump version by @JannikSt in #587
• feat(worker): add WORKER_VISIBLE_DEVICES env var to select worker devices by @mattdf in #589
• refactor: remove unused async from functions by @noot in #571
• chore: dev setup by @JannikSt in #592
• chore: terminate tmuxinator to improve dev exp. by @JannikSt in #593
• chore: add more details to ubuntu by @JannikSt in #594
• imp: development setup doc by @JannikSt in #595
• release: 0.3.9 by @JannikSt in #598
• fix: increase pending transaction timeout by @noot in #603
• temp disable orchestrator active check by @JannikSt in #606
• bump version by @JannikSt in #608
• fix(worker): cleanup eth formatting by @JannikSt in #503
• fix clippy match_same_arms warnings by @noot in #599
• fix: change default task bridge socket file to be in home dir by @noot in #605
• update smart-contracts to latest by @noot in #610
• refactor task bridge to handle async tasks concurrently, preventing potential duplicate submission by @noot in #600
• fix unreachable_pub for discovery by @noot in #615
• fix unreachable_pub for orchestrator by @noot in #614
• fix unreachable_pub for validator crate by @noot in #613
• fix manual_let_else clippy errors by @noot in #612
• fix unreachable_pub in worker crate by @noot in #611
• feature: implement ipfs upload in worker by @noot in #609
• feat: exclude nodes with zero balance from node_groups by @noot in #573
• refactor: refactor NodeStatusUpdater::process_nodes to run concurrently by @noot in #585
• adjust release flow by @JannikSt in #618

New Contributors

• @outwrit made their first contribution in #240
• @gsegatti made their first contribution in #153
• @marcogiglio made their first contribution in #255
• @noot made their first contribution in #404
• @lgingerich made their first contribution in #512
• @pokgak made their first contribution in #527

Full Changelog: https://github.com/PrimeIntellect-ai/protocol/commits/v0.3.11