Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
63 changes: 63 additions & 0 deletions .agent/skills/genui-helper/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
---
name: genui-helper
description: >
Development helper for the GenUI repository. Use this skill when the user asks
about GenUI workflows, running tests, creating components, finding A2UI or
Dart references, or adhering to repository standards.
---

# GenUI Development Helper

This skill provides workflows and best practices specific to the `genui` repository.

## Workflows

### 1. Running Tests and Fixes

The repository uses a custom tool to run tests, apply fixes, and format code before committing.
It should typically only be run before committing, since it is inefficient and slow to run it on every change.

It will run `dart fix --apply`, `dart format`, and `flutter test` on all packages in the repository.

**Command:**
```bash
dart run tool/test_and_fix/bin/test_and_fix.dart
```

**When to use:**
- Before committing changes, to ensure project health.
- Instead of running `flutter test` manually for each project in the repo.

### 2. Creating a New Component in the genui package

When creating a new UI component in `genui`:

1. **Location**: Place component files in `packages/genui/lib/src/components/`.
2. **Inheritance**: Components must extend `UiComponent`.
3. **A2UI Compliance**: Ensure the component matches the A2UI specification.
4. **Documentation**: Follow strict Dart documentation standards.

### 3. Updating Documentation

- Documentation source of truth is in `docs/`.
- Use `mkdocs` context if mentioned, but primarily edit the markdown files directly.
- Ensure strict adherence to "Natural Writing" standards (no AI-isms).

## Key Constants & Patterns

- **Current A2UI Version**: v0.9
- **State Management**: Uses `SurfaceController` from `genui`.

## References

- A2UI Specification
- Available in the submodule at @packages/genui/submodules/a2ui
- The specification documentation is available in @packages/genui/submodules/a2ui/specification/v0.9/docs
- The specification schemas are available in @packages/genui/submodules/a2ui/specification/v0.9/json
- Because it is a submodule, you may need to update the submodule to get the latest specification.
- To find out details of a specific dart compiler diagnostic message, use the following url format to look up the details:
- https://dart.dev/tools/diagnostics/<message_id>
- Example: https://dart.dev/tools/diagnostics/ambiguous_import
- To find out details of a specific analyzer lint message, use the following url format to look up the details:
- https://dart.dev/tools/linter-rules/<lint_rule_id>
- Example: https://dart.dev/tools/linter-rules/always_declare_return_types
2 changes: 2 additions & 0 deletions .gemini/GEMINI.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
# Gemini Code Assistant Context

Follow the specifications in `specs/README.md`.

You can find additional skills in @.agent/skills
10 changes: 8 additions & 2 deletions .github/workflows/flutter_packages.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -54,8 +54,14 @@ jobs:

# Get the list of changed files. The method depends on the event type.
if [[ "${{ github.event_name }}" == "pull_request" ]]; then
# For PRs, use the GitHub CLI to get a precise list of changed files.
CHANGED_FILES=$(gh pr diff --name-only ${{ github.event.pull_request.number }})
# Try using gh pr diff first to get the precise list of changed files.
# This avoids false positives if main has moved forward.
# We wrap it in an if statement to catch failures (e.g. diff too large).
if ! CHANGED_FILES=$(gh pr diff --name-only ${{ github.event.pull_request.number }}); then
echo "Warning: 'gh pr diff' failed. This usually happens when the PR is very large."
echo "Falling back to 'git diff' against origin/${{ github.base_ref }}."
CHANGED_FILES=$(git diff --name-only origin/${{ github.base_ref }} HEAD)
fi
else
# For pushes, diff between the two SHAs of the push.
CHANGED_FILES=$(git diff --name-only ${{ github.event.before }} ${{ github.event.after }})
Expand Down
6 changes: 5 additions & 1 deletion .gitmodules
Original file line number Diff line number Diff line change
@@ -1,3 +1,7 @@
[submodule "packages/json_schema_builder/submodules/JSON-Schema-Test-Suite"]
path = packages/json_schema_builder/submodules/JSON-Schema-Test-Suite
url = https://github.com/json-schema-org/JSON-Schema-Test-Suite.git
url = https://github.com/json-schema-org/JSON-Schema-Test-Suite.git
[submodule "a2ui"]
path = packages/genui/submodules/a2ui
url = https://github.com/google/A2UI.git
branch = main
26 changes: 8 additions & 18 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -84,11 +84,9 @@ based on a widget catalog from the developers' project.

## Connecting to an AI agent

The `genui` framework uses a `ContentGenerator` to communicate with a generative AI model,
allowing `genui` to be backend agnostic. You can choose the implementation that best fits
your needs, whether it's `FirebaseAiContentGenerator` for production apps,
`GoogleGenerativeAiContentGenerator` for rapid prototyping, or `A2uiContentGenerator` for
custom agent servers.
The `genui` framework is designed to be backend agnostic. You can use any AI SDK (such as `google_generative_ai`, `dartantic_ai`, or `firebase_vertexai`) to generate content. The framework provides adapters (like `A2uiTransportAdapter`) to ingest the AI response and render it.

For custom agent servers that implement the A2UI protocol, you can use the `genui_a2ui` package.

See the package table below for more details on each.

Expand All @@ -97,10 +95,7 @@ See the package table below for more details on each.
| Package | Description | Version |
| :--- | :--- | :--- |
| [genui](packages/genui/) | The core framework to employ Generative UI. | [![pub package](https://img.shields.io/pub/v/genui.svg)](https://pub.dev/packages/genui) |
| [genui_firebase_ai](packages/genui_firebase_ai/) | Provides **`FirebaseAiContentGenerator`** to connect to Gemini via Firebase AI Logic. This is the recommended approach for production apps based on client-side agents. | [![pub package](https://img.shields.io/pub/v/genui_firebase_ai.svg)](https://pub.dev/packages/genui_firebase_ai) |
| [genui_google_generative_ai](packages/genui_google_generative_ai/) | Provides **`GoogleGenerativeAiContentGenerator`** for connecting to the Google Generative AI API with only an API key. Ideal for getting started quickly. | [![pub package](https://img.shields.io/pub/v/genui_google_generative_ai.svg)](https://pub.dev/packages/genui_google_generative_ai) |
| [genui_a2ui](packages/genui_a2ui/) | Provides **`A2uiContentGenerator`** for connecting to any server that implements the [A2UI protocol](https://a2ui.org). Use this for integrating with custom agent backends. | [![pub package](https://img.shields.io/pub/v/genui_a2ui.svg)](https://pub.dev/packages/genui_a2ui) |
| [genui_dartantic](packages/genui_dartantic/) | Integration package for genui and Dartantic AI. | [![pub package](https://img.shields.io/pub/v/genui_dartantic.svg)](https://pub.dev/packages/genui_dartantic) |
| [genui_a2ui](packages/genui_a2ui/) | Provides **`A2uiAgentConnector`** for connecting to any server that implements the [A2UI protocol](https://a2ui.org). Use this for integrating with custom agent backends. | [![pub package](https://img.shields.io/pub/v/genui_a2ui.svg)](https://pub.dev/packages/genui_a2ui) |
| [genai_primitives](packages/genai_primitives/) | A set of technology-agnostic primitive types and data structures for building Generative AI applications. | [![pub package](https://img.shields.io/pub/v/genai_primitives.svg)](https://pub.dev/packages/genai_primitives) |
| [json_schema_builder](packages/json_schema_builder/) | A fully featured Dart JSON Schema package with validation, used by the core framework to define widget data structures. | [![pub package](https://img.shields.io/pub/v/json_schema_builder.svg)](https://pub.dev/packages/json_schema_builder) |

Expand All @@ -110,24 +105,19 @@ This diagram shows how packages depend on each other and how examples use them.

```mermaid
graph TD
examples/simple_chat --> genui_google_generative_ai
examples/simple_chat --> genui_firebase_ai
examples/travel_app --> genui_google_generative_ai
examples/travel_app --> genui_firebase_ai
examples/catalog_gallery --> genui
examples/simple_chat --> genui
examples/travel_app --> genui
examples/verdure --> genui_a2ui
examples/custom_backend --> genui
genui --> json_schema_builder
genui_a2ui --> genui
genui_dartantic --> genui
genui_firebase_ai --> genui
genui_google_generative_ai --> genui
```

## A2UI Support

The Flutter Gen UI SDK uses the [A2UI protocol](https://a2ui.org) to represent UI content internally. The [genui_a2ui](packages/genui_a2ui/) package allows it to act as a renderer for UIs generated by an A2UI backend agent, similar to the [other A2UI renderers](https://github.com/google/A2UI/tree/main/renderers) which are maintained within the A2UI repository.

The Flutter Gen UI SDK currently supports A2UI v0.8.
The Flutter Gen UI SDK currently supports A2UI v0.9.

## Getting started

Expand Down
1 change: 1 addition & 0 deletions analysis_options.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ analyzer:
linter:
rules:
# consistency
- avoid_print
- combinators_ordering
- directives_ordering
- lines_longer_than_80_chars
Expand Down
78 changes: 78 additions & 0 deletions docs/DESIGN.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,78 @@
# `genui` Package Implementation

This document provides a comprehensive overview of the architecture, purpose, and implementation of the `genui` package.

## Purpose

The `genui` package provides the core framework for building Flutter applications with dynamically generated user interfaces powered by large language models (LLMs). It enables developers to create conversational UIs where the interface is not static or predefined, but is instead constructed by an AI in real-time based on the user's prompts and the flow of the conversation.

The package supplies the essential components for managing the state of the dynamic UI, interacting with the AI model, defining a vocabulary of UI widgets, and rendering the UI surfaces.

## Architecture

The package is designed with a layered architecture, separating concerns to create a flexible and extensible framework. The diagram below shows how the `genui` package integrates with the developer's application and the backend LLM.

![Class Diagram](./assets/class-diagram.svg)

### 1. Transport Layer (`lib/src/transport/` and `lib/src/interfaces/`)

This layer handles the pipeline from raw text input (from an LLM) to parsed UI events.

- **`Transport`**: An interface defining the contract for sending and receiving messages.
- **`A2uiTransportAdapter`**: The default implementation of `Transport`. It manages the input stream (`addChunk`), the parsing pipeline, and communicates with the `Conversation`. It uses the `A2uiParserTransformer` to parse streams.
- **`A2uiParserTransformer`**: A robust stream transformer that parses mixed streams of text and A2UI JSON messages. It handles buffering, validation, and conversion of raw strings into structured `GenerationEvent`s.

### 2. UI State Management Layer (`lib/src/engine/`)

This is the central nervous system of the package, orchestrating the state of all generated UI surfaces.

- **`SurfaceController`**: The core state manager for the dynamic UI. It maintains a map of all active UI "surfaces", where each surface is represented by a `UiDefinition`. The AI interacts with the manager by sending structured A2UI messages, which the controller handles via `handleMessage()`. It exposes a stream of `SurfaceUpdate` events (`SurfaceAdded`, `ComponentsUpdated`, `SurfaceRemoved`) so that the application can react to changes. It also owns the `DataModel` to manage the state of individual widgets and implements `SurfaceHost` to provide `SurfaceContext`s for `Surface` widgets.

### 3. UI Model Layer (`lib/src/model/`)

This layer defines the data structures that represent the dynamic UI and the conversation.

- **`Catalog` and `CatalogItem`**: These classes define the registry of available UI components. The `Catalog` holds a list of `CatalogItem`s, and each `CatalogItem` defines a widget's name, its data schema, and a builder function to render it.
- **`A2uiMessage`**: A sealed class (`lib/src/model/a2ui_message.dart`) representing the commands the AI sends to the UI. It has the following subtypes:
- `CreateSurface`: Signals the start of rendering for a surface.
- `UpdateComponents`: Adds or updates components on a surface.
- `UpdateDataModel`: Modifies data within the `DataModel` for a surface.
- `DeleteSurface`: Requests the removal of a surface.
The schemas for these messages are defined in `lib/src/model/a2ui_schemas.dart`.
- **`UiDefinition` and `UiEvent`**: `UiDefinition` represents the state of a surface, including the definitions of all components on the surface. `UiEvent` is a data object representing a user interaction. `UserActionEvent` is a subtype used for events that should trigger a submission to the AI, like a button tap.
- **`ChatMessage`**: A sealed class representing the different types of messages in a conversation: `UserMessage`, `AiTextMessage`, `ToolResponseMessage`, `AiUiMessage`, `InternalMessage`, and `UserUiInteractionMessage`.
- **`DataModel` and `DataContext`**: The `DataModel` is a centralized, observable key-value store that holds the entire dynamic state of the UI. Widgets receive a `DataContext`, which is a view into the `DataModel` that understands the widget's current scope. This allows widgets to subscribe to changes in the data model and rebuild reactively. This separation of data and UI structure is a core principle of the architecture.

### 4. Widget Catalog Layer (`lib/src/catalog/`)

This layer provides a set of core, general-purpose UI widgets that can be used out-of-the-box.

- **`basic_catalog.dart`**: Defines the `BasicCatalogItems`, which includes fundamental widgets like `AudioPlayer`, `Button`, `Card`, `CheckBox`, `Column`, `DateTimeInput`, `Divider`, `Icon`, `Image`, `List`, `Modal`, `MultipleChoice`, `Row`, `Slider`, `Tabs`, `Text`, `TextField`, and `Video`.
- **Widget Implementation**: Each core widget follows the standard `CatalogItem` pattern: a schema definition, a type-safe data accessor using an `extension type`, the `CatalogItem` instance, and the Flutter widget implementation.

### 5. UI Facade Layer (`lib/src/conversation/`)

This layer provides high-level widgets and controllers for easily building a generative UI application.

- **`Conversation`**: The primary entry point for the package. This facade class encapsulates the `SurfaceController` (engine) and the `Transport`. It manages the conversation loop, piping messages between the transport and the engine.
- **`Surface`**: The Flutter widget responsible for recursively building a UI tree from a `UiDefinition`. It listens for updates from a `SurfaceContext` (typically obtained from a `SurfaceHost` like `SurfaceController`) and rebuilds itself when the definition changes.

### 6. Primitives Layer (`lib/src/primitives/`)

This layer contains basic utilities used throughout the package.

- **`logging.dart`**: Provides a configurable logger (`genUiLogger`).
- **`simple_items.dart`**: Defines a type alias for `JsonMap`.

### 7. Direct Call Integration (`lib/src/facade/direct_call_integration/`)

This directory provides utilities for a more direct interaction with the AI model, potentially bypassing some of the higher-level abstractions of `Conversation`. It includes:

- **`model.dart`**: Defines data models for direct API calls.
- **`utils.dart`**: Contains utility functions to assist with direct calls.

## How It Works: The Generative UI Cycle

The `Conversation` simplifies the process of creating a generative UI by managing the conversation loop and the interaction with the AI.

![Architecture](./assets/architecture.svg)
Comment on lines +1 to +78
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

This file appears to be a subset of docs/Design_Proposal.md. To avoid redundancy and maintain a single source of truth for the design, consider removing this file and promoting Design_Proposal.md (perhaps by renaming it to DESIGN.md) as the canonical design document.

Loading
Loading