-
Notifications
You must be signed in to change notification settings - Fork 125
Add Migration Plan #721
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Draft
gspencergoog
wants to merge
9
commits into
main
Choose a base branch
from
feature/v0.9-migration
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Draft
Add Migration Plan #721
Changes from all commits
Commits
Show all changes
9 commits
Select commit
Hold shift + click to select a range
a6b5627
chore: remove legacy packages and examples
gspencergoog 16e239a
infra: update test_and_fix and json_schema_builder
gspencergoog a36fab8
feat(genui): update core framework to v0.9
gspencergoog 42cfb97
feat(a2ui): update integration package
gspencergoog 0131056
feat(examples): update examples for v0.9
gspencergoog 866d779
docs: update documentation and configuration
gspencergoog 43e5a35
Review changes
gspencergoog f396683
More review changes
gspencergoog 37ae361
Update reviewing guide
gspencergoog File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,63 @@ | ||
| --- | ||
| name: genui-helper | ||
| description: > | ||
| Development helper for the GenUI repository. Use this skill when the user asks | ||
| about GenUI workflows, running tests, creating components, finding A2UI or | ||
| Dart references, or adhering to repository standards. | ||
| --- | ||
|
|
||
| # GenUI Development Helper | ||
|
|
||
| This skill provides workflows and best practices specific to the `genui` repository. | ||
|
|
||
| ## Workflows | ||
|
|
||
| ### 1. Running Tests and Fixes | ||
|
|
||
| The repository uses a custom tool to run tests, apply fixes, and format code before committing. | ||
| It should typically only be run before committing, since it is inefficient and slow to run it on every change. | ||
|
|
||
| It will run `dart fix --apply`, `dart format`, and `flutter test` on all packages in the repository. | ||
|
|
||
| **Command:** | ||
| ```bash | ||
| dart run tool/test_and_fix/bin/test_and_fix.dart | ||
| ``` | ||
|
|
||
| **When to use:** | ||
| - Before committing changes, to ensure project health. | ||
| - Instead of running `flutter test` manually for each project in the repo. | ||
|
|
||
| ### 2. Creating a New Component in the genui package | ||
|
|
||
| When creating a new UI component in `genui`: | ||
|
|
||
| 1. **Location**: Place component files in `packages/genui/lib/src/components/`. | ||
| 2. **Inheritance**: Components must extend `UiComponent`. | ||
| 3. **A2UI Compliance**: Ensure the component matches the A2UI specification. | ||
| 4. **Documentation**: Follow strict Dart documentation standards. | ||
|
|
||
| ### 3. Updating Documentation | ||
|
|
||
| - Documentation source of truth is in `docs/`. | ||
| - Use `mkdocs` context if mentioned, but primarily edit the markdown files directly. | ||
| - Ensure strict adherence to "Natural Writing" standards (no AI-isms). | ||
|
|
||
| ## Key Constants & Patterns | ||
|
|
||
| - **Current A2UI Version**: v0.9 | ||
| - **State Management**: Uses `SurfaceController` from `genui`. | ||
|
|
||
| ## References | ||
|
|
||
| - A2UI Specification | ||
| - Available in the submodule at @packages/genui/submodules/a2ui | ||
| - The specification documentation is available in @packages/genui/submodules/a2ui/specification/v0.9/docs | ||
| - The specification schemas are available in @packages/genui/submodules/a2ui/specification/v0.9/json | ||
| - Because it is a submodule, you may need to update the submodule to get the latest specification. | ||
| - To find out details of a specific dart compiler diagnostic message, use the following url format to look up the details: | ||
| - https://dart.dev/tools/diagnostics/<message_id> | ||
| - Example: https://dart.dev/tools/diagnostics/ambiguous_import | ||
| - To find out details of a specific analyzer lint message, use the following url format to look up the details: | ||
| - https://dart.dev/tools/linter-rules/<lint_rule_id> | ||
| - Example: https://dart.dev/tools/linter-rules/always_declare_return_types |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,3 +1,5 @@ | ||
| # Gemini Code Assistant Context | ||
|
|
||
| Follow the specifications in `specs/README.md`. | ||
|
|
||
| You can find additional skills in @.agent/skills |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,3 +1,7 @@ | ||
| [submodule "packages/json_schema_builder/submodules/JSON-Schema-Test-Suite"] | ||
| path = packages/json_schema_builder/submodules/JSON-Schema-Test-Suite | ||
| url = https://github.com/json-schema-org/JSON-Schema-Test-Suite.git | ||
| url = https://github.com/json-schema-org/JSON-Schema-Test-Suite.git | ||
| [submodule "a2ui"] | ||
| path = packages/genui/submodules/a2ui | ||
| url = https://github.com/google/A2UI.git | ||
| branch = main |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,78 @@ | ||
| # `genui` Package Implementation | ||
|
|
||
| This document provides a comprehensive overview of the architecture, purpose, and implementation of the `genui` package. | ||
|
|
||
| ## Purpose | ||
|
|
||
| The `genui` package provides the core framework for building Flutter applications with dynamically generated user interfaces powered by large language models (LLMs). It enables developers to create conversational UIs where the interface is not static or predefined, but is instead constructed by an AI in real-time based on the user's prompts and the flow of the conversation. | ||
|
|
||
| The package supplies the essential components for managing the state of the dynamic UI, interacting with the AI model, defining a vocabulary of UI widgets, and rendering the UI surfaces. | ||
|
|
||
| ## Architecture | ||
|
|
||
| The package is designed with a layered architecture, separating concerns to create a flexible and extensible framework. The diagram below shows how the `genui` package integrates with the developer's application and the backend LLM. | ||
|
|
||
|  | ||
|
|
||
| ### 1. Transport Layer (`lib/src/transport/` and `lib/src/interfaces/`) | ||
|
|
||
| This layer handles the pipeline from raw text input (from an LLM) to parsed UI events. | ||
|
|
||
| - **`Transport`**: An interface defining the contract for sending and receiving messages. | ||
| - **`A2uiTransportAdapter`**: The default implementation of `Transport`. It manages the input stream (`addChunk`), the parsing pipeline, and communicates with the `Conversation`. It uses the `A2uiParserTransformer` to parse streams. | ||
| - **`A2uiParserTransformer`**: A robust stream transformer that parses mixed streams of text and A2UI JSON messages. It handles buffering, validation, and conversion of raw strings into structured `GenerationEvent`s. | ||
|
|
||
| ### 2. UI State Management Layer (`lib/src/engine/`) | ||
|
|
||
| This is the central nervous system of the package, orchestrating the state of all generated UI surfaces. | ||
|
|
||
| - **`SurfaceController`**: The core state manager for the dynamic UI. It maintains a map of all active UI "surfaces", where each surface is represented by a `UiDefinition`. The AI interacts with the manager by sending structured A2UI messages, which the controller handles via `handleMessage()`. It exposes a stream of `SurfaceUpdate` events (`SurfaceAdded`, `ComponentsUpdated`, `SurfaceRemoved`) so that the application can react to changes. It also owns the `DataModel` to manage the state of individual widgets and implements `SurfaceHost` to provide `SurfaceContext`s for `Surface` widgets. | ||
|
|
||
| ### 3. UI Model Layer (`lib/src/model/`) | ||
|
|
||
| This layer defines the data structures that represent the dynamic UI and the conversation. | ||
|
|
||
| - **`Catalog` and `CatalogItem`**: These classes define the registry of available UI components. The `Catalog` holds a list of `CatalogItem`s, and each `CatalogItem` defines a widget's name, its data schema, and a builder function to render it. | ||
| - **`A2uiMessage`**: A sealed class (`lib/src/model/a2ui_message.dart`) representing the commands the AI sends to the UI. It has the following subtypes: | ||
| - `CreateSurface`: Signals the start of rendering for a surface. | ||
| - `UpdateComponents`: Adds or updates components on a surface. | ||
| - `UpdateDataModel`: Modifies data within the `DataModel` for a surface. | ||
| - `DeleteSurface`: Requests the removal of a surface. | ||
| The schemas for these messages are defined in `lib/src/model/a2ui_schemas.dart`. | ||
| - **`UiDefinition` and `UiEvent`**: `UiDefinition` represents the state of a surface, including the definitions of all components on the surface. `UiEvent` is a data object representing a user interaction. `UserActionEvent` is a subtype used for events that should trigger a submission to the AI, like a button tap. | ||
| - **`ChatMessage`**: A sealed class representing the different types of messages in a conversation: `UserMessage`, `AiTextMessage`, `ToolResponseMessage`, `AiUiMessage`, `InternalMessage`, and `UserUiInteractionMessage`. | ||
| - **`DataModel` and `DataContext`**: The `DataModel` is a centralized, observable key-value store that holds the entire dynamic state of the UI. Widgets receive a `DataContext`, which is a view into the `DataModel` that understands the widget's current scope. This allows widgets to subscribe to changes in the data model and rebuild reactively. This separation of data and UI structure is a core principle of the architecture. | ||
|
|
||
| ### 4. Widget Catalog Layer (`lib/src/catalog/`) | ||
|
|
||
| This layer provides a set of core, general-purpose UI widgets that can be used out-of-the-box. | ||
|
|
||
| - **`basic_catalog.dart`**: Defines the `BasicCatalogItems`, which includes fundamental widgets like `AudioPlayer`, `Button`, `Card`, `CheckBox`, `Column`, `DateTimeInput`, `Divider`, `Icon`, `Image`, `List`, `Modal`, `MultipleChoice`, `Row`, `Slider`, `Tabs`, `Text`, `TextField`, and `Video`. | ||
| - **Widget Implementation**: Each core widget follows the standard `CatalogItem` pattern: a schema definition, a type-safe data accessor using an `extension type`, the `CatalogItem` instance, and the Flutter widget implementation. | ||
|
|
||
| ### 5. UI Facade Layer (`lib/src/conversation/`) | ||
|
|
||
| This layer provides high-level widgets and controllers for easily building a generative UI application. | ||
|
|
||
| - **`Conversation`**: The primary entry point for the package. This facade class encapsulates the `SurfaceController` (engine) and the `Transport`. It manages the conversation loop, piping messages between the transport and the engine. | ||
| - **`Surface`**: The Flutter widget responsible for recursively building a UI tree from a `UiDefinition`. It listens for updates from a `SurfaceContext` (typically obtained from a `SurfaceHost` like `SurfaceController`) and rebuilds itself when the definition changes. | ||
|
|
||
| ### 6. Primitives Layer (`lib/src/primitives/`) | ||
|
|
||
| This layer contains basic utilities used throughout the package. | ||
|
|
||
| - **`logging.dart`**: Provides a configurable logger (`genUiLogger`). | ||
| - **`simple_items.dart`**: Defines a type alias for `JsonMap`. | ||
|
|
||
| ### 7. Direct Call Integration (`lib/src/facade/direct_call_integration/`) | ||
|
|
||
| This directory provides utilities for a more direct interaction with the AI model, potentially bypassing some of the higher-level abstractions of `Conversation`. It includes: | ||
|
|
||
| - **`model.dart`**: Defines data models for direct API calls. | ||
| - **`utils.dart`**: Contains utility functions to assist with direct calls. | ||
|
|
||
| ## How It Works: The Generative UI Cycle | ||
|
|
||
| The `Conversation` simplifies the process of creating a generative UI by managing the conversation loop and the interaction with the AI. | ||
|
|
||
|  | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This file appears to be a subset of
docs/Design_Proposal.md. To avoid redundancy and maintain a single source of truth for the design, consider removing this file and promotingDesign_Proposal.md(perhaps by renaming it toDESIGN.md) as the canonical design document.