Skip to content

Split epics make record#319

Open
ajgdls wants to merge 10 commits intomainfrom
split-epics-make-record
Open

Split epics make record#319
ajgdls wants to merge 10 commits intomainfrom
split-epics-make-record

Conversation

@ajgdls
Copy link
Contributor

@ajgdls ajgdls commented Feb 10, 2026

This resolves #288.
The first commit removes the original builder_callable_from_attribute function from the util file in ca transport and instead calls the record creation directly in the ioc file.
The second commit then splits the _make_record function into _make_in_record and _make_out_record.
Tests updated at each commit.

Summary by CodeRabbit

  • Refactor

    • Split record creation into separate read and write flows, improving initial-value handling and update propagation across datatypes.
    • Added datatype-specific builders for booleans, integers, floats, strings, enums and waveforms and improved error signaling for unsupported types.
  • Chores

    • Removed a deprecated utility and updated tests to align with the new in/out record construction approach.

@ajgdls ajgdls requested a review from GDYendell February 10, 2026 16:46
@codecov
Copy link

codecov bot commented Feb 10, 2026

Codecov Report

❌ Patch coverage is 98.33333% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 90.70%. Comparing base (5d6b41c) to head (ee109fb).

Files with missing lines Patch % Lines
src/fastcs/transports/epics/ca/ioc.py 98.24% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #319      +/-   ##
==========================================
- Coverage   90.84%   90.70%   -0.15%     
==========================================
  Files          70       70              
  Lines        2555     2570      +15     
==========================================
+ Hits         2321     2331      +10     
- Misses        234      239       +5     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copy link
Contributor

@GDYendell GDYendell left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. Minor comments.

@coderabbitai
Copy link

coderabbitai bot commented Feb 11, 2026

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

Split the previous unified record constructor into two functions: _make_in_record for read records and _make_out_record for write records; moved datatype-specific builder selection, metadata construction, and datatype update callbacks into them; removed builder_callable_from_attribute; updated callers and tests.

Changes

Cohort / File(s) Summary
IOC — in/out record builders
src/fastcs/transports/epics/ca/ioc.py
Removed generic _make_record; added _make_in_record(pv, attribute) and _make_out_record(pv, attribute, on_update) with explicit datatype match (Bool, Int, Float, String, Enum, Waveform), in/out-specific metadata, datatype_updater wiring, and FastCSError for unsupported types. Updated call sites to use the new functions.
Utilities — enum/state helpers
src/fastcs/transports/epics/ca/util.py
Removed builder_callable_from_attribute and attribute-based metadata helper; added create_state_keys(datatype) and refactored record_metadata_from_datatype to use centralized enum state-key construction; removed builder/attribute-related imports.
Tests — EPICS util
tests/transports/epics/ca/test_ca_util.py
Deleted tests and imports referencing the removed builder_callable_from_attribute; remaining metadata/casting tests unchanged.
Tests — initial value / softioc / IOC assembly
tests/transports/epics/ca/test_initial_value.py, tests/transports/epics/ca/test_softioc.py
Replaced _make_record usage with _make_in_record / _make_out_record in patches and assertions; updated spies/waits and initial-value aggregation to collect both in/out wrappers; adjusted builder references where required.
Project metadata
manifest-file-name, pyproject.toml, requirements.txt
Project metadata and dependency lists updated (minor edits).

Sequence Diagram(s)

sequenceDiagram
    participant Attr as Attribute
    participant IOC as EPICS IOC / builder
    participant Rec as RecordWrapper

    Attr->>IOC: create in-record (pv, datatype)
    IOC->>IOC: record_metadata_from_datatype(datatype)
    IOC->>IOC: select in-builder for datatype
    IOC->>Rec: instantiate in-record via builder
    IOC->>Rec: attach datatype_updater callback
    Rec-->>Attr: link in-record (read path)

    Attr->>IOC: create out-record (pv, datatype, on_update)
    IOC->>IOC: record_metadata_from_datatype(datatype, out_record=true)
    IOC->>IOC: select out-builder for datatype
    IOC->>Rec: instantiate out-record via builder with on_update
    IOC->>Rec: attach datatype_updater callback
    Rec-->>Attr: link out-record (write path)
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

Poem

🐰 I split one record into two with a hop,
Read hops left, write hops right — builders on top,
Datatypes whisper, callbacks bind,
Metadata stitched to keep in mind,
A rabbit's tidy hop — tests pass, then stop!

🚥 Pre-merge checks | ✅ 5 | ❌ 1
❌ Failed checks (1 warning)
Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 15.38% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title 'Split epics make record' clearly and concisely summarizes the main change: splitting the _make_record function into two specialized functions for reading and writing EPICS records.
Linked Issues check ✅ Passed The PR successfully implements the acceptance criteria from issue #288: it splits _make_record into two functions with explicit match statements, removes builder_callable_from_attribute, and improves code readability.
Out of Scope Changes check ✅ Passed All changes are directly related to issue #288 objectives: splitting record creation functions, removing the builder helper, and updating tests accordingly. No unrelated changes detected.
Merge Conflict Detection ✅ Passed ✅ No merge conflicts detected when merging into main

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing touches
  • 📝 Generate docstrings
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch split-epics-make-record

Comment @coderabbitai help to get the list of available commands and usage tips.

@ajgdls
Copy link
Contributor Author

ajgdls commented Feb 11, 2026

@GDYendell ready.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
tests/transports/epics/ca/test_softioc.py (1)

547-578: ⚠️ Potential issue | 🟡 Minor

Assert the out-record builder after switching to _make_out_record.
The test at line 571 still asserts builder.longIn when it should assert builder.longOut, as _make_out_record uses the out-record builder for write attributes. The assertion also needs to include the correct parameters.

🔧 Proposed fix
-    builder.longIn.assert_called_once_with(
-        pv_name,
-        **record_metadata_from_attribute(attr_w),
-        **record_metadata_from_datatype(attr_w.datatype),
-    )
+    builder.longOut.assert_called_once_with(
+        pv_name,
+        always_update=True,
+        blocking=True,
+        on_update=mocker.ANY,
+        **record_metadata_from_attribute(attr_w),
+        **record_metadata_from_datatype(attr_w.datatype, out_record=True),
+    )
🤖 Fix all issues with AI agents
In `@tests/transports/epics/ca/test_initial_value.py`:
- Around line 68-74: The wait loop condition currently uses "and" which exits as
soon as one list is non-empty; change the loop to wait until both spies have
populated lists by using a condition that continues while either list is empty
(e.g., while not (record_spy.spy_return_list and
record_spy_out.spy_return_list)) so initial_values built from
record_spy.spy_return_list + record_spy_out.spy_return_list is complete; update
the while loop surrounding the asyncio.sleep(0) accordingly (refer to the
variables record_spy, record_spy_out and the initial_values comprehension).

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
tests/transports/epics/ca/test_softioc.py (1)

546-583: ⚠️ Potential issue | 🟡 Minor

Assert longOut for output records in datatype update test.

After switching to _make_out_record, the test still asserts builder.longIn, which doesn’t validate the output record builder path. Consider asserting longOut with out_record=True metadata instead. Line 573.

✅ Suggested test fix
-    builder.longIn.assert_called_once_with(
-        pv_name,
-        **record_metadata_from_attribute(attr_w),
-        **record_metadata_from_datatype(attr_w.datatype),
-    )
+    builder.longOut.assert_called_once_with(
+        pv_name,
+        always_update=True,
+        blocking=True,
+        on_update=mocker.ANY,
+        **record_metadata_from_attribute(attr_w),
+        **record_metadata_from_datatype(attr_w.datatype, out_record=True),
+    )
🤖 Fix all issues with AI agents
In `@src/fastcs/transports/epics/ca/ioc.py`:
- Around line 241-301: datatype_updater currently passes all items from
record_metadata_from_datatype(..., out_record=True) into record.set_field, but
that metadata can include a callable under the "validate" key (for long Enums)
which RecordWrapper.set_field cannot accept; update the datatype_updater closure
to filter out the "validate" key (or any non-scalar/callable values) before
calling record.set_field so only EPICS DB field values are set; ensure this
change is applied where datatype_updater is added via
attribute.add_update_datatype_callback so long Enum handling and other datatypes
remain correct.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Fix all issues with AI agents
In `@src/fastcs/transports/epics/ca/ioc.py`:
- Around line 295-299: datatype_updater is passing the full dict from
record_metadata_from_datatype(..., out_record=True) into record.set_field, which
can include builder-only keys like validate (used by builder.longStringOut) that
are not settable; update datatype_updater to filter out non-field keys (at
minimum remove "validate") from the metadata dict before iterating/calling
record.set_field (use the output of record_metadata_from_datatype but drop keys
such as "validate" or any known builder-only params) so only actual record field
names are passed to record.set_field.
🧹 Nitpick comments (1)
tests/transports/epics/ca/test_softioc.py (1)

547-572: Assert the out-record builder call to fully cover the new out path.

After _make_out_record, the test reasserts longIn with non‑out metadata, so it doesn’t validate the output builder call or out_record=True metadata. Consider asserting longOut with the update kwargs to exercise the out‑record path.

Suggested test adjustment
-    builder.longIn.assert_called_once_with(
-        pv_name,
-        **record_metadata_from_attribute(attr_w),
-        **record_metadata_from_datatype(attr_w.datatype),
-    )
+    builder.longOut.assert_called_once_with(
+        pv_name,
+        always_update=True,
+        blocking=True,
+        on_update=mocker.ANY,
+        **record_metadata_from_attribute(attr_w),
+        **record_metadata_from_datatype(attr_w.datatype, out_record=True),
+    )

@GDYendell
Copy link
Contributor

The nitpick is also correct! The assertion is wrong.


🤖 Fix all issues with AI agents
In `@src/fastcs/transports/epics/ca/ioc.py`:
- Around line 295-299: datatype_updater is passing the full dict from
record_metadata_from_datatype(..., out_record=True) into record.set_field, which
can include builder-only keys like validate (used by builder.longStringOut) that
are not settable; update datatype_updater to filter out non-field keys (at
minimum remove "validate") from the metadata dict before iterating/calling
record.set_field (use the output of record_metadata_from_datatype but drop keys
such as "validate" or any known builder-only params) so only actual record field
names are passed to record.set_field.

🧹 Nitpick comments (1)
tests/transports/epics/ca/test_softioc.py (1)

547-572: Assert the out-record builder call to fully cover the new out path.

After _make_out_record, the test reasserts longIn with non‑out metadata, so it doesn’t validate the output builder call or out_record=True metadata. Consider asserting longOut with the update kwargs to exercise the out‑record path.

Suggested test adjustment
-    builder.longIn.assert_called_once_with(
-        pv_name,
-        **record_metadata_from_attribute(attr_w),
-        **record_metadata_from_datatype(attr_w.datatype),
-    )
+    builder.longOut.assert_called_once_with(
+        pv_name,
+        always_update=True,
+        blocking=True,
+        on_update=mocker.ANY,
+        **record_metadata_from_attribute(attr_w),
+        **record_metadata_from_datatype(attr_w.datatype, out_record=True),
+    )

builder.longIn.assert_called_once_with(
pv_name,
**record_metadata_from_attribute(attr_r),
**record_metadata_from_datatype(attr_r.datatype),
)
record_r.set_field.assert_not_called()
attr_r.update_datatype(Int(units="m", min_alarm=-3))
record_r.set_field.assert_any_call("EGU", "m")
record_r.set_field.assert_any_call("LOPR", -3)
with pytest.raises(
ValueError,
match="Attribute datatype must be of type <class 'fastcs.datatypes.int.Int'>",
):
attr_r.update_datatype(String()) # type: ignore
attr_w = AttrW(Int())
record_w = _make_out_record(pv_name, attr_w, on_update=mocker.ANY)
builder.longIn.assert_called_once_with(
pv_name,
**record_metadata_from_attribute(attr_w),
**record_metadata_from_datatype(attr_w.datatype),
)

Line 566 should assert longOut is called.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
tests/transports/epics/ca/test_softioc.py (1)

562-569: ⚠️ Potential issue | 🟡 Minor

Test assertion uses wrong builder method but happens to pass.

The test at lines 562-569 asserts ioc_builder.aOut.assert_called_once_with(...) inside pytest.raises(AssertionError) for long command names. However, commands use builder.Action (see ioc.py line 404), not aOut. This test passes coincidentally because aOut is never called for commands, not because of the long name check.

Consider updating to assert against Action for clarity, or remove this redundant check since ioc_builder.Action.assert_called_once_with earlier in the test already validates that only the short command was created.

🧹 Nitpick comments (4)
src/fastcs/transports/epics/ca/ioc.py (1)

305-315: Validator closure captures datatype at definition time.

The _verify_in_datatype closure on line 307-308 captures datatype by reference at definition time. If update_datatype is later called with a modified Enum (e.g., different member names), the validator will still reference the original datatype's names rather than the updated ones.

This is likely acceptable if Enum member changes via update_datatype are not a supported use case, but worth noting for awareness.

tests/transports/epics/ca/test_initial_value.py (2)

75-75: Remove debug print statement.

The print(f"{initial_values}") statement appears to be leftover debug code that should be removed before merge.

Proposed fix
-        print(f"{initial_values}")

113-113: Remove debug print statement.

The print(f"{name} => {value} {initial_values[name]}") statement appears to be leftover debug code that should be removed before merge.

Proposed fix
-            print(f"{name}  =>  {value}  {initial_values[name]}")
tests/transports/epics/ca/test_softioc.py (1)

98-108: Consider cleaning up commented code.

The commented-out code on lines 103-107 suggests the initial_value comparison was problematic. While using mocker.ANY is a valid workaround, consider adding a brief comment explaining why this is necessary (numpy array comparison issues) or removing the dead comments.

@ajgdls ajgdls force-pushed the split-epics-make-record branch from de53cda to 39c7deb Compare February 13, 2026 16:12
Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Fix all issues with AI agents
In `@src/fastcs/transports/epics/ca/ioc.py`:
- Around line 345-349: datatype_updater is passing every key from
record_metadata_from_datatype(datatype, out_record=True) into record.set_field,
but for long Enums that metadata includes a "validate" callable which
RecordWrapper.set_field cannot accept; update datatype_updater to skip the
"validate" key (e.g., filter out entries where name == "validate" before calling
record.set_field) so only EPICS DB field values are passed to record.set_field.
🧹 Nitpick comments (3)
tests/transports/epics/ca/test_initial_value.py (2)

75-75: Consider removing debug print statement.

This debug print appears to be leftover from development/debugging. Consider removing it before merge for cleaner test output.

Proposed fix
-        print(f"{initial_values}")

113-113: Consider removing debug print statement.

Similarly, this per-item debug print could be removed for cleaner test output.

Proposed fix
-            print(f"{name}  =>  {value}  {initial_values[name]}")
tests/transports/epics/ca/test_softioc.py (1)

617-641: Consider adding a test case for long enum datatype updates.

The current test validates Int datatype updates for out-records. Given that record_metadata_from_datatype(datatype, out_record=True) returns a validate callable for long enums (>16 members), consider adding a test case that exercises the datatype update path for a LongEnum attribute to ensure the validate key is properly filtered before calling set_field.

Would you like me to generate a test case for long enum datatype updates?

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Fix all issues with AI agents
In `@src/fastcs/transports/epics/ca/ioc.py`:
- Around line 252-255: In _make_in_record, update the inner function
datatype_updater(datatype: DataType) to filter out builder-only metadata keys
(e.g., "length" returned by record_metadata_from_datatype for String/Waveform)
before calling record.set_field; locate datatype_updater in _make_in_record and
apply the same filtering logic used in _make_out_record so you only iterate and
call record.set_field for allowed (non-builder) keys.

Comment on lines +252 to +255
def datatype_updater(datatype: DataType):
for name, value in record_metadata_from_datatype(datatype).items():
record.set_field(name, value)

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

🧩 Analysis chain

🏁 Script executed:

rg -n -A40 "def record_metadata_from_datatype" src/fastcs/transports/epics/ca/util.py

Repository: DiamondLightSource/FastCS

Length of output: 1651


🏁 Script executed:

rg -n "def _make_in_record" -A20 src/fastcs/transports/epics/ca/ioc.py

Repository: DiamondLightSource/FastCS

Length of output: 896


🏁 Script executed:

rg -n "def _make_out_record" -A25 src/fastcs/transports/epics/ca/ioc.py

Repository: DiamondLightSource/FastCS

Length of output: 971


🏁 Script executed:

rg -n "builder_only_keys" src/fastcs/transports/epics/ca/ioc.py

Repository: DiamondLightSource/FastCS

Length of output: 169


🏁 Script executed:

rg -n "def _make_in_record" -A50 src/fastcs/transports/epics/ca/ioc.py

Repository: DiamondLightSource/FastCS

Length of output: 2160


🏁 Script executed:

rg -n -B5 -A15 "def datatype_updater" src/fastcs/transports/epics/ca/ioc.py

Repository: DiamondLightSource/FastCS

Length of output: 1572


Add filtering for builder-only keys in _make_in_record datatype updater.

The datatype_updater in _make_in_record (lines 252–254) calls record_metadata_from_datatype(datatype) without filtering. For String and Waveform types, this returns the "length" key, which is a builder-only metadata field that cannot be updated via record.set_field() after record creation. When a datatype is updated dynamically, this will cause a runtime error.

Apply the same filtering used in _make_out_record:

🔧 Suggested fix
     def datatype_updater(datatype: DataType):
+        builder_only_keys = {"validate", "length"}
         for name, value in record_metadata_from_datatype(datatype).items():
+            if name in builder_only_keys:
+                continue
             record.set_field(name, value)
🤖 Prompt for AI Agents
In `@src/fastcs/transports/epics/ca/ioc.py` around lines 252 - 255, In
_make_in_record, update the inner function datatype_updater(datatype: DataType)
to filter out builder-only metadata keys (e.g., "length" returned by
record_metadata_from_datatype for String/Waveform) before calling
record.set_field; locate datatype_updater in _make_in_record and apply the same
filtering logic used in _make_out_record so you only iterate and call
record.set_field for allowed (non-builder) keys.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Split EPICS _make_record and remove builder_callable_from_attribute

2 participants