Enhancements for the esp32_flash task

The `offset` for the beam application partition is now read from the partition table on the attached
esp32 device. When a custom partition table is used that does not use `main.avm` for the beam app
partition name the `app_partition` parameter should be used to specify the application partition
to be flashed. Valid application partition sub-types (the type is `data`) are `phy` or `0xAA`.

If the `offset` parameter is specified it will be used to verify that the offset address of the
application partition matched the expected value. This may be used to prevent flashing to a
standard build of AtomMV for application that require a custom partition table.

The `port` that the ESP32 is attached to is now auto discovered by default. When more than one
ESP32 device is plugged into USB the port should be specified to control which device is flashed.

Error reporting has been improved with descriptive error messages.

Dialyzer warnings for the esp32_flash task have been fixed when analyzing atomvm_rebar3_plugin.

Signed-off-by: Winford <winford@object.stream>

Author: UncleGrumpy

CHANGELOG.md

@@ -21,13 +21,26 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - Added dialyzer task to simplify running dialyzer on AtomVM applications.
 - Added support for rp2350 devices to allow for default detection of the device mount path.
 - Added configuration paramenter for setting the path to picotool for the pico_flash task.
+- Added `app_partition` parameter to `esp32_flash` task. This is only needed to be provided for
+custom partition tables that do not use `main.avm` for the beam application partition name, or to
+flash to a custom alternate partition.
 
 ### Changed
 - The `uf2create` task now creates `universal` format uf2 files by default, suitable for both
 rp2040 or rp2350 devices.
 - The `pico_flash` task now checks that a device is an RP2 platform before resetting to `BOOTSEL`
 mode, preventing interference with other MCUs that may be attached to the host system.
 - The `pico_flash` task now aborts on all errors rather than trying to continue after a failure.
+- The `offset` used by the `esp32_flash` task is now read from the partition table of the device.
+When this parameter is provided it will be used to verify the offset of the application partition on
+flash matches the expected value.
+- The `esp32_flash` task now uses auto discovery for the `port` by default.
+- Stacktraces are not shown by default if the `esp32_flash` fails, instead a descriptive error
+message is displayed. To view the stacktrace use diagnostic mode.
+
+### Fixed
+- The `esp32_flash` task aborts when an error occurs, rather than attempt to continue after a step
+has failed.
 
 ## [0.7.5] (2025.05.27)
 

README.md

@@ -260,24 +260,30 @@ Running this AVM file will boot the `myapp` application automatically, without h
 
 You may use the `esp32_flash` task to flash the generated AtomVM packbeam application to the flash storage on an ESP32 device connected over a serial connection.
 
-    shell$ rebar3 help atomvm esp32_flash
-
-    Use this plugin to flash an AtomVM packbeam file to an ESP32 device.
-
-    Usage: rebar3 atomvm esp32_flash [-e <esptool>] [-c <chip>] [-p <port>]
-                                    [-b <baud>] [-o <offset>]
-
-    -e, --esptool  Path to esptool.py
-    -c, --chip     ESP chip (default auto)
-    -p, --port     Device port (default /dev/ttyUSB0)
-    -b, --baud     Baud rate (default 115200)
-    -o, --offset   Offset (default 0x210000)
+```shell
+shell$ rebar3 help atomvm esp32_flash
+
+Use this plugin to flash an AtomVM packbeam file to an ESP32 device.
+
+Usage: rebar3 atomvm esp32_flash [-e <esptool>] [-c <chip>] [-p <port>]
+                                 [-b <baud>] [-o <offset>]
+                                 [-a <app_partition>]
+
+  -e, --esptool        Path to esptool.py
+  -c, --chip           ESP chip (default auto)
+  -p, --port           Device port (default auto discovery)
+  -b, --baud           Baud rate (default 115200)
+  -o, --offset         Offset (default read from device) *old behavior
+                       deprecated, use app_partition. When given, verifies
+                       expected offset to actual
+  -a, --app_partition  Application partition name (default main.avm)
+```
 
 The `esp32_flash` task will use the `esptool.py` command to flash the ESP32 device.  This tool is available via the <a href="https://docs.espressif.com/projects/esp-idf/en/latest/esp32/">IDF SDK</a>, or directly via <a href="https://github.com/espressif/esptool">github</a>.  The `esptool.py` command is also available via many package managers (e.g., MacOS Homebrew).
 
-By default, the `esp32_flash` task will assume the `esptool.py` command is available on the user's executable path.  Alternatively, you may specify the full path to the `esptool.py` command via the `-e` (or `--esptool`) option
+By default, the `esp32_flash` task will assume the `esptool.py` command is available on the user's executable path.  Alternatively, you may specify the full path to the `esptool.py` command via the `-e` (or `--esptool`) option.
 
-By default, the `esp32_flash` task will write to port `/dev/ttyUSB0` at a baud rate of `115200`.  You may control the port and baud settings for connecting to your ESP device via the `-port` and `-baud` options to the `esp32_flash` task, e.g.,
+By default, the `esp32_flash` task uses port auto discovery at a baud rate of `115200`.  You may control the port and baud settings for connecting to your ESP device via the `-port` and `-baud` options to the `esp32_flash` task, e.g.,
 
     shell$ rebar3 atomvm esp32_flash --port /dev/tty.SLAB_USBtoUART --baud 921600
     ...
@@ -306,7 +312,8 @@ The following table enumerates the properties that may be defined in your projec
 | `chip` | `string()` | ESP32 chip type |
 | `port` | `string()` | Device port on which the ESP32 can be located |
 | `baud` | `integer()` | Device BAUD rate |
-| `offset` | `string()` | Offset into which to write AtomVM application |
+| `offset` | `string()` | Optionally verify offset on flash matches expected value. Original behavior deprecated: use `app_partition` for custom images or to target a different partition |
+| `app_partition` | `string()` | Name of application partition to write AtomVM application for custom partition tables |
 
 Example:
 
@@ -319,9 +326,21 @@ Alternatively, the following environment variables may be used to control the ab
 * `ATOMVM_REBAR3_PLUGIN_ESP32_FLASH_PORT`
 * `ATOMVM_REBAR3_PLUGIN_ESP32_FLASH_BAUD`
 * `ATOMVM_REBAR3_PLUGIN_ESP32_FLASH_OFFSET`
+* `ATOMVM_REBAR3_PLUGIN_ESP32_APP_PARTITION`
 
 Any setting specified on the command line take precedence over settings in `rebar.config`, which in turn take precedence over environment variable settings, which in turn take precedence over the default values specified above.
 
+```note
+The  behavior of the `offset` configuration option has changed, the correct offset for standard
+AtomVM builds are determined by the partition table flashed to the device. Elixir supported builds
+are recognized and the correct offset will be used. When using a custom partition table it is
+necessary to supply the `app_partition` name. If an offset is given it will be compared to the
+address of the discovered `app_partition` and an error will be raised if they do not match. The
+partition name of the application partition for standard AtomVM partition tables is `main.avm`, but
+it is not necessary to supply the name when using a release image or one of the standard partition
+tables.
+```
+
 The `esp32_flash` task depends on the `packbeam` task, so the packbeam file will get automatically built if any changes have been made to its dependencies.
 
 ### The `stm32_flash` task

UPDATING.md

@@ -6,6 +6,17 @@
 
 # `atomvm_rebar3_plugin` Update Instructions
 
+## (unreleased)
+
+- The `esp32_flash` task now reads the application `offset` from the partition table on the device.
+If you are using a custom partition table that does not use `main.avm` for the application partition
+name you should supply the name used with the `app_partition` parameter. An `offset` may optionally
+be supplied to assure the offset of the application partition matches the expected offset, this may
+be helpful to assure that specific applications are only flashed to devices with a custom build of
+AtomVM.
+- Pico 2 (RP2350) devices are recognized and now work with default parameters. Specifying device
+path and uf2 flavor for these chipsets is no longer necessary.
+
 ## 0.6.* -> 0.7.*
 
 - The `atomvm_rebar3_plugin` tasks have been moved into the `atomvm` namespace (from the [`rebar3`](https://rebar3.org) `default` namespace).  The "legacy" tasks in the `default` namespace are deprecated, and users will be issued a warning when used.  Be sure to use the `atomvm` namespace in any future usage of this plugin, as the deprecated tasks may be removed without warning.  E.g., `rebar3 atomvm packbeam ...`

src/atomvm_esp32_flash_provider.erl

@@ -30,17 +30,22 @@
 -define(OPTS, [
     {esptool, $e, "esptool", string, "Path to esptool.py"},
     {chip, $c, "chip", string, "ESP chip (default auto)"},
-    {port, $p, "port", string, "Device port (default /dev/ttyUSB0)"},
+    {port, $p, "port", string, "Device port (default auto discovery)"},
     {baud, $b, "baud", integer, "Baud rate (default 115200)"},
-    {offset, $o, "offset", string, "Offset (default 0x210000)"}
+    {offset, $o, "offset", string,
+        "Offset (default read from device) *deprecated, use app_partition.\n"
+        "When given, a warning will be issued if the partition name does not match the\n"
+        "expected name, as long as the offset aligns with a valid application partition."},
+    {app_partition, $a, "app_partition", string, "Application partition name (default main.avm)"}
 ]).
 
 -define(DEFAULT_OPTS, #{
     esptool => "esptool.py",
     chip => "auto",
-    port => "/dev/ttyUSB0",
-    baud => 115200,
-    offset => "0x210000"
+    port => "auto",
+    baud => "115200",
+    offset => "auto",
+    app_partition => "main.avm"
 }).
 
 %%
@@ -81,16 +86,25 @@ do(State) ->
             maps:get(chip, Opts),
             maps:get(port, Opts),
             maps:get(baud, Opts),
-            maps:get(offset, Opts)
+            integer_from_string(maps:get(offset, Opts)),
+            maps:get(app_partition, Opts),
+            State
         ),
         {ok, State}
     catch
-        C:E:S ->
+        C:rebar_abort:S ->
             rebar_api:error(
-                "An error occurred in the ~p task.  Error=~p",
-                [?PROVIDER, E]
+                "A fatal error occurred in the ~p task.",
+                [?PROVIDER]
             ),
+            rebar_api:debug("Class=~p, Error=~p~nSTACKTRACE:~n~p~n", [C, rebar_abort, S]),
+            {error, rebar_abort};
+        C:E:S ->
             rebar_api:debug("Class=~p, Error=~p~nSTACKTRACE:~n~p~n", [C, E, S]),
+            rebar_api:abort(
+                "An unhandled error occurred in the ~p task.  Error=~p",
+                [?PROVIDER, E]
+            ),
             {error, E}
     end.
 
@@ -105,6 +119,7 @@ format_error(Reason) ->
 %% @private
 get_opts(State) ->
     {ParsedArgs, _} = rebar_state:command_parsed_args(State),
+    rebar_api:debug("ParsedArgs = ~w", [ParsedArgs]),
     RebarOpts = atomvm_rebar3_plugin:get_atomvm_rebar_provider_config(State, ?PROVIDER),
     ParsedOpts = atomvm_rebar3_plugin:proplist_to_map(ParsedArgs),
     maps:merge(
@@ -127,39 +142,86 @@ env_opts() ->
             "ATOMVM_REBAR3_PLUGIN_ESP32_FLASH_PORT",
             maps:get(port, ?DEFAULT_OPTS)
         ),
-        baud => maybe_convert_string(
+        baud => integer_from_string(
             os:getenv(
                 "ATOMVM_REBAR3_PLUGIN_ESP32_FLASH_BAUD",
                 maps:get(baud, ?DEFAULT_OPTS)
             )
         ),
-        offset => os:getenv(
-            "ATOMVM_REBAR3_PLUGIN_ESP32_FLASH_OFFSET",
-            maps:get(offset, ?DEFAULT_OPTS)
+        offset => integer_from_string(
+            os:getenv(
+                "ATOMVM_REBAR3_PLUGIN_ESP32_FLASH_OFFSET",
+                maps:get(offset, ?DEFAULT_OPTS)
+            )
+        ),
+        app_partition => os:getenv(
+            "ATOMVM_REBAR3_PLUGIN_ESP32_APP_PARTITION",
+            maps:get(app_partition, ?DEFAULT_OPTS)
         )
     }.
 
 %% @private
-maybe_convert_string(S) when is_list(S) ->
-    list_to_integer(S);
-maybe_convert_string(I) ->
-    I.
+-spec integer_from_string(S :: list() | integer()) -> Value :: integer().
+integer_from_string("auto") ->
+    -1;
+integer_from_string(I) when is_integer(I) ->
+    I;
+integer_from_string(S) when is_list(S) ->
+    S0 = string:trim(S),
+    case lists:prefix("0x", S0) of
+        true ->
+            list_to_integer(lists:subtract(S0, "0x"), 16);
+        false ->
+            list_to_integer(S0)
+    end.
 
 %% @private
-do_flash(ProjectApps, EspTool, Chip, Port, Baud, Offset) ->
+do_flash(ProjectApps, EspTool, Chip, Port, Baud, Offset, Partition, State) ->
+    TempFile = get_part_tempfile(State),
+    Address =
+        case Offset of
+            -1 ->
+                try
+                    read_flash_offset(EspTool, Port, Partition, TempFile)
+                catch
+                    error:{partition_not_found, Partition} ->
+                        rebar_api:error("Partition ~s not found!", [Partition]);
+                    error:invalid_partition_table ->
+                        rebar_api:error(
+                            "Error parsing partition table, possible corruption during transfer", []
+                        )
+                end;
+            Val ->
+                Offset0 = read_flash_offset(EspTool, Port, Partition, TempFile),
+                case Val =:= Offset0 of
+                    true ->
+                        Offset0;
+                    false ->
+                        {ok, Offset1} = validate_offset(EspTool, Port, Val, Partition, TempFile),
+                        Offset1
+                end
+        end,
     [ProjectAppAVM | _] = [get_avm_file(ProjectApp) || ProjectApp <- ProjectApps],
     Portparam =
         case Port of
             "auto" -> "";
-            _ -> ["--port ", Port]
+            _ -> io_lib:format("--port ~s", [Port])
+        end,
+    Baudparam =
+        case Baud of
+            -1 -> "";
+            _ -> io_lib:format("--baud ~w", [Baud])
+        end,
+    Chipparam =
+        case Chip of
+            "auto" -> "";
+            _ -> io_lib:format("--chip ~s", [Chip])
         end,
     Cmd = lists:join(" ", [
         EspTool,
-        "--chip",
-        Chip,
+        Chipparam,
         Portparam,
-        "--baud",
-        integer_to_list(Baud),
+        Baudparam,
         "--before",
         "default_reset",
         "--after",
@@ -172,9 +234,12 @@ do_flash(ProjectApps, EspTool, Chip, Port, Baud, Offset) ->
         "keep",
         "--flash_size",
         "detect",
-        Offset,
+        "0x" ++ integer_to_list(Address, 16),
         ProjectAppAVM
     ]),
+    AVMApp = filename:basename(ProjectAppAVM),
+    rebar_api:info("Flashing ~s to device.", [AVMApp]),
+    %% The following log output is parsed by the tests and should not be changed or removed.
     rebar_api:info("~s~n", [Cmd]),
     rebar_api:console("~s", [os:cmd(Cmd)]),
     ok.
@@ -185,3 +250,77 @@ get_avm_file(App) ->
     Name = binary_to_list(rebar_app_info:name(App)),
     DirName = filename:dirname(OutDir),
     filename:join(DirName, Name ++ ".avm").
+
+%% @private
+read_flash_offset(Esptool, Port, PartName, TempFile) ->
+    rebar_api:info("Reading application partition offset from device...", []),
+    try esp_part_dump:read_app_offset(Esptool, Port, PartName, TempFile) of
+        Offset ->
+            Offset
+    catch
+        error:invalid_partition_table ->
+            rebar_api:error("Invalid partition data!", []);
+        error:no_device ->
+            rebar_api:error("No ESP32 device attached!", []);
+        error:{partition_not_found, Partition} ->
+            rebar_api:error("The partition ~s was not found in device partition table!", [Partition]),
+            rebar_api:error(
+                "When using a custom partition table always specify the 'app_partition' NAME.", []
+            );
+        error:{invalid_subtype, Type} ->
+            rebar_api:error("The partition ~s was found, but used invalid subtype ~w.", [
+                PartName, Type
+            ]);
+        error:invalid_partition_type ->
+            rebar_api:error("The partition ~s is not a data partition!~nOnly data partitions may be used for AtomVM applications", [
+                PartName
+            ]);
+        error:corrupt_partition_data ->
+            rebar_api:error("Fatal error! Corrupt partition table data!", []);
+        error:no_data_partitions ->
+            rebar_api:error("The partition table on the device contains no data partitions");
+        error:Error ->
+            rebar_api:error("Unexpected error reading partition table from device, ~p.", [Error])
+    end.
+
+validate_offset(Esptool, Port, Offset, Partition, TempFile) ->
+    rebar_api:debug("Checking that offset aligns with partition from table on device...", []),
+    try esp_part_dump:partition_at_offset(Esptool, Port, Offset, TempFile) of
+        {error, {Name, Reason}} ->
+            rebar_api:error(
+                "The configured offset 0x~.16B contains partition ~s, which cannot be used for reason ~p.~n",
+                [Offset, Name, Reason]
+            );
+        {Found, _Type} ->
+            rebar_api:warn(
+                "The discovered partition ~s at offset 0x~.16B will be used, not the expected partition named ~s.",
+                [Found, Offset, Partition]
+            ),
+            {ok, Offset}
+    catch
+        _:no_device ->
+            rebar_api:error("No ESP32 device attached!", []);
+        _:corrupt_partition_data ->
+            rebar_api:error("Fatal error! Corrupt partition table data!", []);
+        C:E:S ->
+            rebar_api:debug("===== Crash Stacktrace:~n~p~n", [S]),
+            rebar_api:error(
+                "Unexpected error reading partition table from device. Class = ~p, Error = ~p.", [
+                    C, E
+                ]
+            )
+    end.
+
+%% @private
+get_part_tempfile(State) ->
+    OutDir = filename:absname(rebar_dir:base_dir(State)),
+    TempFile = filename:absname_join(OutDir, "part.tmp"),
+    case filelib:is_file(TempFile) of
+        true ->
+            rebar_api:debug("Removing possibly stale partition dump data ~s", [TempFile]),
+            Cmd = lists:join(" ", ["rm", TempFile]),
+            os:cmd(Cmd);
+        false ->
+            ok
+    end,
+    TempFile.