Enhancements for the esp32_flash task

The `offset` for the beam application partition is now read from the partition table on the attached
esp32 device. When a custom partition table is used that does not use `main.avm` for the beam app
partition name the `app_partition` parameter should be used to specify the application partition
to be flashed. Valid application partition sub-types (the type is `data`) are `phy` or `0xAA`.

If the `offset` parameter is specified it will be used to verify that the offset address of the
application partition matched the expected value. This may be used to prevent flashing to a
standard build of AtomMV for application that require a custom partition table.

The `port` that the ESP32 is attached to is now auto discovered by default. When more than one
ESP32 device is plugged into USB the port should be specified to control which device is flashed.

Error reporting has been improved with descriptive error messages.

Dialyzer warnings for the esp32_flash task have been fixed when analyzing atomvm_rebar3_plugin.

Signed-off-by: Winford <winford@object.stream>

Author: UncleGrumpy

CHANGELOG.md

@@ -21,13 +21,26 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - Added dialyzer task to simplify running dialyzer on AtomVM applications.
 - Added support for rp2350 devices to allow for default detection of the device mount path.
 - Added configuration paramenter for setting the path to picotool for the pico_flash task.
+- Added `app_partition` parameter to `esp32_flash` task. This is only needed to be provided for
+custom partition tables that do not use `main.avm` for the beam application partition name, or to
+flash to a custom alternate partition.
 
 ### Changed
 - The `uf2create` task now creates `universal` format uf2 files by default, suitable for both
 rp2040 or rp2350 devices.
 - The `pico_flash` task now checks that a device is an RP2 platform before resetting to `BOOTSEL`
 mode, preventing interference with other MCUs that may be attached to the host system.
 - The `pico_flash` task now aborts on all errors rather than trying to continue after a failure.
+- The `offset` used by the `esp32_flash` task is now read from the partition table of the device.
+When this parameter is provided it will be used to verify the offset of the application partition on
+flash matches the expected value.
+- The `esp32_flash` task now uses auto discovery for the `port` by default.
+- Stacktraces are not shown by default if the `esp32_flash` fails, instead a descriptive error
+message is displayed. To view the stacktrace use diagnostic mode.
+
+### Fixed
+- The `esp32_flash` task aborts when an error occurs, rather than attempt to continue after a step
+has failed.
 
 ## [0.7.5] (2025.05.27)
 

README.md

@@ -260,24 +260,30 @@ Running this AVM file will boot the `myapp` application automatically, without h
 
 You may use the `esp32_flash` task to flash the generated AtomVM packbeam application to the flash storage on an ESP32 device connected over a serial connection.
 
-    shell$ rebar3 help atomvm esp32_flash
-
-    Use this plugin to flash an AtomVM packbeam file to an ESP32 device.
-
-    Usage: rebar3 atomvm esp32_flash [-e <esptool>] [-c <chip>] [-p <port>]
-                                    [-b <baud>] [-o <offset>]
-
-    -e, --esptool  Path to esptool.py
-    -c, --chip     ESP chip (default auto)
-    -p, --port     Device port (default /dev/ttyUSB0)
-    -b, --baud     Baud rate (default 115200)
-    -o, --offset   Offset (default 0x210000)
+```shell
+shell$ rebar3 help atomvm esp32_flash
+
+Use this plugin to flash an AtomVM packbeam file to an ESP32 device.
+
+Usage: rebar3 atomvm esp32_flash [-e <esptool>] [-c <chip>] [-p <port>]
+                                 [-b <baud>] [-o <offset>]
+                                 [-a <app_partition>]
+
+  -e, --esptool        Path to esptool.py
+  -c, --chip           ESP chip (default auto)
+  -p, --port           Device port (default auto discovery)
+  -b, --baud           Baud rate (default 115200)
+  -o, --offset         Offset (default read from device) *old behavior
+                       deprecated, use app_partition. When given, verifies
+                       expected offset to actual
+  -a, --app_partition  Application partition name (default main.avm)
+```
 
 The `esp32_flash` task will use the `esptool.py` command to flash the ESP32 device.  This tool is available via the <a href="https://docs.espressif.com/projects/esp-idf/en/latest/esp32/">IDF SDK</a>, or directly via <a href="https://github.com/espressif/esptool">github</a>.  The `esptool.py` command is also available via many package managers (e.g., MacOS Homebrew).
 
-By default, the `esp32_flash` task will assume the `esptool.py` command is available on the user's executable path.  Alternatively, you may specify the full path to the `esptool.py` command via the `-e` (or `--esptool`) option
+By default, the `esp32_flash` task will assume the `esptool.py` command is available on the user's executable path.  Alternatively, you may specify the full path to the `esptool.py` command via the `-e` (or `--esptool`) option.
 
-By default, the `esp32_flash` task will write to port `/dev/ttyUSB0` at a baud rate of `115200`.  You may control the port and baud settings for connecting to your ESP device via the `-port` and `-baud` options to the `esp32_flash` task, e.g.,
+By default, the `esp32_flash` task uses port auto discovery at a baud rate of `115200`.  You may control the port and baud settings for connecting to your ESP device via the `-port` and `-baud` options to the `esp32_flash` task, e.g.,
 
     shell$ rebar3 atomvm esp32_flash --port /dev/tty.SLAB_USBtoUART --baud 921600
     ...
@@ -306,7 +312,8 @@ The following table enumerates the properties that may be defined in your projec
 | `chip` | `string()` | ESP32 chip type |
 | `port` | `string()` | Device port on which the ESP32 can be located |
 | `baud` | `integer()` | Device BAUD rate |
-| `offset` | `string()` | Offset into which to write AtomVM application |
+| `offset` | `string()` | Optionally verify offset on flash matches expected value. Original behavior deprecated: use `app_partition` for custom images or to target a different partition |
+| `app_partition` | `string()` | Name of application partition to write AtomVM application for custom partition tables |
 
 Example:
 
@@ -319,9 +326,21 @@ Alternatively, the following environment variables may be used to control the ab
 * `ATOMVM_REBAR3_PLUGIN_ESP32_FLASH_PORT`
 * `ATOMVM_REBAR3_PLUGIN_ESP32_FLASH_BAUD`
 * `ATOMVM_REBAR3_PLUGIN_ESP32_FLASH_OFFSET`
+* `ATOMVM_REBAR3_PLUGIN_ESP32_APP_PARTITION`
 
 Any setting specified on the command line take precedence over settings in `rebar.config`, which in turn take precedence over environment variable settings, which in turn take precedence over the default values specified above.
 
+```note
+The  behavior of the `offset` configuration option has changed, the correct offset for standard
+AtomVM builds are determined by the partition table flashed to the device. Elixir supported builds
+are recognized and the correct offset will be used. When using a custom partition table it is
+necessary to supply the `app_partition` name. If an offset is given it will be compared to the
+address of the discovered `app_partition` and an error will be raised if they do not match. The
+partition name of the application partition for standard AtomVM partition tables is `main.avm`, but
+it is not necessary to supply the name when using a release image or one of the standard partition
+tables.
+```
+
 The `esp32_flash` task depends on the `packbeam` task, so the packbeam file will get automatically built if any changes have been made to its dependencies.
 
 ### The `stm32_flash` task

UPDATING.md

@@ -6,6 +6,17 @@
 
 # `atomvm_rebar3_plugin` Update Instructions
 
+## (unreleased)
+
+- The `esp32_flash` task now reads the application `offset` from the partition table on the device.
+If you are using a custom partition table that does not use `main.avm` for the application partition
+name you should supply the name used with the `app_partition` parameter. An `offset` may optionally
+be supplied to assure the offset of the application partition matches the expected offset, this may
+be helpful to assure that specific applications are only flashed to devices with a custom build of
+AtomVM.
+- Pico 2 (RP2350) devices are recognized and now work with default parameters. Specifying device
+path and uf2 flavor for these chipsets is no longer necessary.
+
 ## 0.6.* -> 0.7.*
 
 - The `atomvm_rebar3_plugin` tasks have been moved into the `atomvm` namespace (from the [`rebar3`](https://rebar3.org) `default` namespace).  The "legacy" tasks in the `default` namespace are deprecated, and users will be issued a warning when used.  Be sure to use the `atomvm` namespace in any future usage of this plugin, as the deprecated tasks may be removed without warning.  E.g., `rebar3 atomvm packbeam ...`

src/atomvm_esp32_flash_provider.erl

@@ -30,17 +30,21 @@
 -define(OPTS, [
     {esptool, $e, "esptool", string, "Path to esptool.py"},
     {chip, $c, "chip", string, "ESP chip (default auto)"},
-    {port, $p, "port", string, "Device port (default /dev/ttyUSB0)"},
+    {port, $p, "port", string, "Device port (default auto discovery)"},
     {baud, $b, "baud", integer, "Baud rate (default 115200)"},
-    {offset, $o, "offset", string, "Offset (default 0x210000)"}
+    {offset, $o, "offset", string,
+        "Offset (default read from device) *old behavior deprecated, use app_partition."
+        " When given, verifies expected offset to actual"},
+    {app_partition, $a, "app_partition", string, "Application partition name (default main.avm)"}
 ]).
 
 -define(DEFAULT_OPTS, #{
     esptool => "esptool.py",
     chip => "auto",
-    port => "/dev/ttyUSB0",
-    baud => 115200,
-    offset => "0x210000"
+    port => "auto",
+    baud => "115200",
+    offset => auto,
+    app_partition => "main.avm"
 }).
 
 %%
@@ -81,13 +85,22 @@ do(State) ->
             maps:get(chip, Opts),
             maps:get(port, Opts),
             maps:get(baud, Opts),
-            maps:get(offset, Opts)
+            maybe_convert_string(maps:get(offset, Opts)),
+            list_to_binary(maps:get(app_partition, Opts)),
+            State
         ),
         {ok, State}
     catch
+        C:rebar_abort:S ->
+            rebar_api:error(
+                "A fatal error occurred in the ~p task.",
+                [?PROVIDER]
+            ),
+            rebar_api:debug("Class=~p, Error=~p~nSTACKTRACE:~n~p~n", [C, rebar_abort, S]),
+            {error, rebar_abort};
         C:E:S ->
             rebar_api:error(
-                "An error occurred in the ~p task.  Error=~p",
+                "An unhandled error occurred in the ~p task.  Error=~p",
                 [?PROVIDER, E]
             ),
             rebar_api:debug("Class=~p, Error=~p~nSTACKTRACE:~n~p~n", [C, E, S]),
@@ -105,6 +118,7 @@ format_error(Reason) ->
 %% @private
 get_opts(State) ->
     {ParsedArgs, _} = rebar_state:command_parsed_args(State),
+    rebar_api:debug("ParsedArgs = ~p", [ParsedArgs]),
     RebarOpts = atomvm_rebar3_plugin:get_atomvm_rebar_provider_config(State, ?PROVIDER),
     ParsedOpts = atomvm_rebar3_plugin:proplist_to_map(ParsedArgs),
     maps:merge(
@@ -133,20 +147,47 @@ env_opts() ->
                 maps:get(baud, ?DEFAULT_OPTS)
             )
         ),
-        offset => os:getenv(
-            "ATOMVM_REBAR3_PLUGIN_ESP32_FLASH_OFFSET",
-            maps:get(offset, ?DEFAULT_OPTS)
+        offset => maybe_convert_string(
+            os:getenv(
+                "ATOMVM_REBAR3_PLUGIN_ESP32_FLASH_OFFSET",
+                maps:get(offset, ?DEFAULT_OPTS)
+            )
+        ),
+        app_partition => os:getenv(
+            "ATOMVM_REBAR3_PLUGIN_ESP32_APP_PARTITION",
+            maps:get(app_partition, ?DEFAULT_OPTS)
         )
     }.
 
 %% @private
 maybe_convert_string(S) when is_list(S) ->
-    list_to_integer(S);
+    case lists:prefix("0x", S) of
+        true ->
+            list_to_integer(lists:subtract(S, "0x"), 16);
+        false ->
+            list_to_integer(S)
+    end;
 maybe_convert_string(I) ->
     I.
 
 %% @private
-do_flash(ProjectApps, EspTool, Chip, Port, Baud, Offset) ->
+do_flash(ProjectApps, EspTool, Chip, Port, Baud, Address, Partition, State) ->
+    Offset =
+        case Address of
+            auto ->
+                read_flash_offset(EspTool, Port, Partition, State);
+            Val ->
+                Offset0 = read_flash_offset(EspTool, Port, Partition, State),
+                case Val =:= Offset0 of
+                    true ->
+                        Offset0;
+                    false ->
+                        rebar_api:abort(
+                            "The configured offset 0x~.16B does not match the partition table on the device (0x~.16B).",
+                            [Val, Offset0]
+                        )
+                end
+        end,
     [ProjectAppAVM | _] = [get_avm_file(ProjectApp) || ProjectApp <- ProjectApps],
     Portparam =
         case Port of
@@ -172,9 +213,12 @@ do_flash(ProjectApps, EspTool, Chip, Port, Baud, Offset) ->
         "keep",
         "--flash_size",
         "detect",
-        Offset,
+        integer_to_list(Offset),
         ProjectAppAVM
     ]),
+    AVMApp = filename:basename(ProjectAppAVM),
+    rebar_api:info("Flashing ~s to device.", [AVMApp]),
+    %% The following log output is parsed by the tests and should not be changed or removed.
     rebar_api:info("~s~n", [Cmd]),
     rebar_api:console("~s", [os:cmd(Cmd)]),
     ok.
@@ -185,3 +229,44 @@ get_avm_file(App) ->
     Name = binary_to_list(rebar_app_info:name(App)),
     DirName = filename:dirname(OutDir),
     filename:join(DirName, Name ++ ".avm").
+
+%% @private
+read_flash_offset(Esptool, Port, PartName, State) ->
+    TempFile = get_part_tempfile(State),
+    rebar_api:info("Reading application partition offset from device...", []),
+    try esp_part_dump:read_app_offset(Esptool, Port, PartName, TempFile) of
+        Offset ->
+            Offset
+    catch
+        _:invalid_partition_table:_ ->
+            rebar_api:abort("Invalid partition data!", []);
+        _:no_device:_ ->
+            rebar_api:abort("No ESP32 device attached!", []);
+        _:{partition_not_found, Partition}:_ ->
+            rebar_api:error("The partition ~s was not fount on device partition table!", [Partition]),
+            rebar_api:abort(
+                "When using a custom partition table always specify the 'app_partition' NAME.", []
+            );
+        _:{invalid_subtype, Type}:_ ->
+            rebar_api:abort("The partition ~s was found, but used invalid subtype 0x~s.", [
+                PartName, Type
+            ]);
+        _:invalid_partition_data:_ ->
+            rebar_api:abort("The partition ~s was found, but partition data is invalid.", [PartName]);
+        _:Error:_ ->
+            rebar_api:abort("Unexpected error reading partition table from device, ~p.", [Error])
+    end.
+
+%% @private
+get_part_tempfile(State) ->
+    OutDir = filename:absname(rebar_dir:base_dir(State)),
+    TempFile = filename:absname_join(OutDir, "part.tmp"),
+    case filelib:is_file(TempFile) of
+        true ->
+            rebar_api:debug("Removing possibly stale partition dump data ~s", [TempFile]),
+            Cmd = lists:join(" ", ["rm", TempFile]),
+            os:cmd(Cmd);
+        false ->
+            ok
+    end,
+    TempFile.

test/driver/scripts/README.md

@@ -0,0 +1,41 @@
+<!--
+ Copyright 2025 Winford (Uncle Grumpy) <winford@object.stream>
+
+ SPDX-License-Identifier: Apache-2.0 OR LGPL-2.1-or-later
+-->
+# esptool.sh and partition binary files
+
+The partition binary files in this directory are used by esptool.sh to simulate the real esptool.py
+dummping the partiton table from an AtomVM installed device. They were built using esp-idf and the
+following partiton.csv contents were used to generate the partition tables:
+
+## partition.bin
+This partition table starts with the standard Erlang only partiton table, but adds several extra
+partitons used for tests to flash to an alternate partition and test failures for invalid partiton
+tyes.
+
+```csv
+# Name,   Type, SubType, Offset,  Size, Flags
+# Note: if you change the phy_init or app partition offset, make sure to change the offset in Kconfig.projbuild
+nvs,      data, nvs,       0x9000,     0x6000,   \
+phy_init, data, phy,       0xf000,     0x1000,    \
+factory,  app,  factory,  0x10000,   0x1C0000,     > these are standard partitons
+boot.avm, data, phy,     0x1D0000,    0x40000,    /
+main.avm, data, phy,     0x210000,   0x100000,   /
+app1.avm, data, 0xAA,    0x310000,   0x070000,   \
+bad1,   data, undefined, 0x380000,   0x010000,    > extra test partitons
+bad2,   app,    test,    0x390000,   0x010000    /
+```
+
+## partition_elixir.bin
+This partition talbe is just the standard Elixir supported build partiton table.
+
+```csv
+# Name,   Type, SubType, Offset,  Size, Flags
+# Note: if you change the phy_init or app partition offset, make sure to change the offset in Kconfig.projbuild
+nvs,      data, nvs,       0x9000,     0x6000,
+phy_init, data, phy,       0xf000,     0x1000,
+factory,  app,  factory,  0x10000,   0x1C0000,
+boot.avm, data, 0xAB,    0x1D0000,    0x80000,
+main.avm, data, 0xAA,    0x250000,   0x100000
+```

test/driver/scripts/esptool.sh

@@ -0,0 +1,21 @@
+#!/bin/sh
+##
+## Copyright (c) Winford (UncleGrumpy) <winford@object.stream>
+## All rights reserved.
+##
+# SPDX-License-Identifier: Apache-2.0 OR LGPL-2.1-or-later
+
+while [ "${1}" != "" ]; do
+    case ${1} in
+        read_flash )    shift
+                        if ( [ ${1} = "0x8000" ] && [ ${2} = "0xc00" ] ); then
+                            target=${3}
+                            cp ${ATOMVM_REBAR3_PLUGIN_PARTITION_DATA:=${ATOMVM_REBAR3_PLUGIN_ESP32_PARTITION_DUMP}} ${target}
+                            unset ATOMVM_REBAR3_PLUGIN_PARTITION_DATA
+                            break;
+                        fi
+    esac
+    shift
+done
+
+echo "${@}"

test/driver/scripts/partition.bin

@@ -0,0 +1,2 @@
+SPDX-License-Identifier: Apache-2.0
+SPDX-FileCopyrightText: Winford (Uncle Grumpy) <winford@object.stream>

test/driver/scripts/partition.bin.license

@@ -0,0 +1,2 @@
+SPDX-License-Identifier: Apache-2.0
+SPDX-FileCopyrightText: Winford (Uncle Grumpy) <winford@object.stream>