From 1bc3a5da81933d0c431b918e047f24b0dc265d77 Mon Sep 17 00:00:00 2001 From: Prakhar Thapliyal Date: Sun, 23 Nov 2025 18:54:30 -0500 Subject: [PATCH 1/2] Added websocket support for Arduino library --- README.md | 140 +++++++++++++++++--------------- src/Enes100.h | 1 + src/WSVisionSystemClient.cpp | 151 +++++++++++++++++++++++++++++++++++ src/WSVisionSystemClient.hpp | 41 ++++++++++ 4 files changed, 268 insertions(+), 65 deletions(-) create mode 100644 src/WSVisionSystemClient.cpp create mode 100644 src/WSVisionSystemClient.hpp diff --git a/README.md b/README.md index d7855f1..65c5846 100644 --- a/README.md +++ b/README.md @@ -1,7 +1,7 @@ # Enes100ArduinoLibrary An Arduino library for use in the ENES100 course to allow Arduino boards to communicate with the ENES100 Vision System -via ESP8266 Wi-Fi modules. +via ESP8266 Wi-Fi modules. Also supports usage of Arduino IDE with in-built Wi-Fi modules on the ESP-32 through WebSockets implementation. ## Download and Installation @@ -13,6 +13,10 @@ the [Arduino website](https://www.arduino.cc/en/Main/Software). **If you have an older version of the library on your computer, you _must_ delete it before adding a newer version.** Failure to do this may cause file conflicts, and it is not guaranteed that the library will work properly. +**If using an in-built Wi-Fi module, you must download:** +ArduinoWebSockets - Gil Maimon +ArduinoJson - Benoit Blanchon + ## Setup Communication with the Vision System is done using ESP8266 WiFi-enabled microcontrollers. Wi-Fi modules are available @@ -20,7 +24,7 @@ for checkout through a Teaching Fellow. The Wi-Fi module has 4 pins: | Pin | Description | Connect to... | -|-----|-----------------|---------------------| +| --- | --------------- | ------------------- | | GND | Ground | Common ground | | VCC | Voltage supply | +5 V | | TX | Serial transmit | Arduino digital pin | @@ -40,6 +44,9 @@ Allowed Pins: ## Usage +**MAKE SURE THE CORRECT .HPP FILE IS INCLUDED IN Enes100.h** +(uncomment either VisionSystemClient.hpp for external Wi-Fi module and WSVisionSystemClient.hpp for internal Wi-Fi modules) + To use the library, you have to direct the compiler to include it in your code. Go to **Sketch > Include Library > ENES100**, or add it manually by typing `#include "Enes100.h"` at the very top of your file. @@ -48,20 +55,20 @@ ENES100**, or add it manually by typing Format: -```Enes100.begin(const char* teamName, byte teamType, int markerId, int wifiModuleTX, int wifiModuleRX)``` +`Enes100.begin(const char* teamName, byte teamType, int markerId, int wifiModuleTX, int wifiModuleRX)` Initializes the ENES100 library and establishes communication with the Vision System. -The `wifiModuleTX` and `wifiModuleRX` described below refer to the digital pins that will be connected to the __Tx__ and -__Rx__ of the -__Wi-Fi module__. +The `wifiModuleTX` and `wifiModuleRX` described below refer to the digital pins that will be connected to the **Tx** and +**Rx** of the +**Wi-Fi module**. -* teamName: Name of the team that will show up in the Vision System -* teamType: Type of mission your team is running. - * Valid Mission Types: `CRASH_SITE`, `DATA`, `MATERIAL`, `FIRE`, `WATER`, `SEED` -* markerID: ID of your Aruco Marker -* wifiModuleTX: Digital Pin that will be connected to the __Tx pin on the Wi-Fi module__. -* wifiModuleRX: Digital Pin that will be connected to the __Rx pin on the Wi-Fi module__. +- teamName: Name of the team that will show up in the Vision System +- teamType: Type of mission your team is running. + - Valid Mission Types: `CRASH_SITE`, `DATA`, `MATERIAL`, `FIRE`, `WATER`, `SEED` +- markerID: ID of your Aruco Marker +- wifiModuleTX: Digital Pin that will be connected to the **Tx pin on the Wi-Fi module**. +- wifiModuleRX: Digital Pin that will be connected to the **Rx pin on the Wi-Fi module**. Boards have limitations around what pins can be connected to Wi-Fi module TX. Allowed Pins: @@ -75,6 +82,8 @@ Allowed Pins: - Romeo V1 - Same as Uno - Romeo V2 - Same as Leonardo +**If using the ESP-32, simply omit the wifiModuleTX and wifiModuleRX parameters.** + #### **NOTE**: Want to call begin multiple times? There are some of you who will wish to use begin in combination with the isConnected method to 'reconnect' if needed. @@ -93,7 +102,7 @@ void setup() { //Connect digital pin 8 to the Tx pin of the wifi module. //Connect digital pin 9 to the Rx pin of the wifi module. Enes100.begin("It's lit", FIRE, 3, 8, 9); - + // Some other setup code... } ``` @@ -102,17 +111,17 @@ void setup() { The Aruco Marker has 4 values -* x: x-coordinate of the Aruco Marker (from 0.0 to 4.0) -* y: y-coordinate of the Aruco Marker (From 0.0 to 2.0) -* theta: angle of the Aruco Marker (from -pi radians to pi radians) -* visibility: whether the ArUco marker is visible (true or false) +- x: x-coordinate of the Aruco Marker (from 0.0 to 4.0) +- y: y-coordinate of the Aruco Marker (From 0.0 to 2.0) +- theta: angle of the Aruco Marker (from -pi radians to pi radians) +- visibility: whether the ArUco marker is visible (true or false) These values can be queried by calling the following functions: -* `Enes100.getX()` -* `Enes100.getY()` -* `Enes100.getTheta()` -* `Enes100.isVisible()` +- `Enes100.getX()` +- `Enes100.getY()` +- `Enes100.getTheta()` +- `Enes100.isVisible()` Enes100.get variants will make sure you get the latest data available to you about your OTV's location. The first time getX is @@ -149,7 +158,7 @@ Sends a message to the vision system with a new line. Any messages sent after wi println' ```arduino -//These two lines will output +//These two lines will output // "Hello World! // Hello World!" Enes100.println("Hello World!") @@ -163,8 +172,8 @@ for that for specifics on types. Sends value for a mission objective. -* type: what type of mission call you are sending -* message: mission value associated with the mission type. +- type: what type of mission call you are sending +- message: mission value associated with the mission type. **NOTE:** Some of the values passed as arguments in the examples below are **C++ preprocessor definitions** (#define), defined in `Enes100.h`. @@ -176,82 +185,82 @@ For the valid mission calls below, the value `i` will denote an integer value. Valid calls for **CRASH_SITE**: -* `Enes100.mission(LENGTH, i);` *i is in millimeters* -* `Enes100.mission(HEIGHT, i);` *i is in millimeters* -* `Enes100.mission(DIRECTION, NORMAL_X);` *the normal of the exposed panels points in the positive and negative x - direction* -* `Enes100.mission(DIRECTION, NORMAL_Y);` *the normal of the exposed panels points in the positive and negative y - direction* +- `Enes100.mission(LENGTH, i);` _i is in millimeters_ +- `Enes100.mission(HEIGHT, i);` _i is in millimeters_ +- `Enes100.mission(DIRECTION, NORMAL_X);` _the normal of the exposed panels points in the positive and negative x + direction_ +- `Enes100.mission(DIRECTION, NORMAL_Y);` _the normal of the exposed panels points in the positive and negative y + direction_ Valid calls for **DATA**: -* `Enes100.mission(CYCLE, i);` *i is the duty cycle percent (ex. 10, 30, 50, 70, 90)* -* `Enes100.mission(MAGNETISM, MAGNETIC);` -* `Enes100.mission(MAGNETISM, NOT_MAGNETIC);` +- `Enes100.mission(CYCLE, i);` _i is the duty cycle percent (ex. 10, 30, 50, 70, 90)_ +- `Enes100.mission(MAGNETISM, MAGNETIC);` +- `Enes100.mission(MAGNETISM, NOT_MAGNETIC);` Valid calls for **MATERIAL**: -* `Enes100.mission(WEIGHT, HEAVY);` -* `Enes100.mission(WEIGHT, MEDIUM);` -* `Enes100.mission(WEIGHT, LIGHT);` -* `Enes100.mission(MATERIAL_TYPE, FOAM);` -* `Enes100.mission(MATERIAL_TYPE, PLASTIC);` +- `Enes100.mission(WEIGHT, HEAVY);` +- `Enes100.mission(WEIGHT, MEDIUM);` +- `Enes100.mission(WEIGHT, LIGHT);` +- `Enes100.mission(MATERIAL_TYPE, FOAM);` +- `Enes100.mission(MATERIAL_TYPE, PLASTIC);` Valid calls for **FIRE**: -* `Enes100.mission(NUM_CANDLES, i);` *i is an integer (0, 1, 2, 3, 4, 5)* -* `Enes100.mission(TOPOGRAPHY, TOP_A);` -* `Enes100.mission(TOPOGRAPHY, TOP_B);` -* `Enes100.mission(TOPOGRAPHY, TOP_C);` +- `Enes100.mission(NUM_CANDLES, i);` _i is an integer (0, 1, 2, 3, 4, 5)_ +- `Enes100.mission(TOPOGRAPHY, TOP_A);` +- `Enes100.mission(TOPOGRAPHY, TOP_B);` +- `Enes100.mission(TOPOGRAPHY, TOP_C);` Valid calls for **WATER**: -* `Enes100.mission(DEPTH, i);` *i is in mm* -* `Enes100.mission(WATER_TYPE, FRESH_UNPOLLUTED);` -* `Enes100.mission(WATER_TYPE, FRESH_POLLUTED);` -* `Enes100.mission(WATER_TYPE, SALT_UNPOLLUTED);` -* `Enes100.mission(WATER_TYPE, SALT_POLLUTED);` +- `Enes100.mission(DEPTH, i);` _i is in mm_ +- `Enes100.mission(WATER_TYPE, FRESH_UNPOLLUTED);` +- `Enes100.mission(WATER_TYPE, FRESH_POLLUTED);` +- `Enes100.mission(WATER_TYPE, SALT_UNPOLLUTED);` +- `Enes100.mission(WATER_TYPE, SALT_POLLUTED);` Valid calls for **SEED**: -* `Enes100.mission(LOCATION, plot);` *where plot is a single character A, B, C, or D* - +- `Enes100.mission(LOCATION, plot);` _where plot is a single character A, B, C, or D_ ## Machine Learning Functions ### About -The ESPCAM will be mounted to your OTV and act as your wifi module, with the added capabilities of camera vision. Note, this is not the overhead vision system above the arena - this is a development board provided that has a camera on it that will put on your otv. +The ESPCAM will be mounted to your OTV and act as your wifi module, with the added capabilities of camera vision. Note, this is not the overhead vision system above the arena - this is a development board provided that has a camera on it that will put on your otv. ### int Enes100.MLGetPrediction(int modelIndex) Sends current image from the ESPCAM to the Vision System to get processed by your team's machine learning model with the provided index. Models must have been uploaded to the [ENES100 Model Uploader](https://enes100.umd.edu/uploadmodel) with the index requested beforehand to use this function. The function uses your team name (from the Enes100.begin() statement) to find your model. As such, **make sure your team name matches the model name exactly**. Example: -If your ML model contained the categories: **Thumbs Up**, **Thumbs Down**, **Thumb Sideways** in an array in that order, -calling `Enes100.MLGetPrediction()` would return `0` if **Thumbs Up** is predicted, `1` if **Thumbs Down** is predicted, and `2` if **Thumb Sideways** is predicted. +If your ML model contained the categories: **Thumbs Up**, **Thumbs Down**, **Thumb Sideways** in an array in that order, +calling `Enes100.MLGetPrediction()` would return `0` if **Thumbs Up** is predicted, `1` if **Thumbs Down** is predicted, and `2` if **Thumb Sideways** is predicted. ## Example Code and Debugging -Example code for each type of mission is included with the library. To view examples, open Arduino IDE and go to * -*File > Examples > ENES100**. (You must restart the IDE after installing the library for the examples to show.) +Example code for each type of mission is included with the library. To view examples, open Arduino IDE and go to \* +\*File > Examples > ENES100\*\*. (You must restart the IDE after installing the library for the examples to show.) ## Product Demonstration Procedures During the product demonstration, messages sent using `print()` and `println()` will not be shown on the Vision System console. The console will only print out the values that you send using `mission()`. -## For Simulator +## For Simulator -The function calls to get the Aurco values are a little bit different than what they will be when you code for your otv -but they are functionally the same. +The function calls to get the Aurco values are a little bit different than what they will be when you code for your otv +but they are functionally the same. ```arduino void setup() { // Your begin statement with arbitrary values - Enes100.begin("It's lit", FIRE, 3, 8, 9); + Enes100.begin("It's lit", FIRE, 3, 8, 9); } ``` + Once this is called you will use the function `Enes100.updateLocation()` @@ -259,20 +268,21 @@ Once this is called you will use the function Calling this will allow you to access the values of aurco marker by writing the following -* `Enes100.location.x` -* `Enes100.location.y` -* `Enes100.location.theta` +- `Enes100.location.x` +- `Enes100.location.y` +- `Enes100.location.theta` ```arduino void loop() { - // This code will continuously print out the x cordinate + // This code will continuously print out the x cordinate Enes100.updateLocation(); Enes100.println(Enes100.location.x); setBothMotors(25); - + } ``` + #### **Common Issues**: -If you get an error **AxiosError: Network Error** remove the 's' from the -https link in url and reload the simulator +If you get an error **AxiosError: Network Error** remove the 's' from the +https link in url and reload the simulator diff --git a/src/Enes100.h b/src/Enes100.h index ea18a68..4827ecb 100644 --- a/src/Enes100.h +++ b/src/Enes100.h @@ -2,6 +2,7 @@ #define Enes100_h #include "VisionSystemClient.hpp" +// #include "WSVisionSystemClient.hpp" #define CRASH_SITE 0 #define DATA 1 diff --git a/src/WSVisionSystemClient.cpp b/src/WSVisionSystemClient.cpp new file mode 100644 index 0000000..0fe3068 --- /dev/null +++ b/src/WSVisionSystemClient.cpp @@ -0,0 +1,151 @@ +#include "WSVisionSystemClient.hpp" + +VisionSystemClient *VisionSystemClient::instance = nullptr; + +VisionSystemClient::VisionSystemClient() +{ + instance = this; + x = y = theta = -1; + visible = false; + connected = false; +} + +void VisionSystemClient::begin(const char *tName, int mType, int aId, int roomNum) +{ + teamName = tName; + missionType = mType; + markerId = aId; + roomNumber = roomNum; + + // Connect to VS WiFi + char ssid[32]; + sprintf(ssid, "VisionSystem%d-2.4", roomNumber); + const char *password = "@R6u!n01"; + + WiFi.begin(ssid, password); + while (WiFi.status() != WL_CONNECTED) + delay(10); + + // WebSocket callbacks + ws.onEvent(onWebSocketEvent); + ws.begin("192.168.1.2", 7755, "/"); + ws.setReconnectInterval(2000); +} + +void VisionSystemClient::sendPacket(JsonDocument &doc) +{ + String json; + serializeJson(doc, json); + ws.sendTXT(json); +} + +void VisionSystemClient::onWebSocketEvent(WStype_t type, uint8_t *payload, size_t length) +{ + if (!instance) + return; + + switch (type) + { + case WStype_CONNECTED: + { + instance->connected = true; + + // Send begin doc + StaticJsonDocument<128> doc; + doc["op"] = "begin"; + doc["teamName"] = instance->teamName; + doc["aruco"] = instance->markerId; + doc["teamType"] = instance->missionType; + instance->sendPacket(doc); + break; + } + + case WStype_DISCONNECTED: + instance->connected = false; + break; + + case WStype_TEXT: + { + StaticJsonDocument<256> doc; + DeserializationError err = deserializeJson(doc, payload, length); + if (err) + return; + + if (doc["op"] == "aruco") + { + instance->visible = doc["aruco"]["visible"]; + instance->x = doc["aruco"]["x"]; + instance->y = doc["aruco"]["y"]; + instance->theta = doc["aruco"]["theta"]; + } + break; + } + } +} + +bool VisionSystemClient::isConnected() +{ + ws.loop(); + return connected; +} + +bool VisionSystemClient::isVisible() +{ + ws.loop(); + return visible; +} + +float VisionSystemClient::getX() +{ + ws.loop(); + return x; +} +float VisionSystemClient::getY() +{ + ws.loop(); + return y; +} +float VisionSystemClient::getTheta() +{ + ws.loop(); + return theta; +} + +void VisionSystemClient::mission(const char *call, int msg) +{ + StaticJsonDocument<128> doc; + doc["op"] = "mission"; + doc["teamName"] = teamName; + doc["type"] = call; + doc["message"] = msg; + sendPacket(doc); +} + +void VisionSystemClient::mission(const char *call, float msg) +{ + StaticJsonDocument<128> doc; + doc["op"] = "mission"; + doc["teamName"] = teamName; + doc["type"] = call; + doc["message"] = msg; + sendPacket(doc); +} + +void VisionSystemClient::mission(const char *call, const char *msg) +{ + StaticJsonDocument<128> doc; + doc["op"] = "mission"; + doc["teamName"] = teamName; + doc["type"] = call; + doc["message"] = msg; + sendPacket(doc); +} + +void VisionSystemClient::print(const char *msg) +{ + StaticJsonDocument<128> doc; + doc["op"] = "print"; + doc["teamName"] = teamName; + doc["message"] = msg; + sendPacket(doc); +} diff --git a/src/WSVisionSystemClient.hpp b/src/WSVisionSystemClient.hpp new file mode 100644 index 0000000..c532098 --- /dev/null +++ b/src/WSVisionSystemClient.hpp @@ -0,0 +1,41 @@ +#pragma once +#include +#include +#include +#include + +class VisionSystemClient +{ +public: + VisionSystemClient(); + + void begin(const char *teamName, int missionType, int markerId, int roomNumber); + void mission(const char *missionCall, const char *msg); + void mission(const char *missionCall, int msg); + void mission(const char *missionCall, float msg); + + bool isConnected(); + bool isVisible(); + + float getX(); + float getY(); + float getTheta(); + + void print(const char *message); + +private: + void sendPacket(JsonDocument &doc); + static void onWebSocketEvent(WStype_t type, uint8_t *payload, size_t length); + + static VisionSystemClient *instance; + WebSocketsClient ws; + + String teamName; + int missionType; + int markerId; + int roomNumber; + + volatile float x, y, theta; + volatile bool visible; + volatile bool connected; +}; From d921aa70b34d1a4544c5f3b1b4e6d54f46b3a2d1 Mon Sep 17 00:00:00 2001 From: Prakhar Thapliyal <88155905+OPGuy2000@users.noreply.github.com> Date: Sun, 23 Nov 2025 19:08:36 -0500 Subject: [PATCH 2/2] Fix README formatting --- README.md | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/README.md b/README.md index 65c5846..5c02184 100644 --- a/README.md +++ b/README.md @@ -13,8 +13,8 @@ the [Arduino website](https://www.arduino.cc/en/Main/Software). **If you have an older version of the library on your computer, you _must_ delete it before adding a newer version.** Failure to do this may cause file conflicts, and it is not guaranteed that the library will work properly. -**If using an in-built Wi-Fi module, you must download:** -ArduinoWebSockets - Gil Maimon +**If using an in-built Wi-Fi module, you must download:** +ArduinoWebSockets - Gil Maimon, ArduinoJson - Benoit Blanchon ## Setup @@ -44,8 +44,8 @@ Allowed Pins: ## Usage -**MAKE SURE THE CORRECT .HPP FILE IS INCLUDED IN Enes100.h** -(uncomment either VisionSystemClient.hpp for external Wi-Fi module and WSVisionSystemClient.hpp for internal Wi-Fi modules) +**MAKE SURE THE CORRECT .HPP FILE IS INCLUDED IN Enes100.h** +(uncomment either `VisionSystemClient.hpp` for external Wi-Fi module and `WSVisionSystemClient.hpp` for internal Wi-Fi modules) To use the library, you have to direct the compiler to include it in your code. Go to **Sketch > Include Library > ENES100**, or add it manually by typing @@ -82,6 +82,8 @@ Allowed Pins: - Romeo V1 - Same as Uno - Romeo V2 - Same as Leonardo + + **If using the ESP-32, simply omit the wifiModuleTX and wifiModuleRX parameters.** #### **NOTE**: Want to call begin multiple times?