Introduction

What does an "app" mean in this article is a pre-compiled binary file, which can be run directly with Arduino boards without using Arduino IDE.

And because it is a file, the "app" can be distributed through SD card, Ethernet, WiFi, or any suitable methods.

The title diagram shows a MKR ZERO board executing Arduino app, RTT-QRCode.

Are you be interested?

(This article is based on Arduino RT-Thread library v0.6.0.)

Dynamic Module

In RT-Thread architecture, the "app" is called dynamic module built as dynamic shared library with extension ".mo" or ".so". (What is RT-Thread? => Multitasking on Arduino)

RT-Thread provides APIs to access dynamic module. More interestingly, the MSH (a tiny shell) is able to execute ".mo" file directly (details are in the following sections).

The original dynamic linker of RT-Thread seems doesn't work well with ARM Cortex-M. So I modified the code for Arduino RT-Thread library.

MSH

Module SHell (MSH) is a new feature enabled by default (from v0.5.1 onward) that is built on top of FinSH. (What is FinSH? => Multitasking on Arduino)

Due to Arduino app is executed by MSH, let's have a briefing.

Comparing to FinSH, MSH is more in line with Unix shell's usage habits:

Issuing commands in FinSH

led(0, 1)
copy("datalog.txt", "copy.txt")

Issuing commands in MSH

led 0 1
cp datalog.txt copy.txt

However MSH doesn't support shell variables like the one provided by FinSH.

Another limitation is the prototype of user defined MSH command is fixed:

int my_msh_cmd(int argc, char **argv)

When MSH executing user commands, the parameter argc will be the number of arguments plusone, and argv will be the arguments list (the firstentryiscommandname). As you might have guessed, all the parameters can only be in char array type.

Following is the "led" example in MSH command format.

int led(int argc, char **argv) {
  // argc - the number of arguments
  // argv[0] - command name, e.g. "led"
  // argv[n] - nth argument in the type of char array
  
  rt_uint32_t id;
  rt_uint32_t state;
  
  if (argc != 3) {
    rt_kprintf("Usage: led <id> <state>\
");
    return 1;
  }
  rt_kprintf("led%s=%s\
", argv[1], argv[2]);
  
  // convert arguments to their specific types
  sscanf(argv[1], "%u", &id);
  sscanf(argv[2], "%u", &state);
  if (id != 0) {
    rt_kprintf("Error: Invalid led ID\
");
    return 1;
  }
  
  if (state) {
    digitalWrite(LED_BUILTIN, HIGH);
  } else {
    digitalWrite(LED_BUILTIN, LOW);
  }
  
  return 0;
}

Make Arduino App

First of all, enable CONFIG_USING_MODULE in "rtconfig.h," as it is disabled by default.

Build Executable File

Let's open the "HelloMo" example in Arduino IDE and press "Verify". (The example also available in "Code" section below.) The code now is built into a single executable file including sketch and libraries. We may use GCC tool readelf (provided with Arduino IDE) to verify.

{path_to_gcc_tools}\\arm-none-eabi-readelf -h {path_to_output}\\HelloMo.ino.elf
ELF Header:
 Magic:   7f 45 4c 46 01 01 01 00 00 00 00 00 00 00 00 00
 Class:                             ELF32
 Data:                              2's complement, little endian
 Version:                           1 (current)
 OS/ABI:                            UNIX - System V
 ABI Version:                       0
 Type:                              EXEC (Executable file)
 Machine:                           ARM
 Version:                           0x1
 Entry point address:               0xf7fd
 Start of program headers:          52 (bytes into file)
 Start of section headers:          798052 (bytes into file)
 Flags:                             0x5000002, has entry point, Version5 EABI
 Size of this header:               52 (bytes)
 Size of program headers:           32 (bytes)
 Number of program headers:         2
 Size of section headers:           40 (bytes)
 Number of section headers:         18
 Section header string table index: 15

If you are not sure about the locations of GCC tools and compilation output, please enable the following option in File-> Preferences.

Clicking "Verify" again, you will observe the information in output window:

...
Compiling sketch...
"C:\\\\Users\\\\onelife\\\\AppData\\\\Local\\\\Arduino15\\\\packages\\\\arduino\\\\tools\\\\arm-none-eabi-gcc\\\\4.8.3-2014q1/bin/arm-none-eabi-gcc" -mcpu=cortex-m0plus -mthumb -c -g -Os -Wall -Wextra -std=gnu11 -ffunction-sections -fdata-sections -nostdlib --param max-inline-insns-single=500 -MMD -DF_CPU=48000000L -DARDUINO=10809 -DARDUINO_SAMD_MKRZERO -DARDUINO_ARCH_SAMD -DUSE_ARDUINO_MKR_PIN_LAYOUT -D__SAMD21G18A__ -DUSB_VID=0x2341 -DUSB_PID=0x804f -DUSBCON "-DUSB_MANUFACTURER=\\"Arduino LLC\\"" "-DUSB_PRODUCT=\\"Arduino MKRZero\\"" "-IC:\\\\Users\\\\onelife\\\\AppData\\\\Local\\\\Arduino15\\\\packages\\\\arduino\\\\tools\\\\CMSIS\\\\4.5.0/CMSIS/Include/" "-IC:\\\\Users\\\\onelife\\\\AppData\\\\Local\\\\Arduino15\\\\packages\\\\arduino\\\\tools\\\\CMSIS-Atmel\\\\1.1.0/CMSIS/Device/ATMEL/" "-IC:\\\\Users\\\\onelife\\\\AppData\\\\Local\\\\Arduino15\\\\packages\\\\arduino\\\\hardware\\\\samd\\\\1.6.21\\\\cores\\\\arduino" "-IC:\\\\Users\\\\onelife\\\\AppData\\\\Local\\\\Arduino15\\\\packages\\\\arduino\\\\hardware\\\\samd\\\\1.6.21\\\\variants\\\\mkrzero" "-IC:\\\\Users\\\\onelife\\\\Documents\\\\Arduino\\\\libraries\\\\RT-Thread\\\\src" "-IC:\\\\Users\\\\onelife\\\\AppData\\\\Local\\\\Arduino15\\\\packages\\\\arduino\\\\hardware\\\\samd\\\\1.6.21\\\\libraries\\\\SPI" "C:\\\\Users\\\\onelife\\\\AppData\\\\Local\\\\Temp\\\\arduino_build_508434\\\\sketch\\\\hello_mo.c" -o "C:\\\\Users\\\\onelife\\\\AppData\\\\Local\\\\Temp\\\\arduino_build_508434\\\\sketch\\\\hello_mo.c.o"
...

In my case, the GCC tools are located at "C:\\\\Users\\\\onelife\\\\AppData\\\\Local\\\\Arduino15\\\\packages\\\\arduino\\\\tools\\\\arm-none-eabi-gcc\\\\4.8.3-2014q1/bin/" and the compilation output are located at "C:\\\\Users\\\\onelife\\\\AppData\\\\Local\\\\Temp\\\\arduino_build_508434\\\".

Build App (Dynamic Shared Library)

However, the target "app" we are going to build is a kind of shared library. It has to be position independent, so can be loaded into any RAM address. And to keep it smaller (as our RAM size is limited), the final binary file will not include any function of other libraries. (All the external functions should be provided by firmware side.)

The bad news is Arduino IDE doesn't provide those options. The good news is Arduino IDE does provide all the tools we need. Let's do it.

The first step is compilation.

We have to add the option "-mlong-calls -fPIC" into the original compiling command (looking for "Compiling sketch..." in output window).

{path_to_gcc_tools}\\arm-none-eabi-gcc -mlong-calls -fPIC ... {path_to_output}\\sketch\\hello_mo.c -o {path_to_output}\\sketch\\hello_mo.c.o
  
{path_to_gcc_tools}\\arm-none-eabi-gcc -mlong-calls -fPIC ... {path_to_output}\\sketch\\load_mo.c -o {path_to_output}\\sketch\\load_mo.c.o

The second step is linking.

In this step, we make the choice of either to build the object file as an "app" (".mo" file with entry point) or to build it as a library (".so" file without entry point). In the following example, we build "load_mo.c.o" as "app" and build "hello_mo.c.o" as library.

We modify the linking command (looking for "Linking everything together...") by

keep only the target object file, e.g. "load_mo.c.o", and remove the others
remove option "-Wl,--unresolved-symbols=report-all"
remove option "-L{path_to_output}"
remove option "-T.../flash_with_bootloader.ld"
remove option "-Wl,--start-group ... -Wl,--end-group"
add option "-shared -fPIC -nostdlib -Wl,-marmelf -Wl,-z,max-page-size=0x4"
add option of entry point (e.g. "-Wl,-eload_hello" or "-Wl,-e0" for none)

{path_to_gcc_tools}\\arm-none-eabi-g++ -shared -fPIC -nostdlib -Wl,-e0 -Wl,-marmelf -Wl,-z,max-page-size=0x4 ... -o {path_to_output}\\hello_mo.elf {path_to_output}\\hello_mo.c.o
  
{path_to_gcc_tools}\\arm-none-eabi-g++ -shared -fPIC -nostdlib -Wl,-eload_hello -Wl,-marmelf -Wl,-z,max-page-size=0x4 ... -o {path_to_output}\\load_mo.elf {path_to_output}\\load_mo.c.o

The third step is striping.

To further reduce file size, we have to strip off the unnecessary sections of ELF file.

{path_to_gcc_tools}\\arm-none-eabi-strip -R .hash -R .comment -R .ARM.attributes {path_to_output}\\hello_mo.elf -o {path_to_output}\\hello.so
  
{path_to_gcc_tools}\\arm-none-eabi-strip -R .hash -R .comment -R .ARM.attributes {path_to_output}\\load_mo.elf -o {path_to_output}\\load.mo

The fourth step is check size (optional).

{path_to_gcc_tools}\\arm-none-eabi-size {path_to_output}\\hello.so
  
{path_to_gcc_tools}\\arm-none-eabi-size {path_to_output}\\load.mo

Congratulations! You just built an Arduino app. Let's check the output.

{path_to_gcc_tools}\\arm-none-eabi-readelf -h {path_to_output}\\hello.so
ELF Header:
 Magic:   7f 45 4c 46 01 01 01 00 00 00 00 00 00 00 00 00
 Class:                             ELF32
 Data:                              2's complement, little endian
 Version:                           1 (current)
 OS/ABI:                            UNIX - System V
 ABI Version:                       0
 Type:                              DYN (Shared object file)
 Machine:                           ARM
 Version:                           0x1
 Entry point address:               0x0
 Start of program headers:          52 (bytes into file)
 Start of section headers:          896 (bytes into file)
 Flags:                             0x5000000, Version5 EABI
 Size of this header:               52 (bytes)
 Size of program headers:           32 (bytes)
 Number of program headers:         3
 Size of section headers:           40 (bytes)
 Number of section headers:         9
 Section header string table index: 8
  
{path_to_gcc_tools}\\arm-none-eabi-readelf -h {path_to_output}\\load.mo
ELF Header:
 Magic:   7f 45 4c 46 01 01 01 00 00 00 00 00 00 00 00 00
 Class:                             ELF32
 Data:                              2's complement, little endian
 Version:                           1 (current)
 OS/ABI:                            UNIX - System V
 ABI Version:                       0
 Type:                              DYN (Shared object file)
 Machine:                           ARM
 Version:                           0x1
 Entry point address:               0x285
 Start of program headers:          52 (bytes into file)
 Start of section headers:          1060 (bytes into file)
 Flags:                             0x5000002, has entry point, Version5 EABI
 Size of this header:               52 (bytes)
 Size of program headers:           32 (bytes)
 Number of program headers:         3
 Size of section headers:           40 (bytes)
 Number of section headers:         9
 Section header string table index: 8

It shows that both ".so" and ".mo" files are in "DYN" (dynamic) type. The difference is ".so" file has no entry point but ".mo" file has.

We are not done yet.

The last step is exposing the app required functions.

In the file "mo_sym.h", all the kernel APIs are already exposed if CONFIG_USING_MODULE enabled. You may add your own if necessary.

Issue MSH command "lsym" will list all the exposed symbols:

Run Arduino App

Let's copy "hello.so" and "load.mo" to SD card with the following file structure.

SD_ROOT/
├── lib/
│   └── hello.so
└── mo/
    └── load.mo

The rule is if we pass a relative path to dlopen() or MSH, it will look for ".so" and ".mo" in /lib/ and /mo/ respectively.

Now we insert the card to an Arduino board, in this case MRKZERO, upload "HelloMo" sketch (the sketch does nothing), and then issue command "load".

To show more details about the app execution process, we may enabled the debug message in "dlmodule.c":

#define LOG_LVL LOG_LVL_DBG

The result reveals the following procedures:

MSH thread ("tshell") loads "load.mo" to RAM and creates a new thread ("load") to execute the entry point function "load_hello()"
"load_hello()" then loads "hello.so", calls its "module_init()" function, calls its "say_hello()" function (not a entry point)
After "say_hello()" returned, "load_hello()" closes "hello.so" (calls its "module_cleanup()" function and then destroys its RAM copy)
"load" thread marks to destroy the RAM copy of "load.mo" and then exits
The RAM copy of "load.mo" is finally destroyed by Idle thread ("tidle0")

"module_init()" and "module_cleanup()" are special functions. If defined, the former is called by MSH thread (in case of ".mo" file) after loaded app to RAM, and the later is called by Idle thread (in case of ".mo" file) before destroying the RAM copy.

Let's rebuild the "hello_mo.c" as an app (the entry point is "say_hello()", e.g. -Wl,-esay_hello) and execute.

The result clearly shows that "module_init()" is called by MSH thread ("tshell") and "module_cleanup()" is called by Idle thread ("tidle0"). By the way, the argument passed to those two functions is the pointer to the module descriptor.

Pros & Cons

Pros

(What interested us is) Arduino app can be built once and run on lots of boards. According to Wiki, "Binary instructions available for the Cortex-M0 / Cortex-M0+ / Cortex-M1 can execute without modification on the Cortex-M3 / Cortex-M4 / Cortex-M7. Binary instructions available for the Cortex-M3 can execute without modification on the Cortex-M4 / Cortex-M7 / Cortex-M33 / Cortex-M35P."

So the app built for MKR Zero board (SAMD architecture) should run on Arduino Due (SAM architecture) without issue.

This feature may enable something like add or update functions remotely without restart (comparing to OTA firmware update) and so on.

Cons

Comparing to MSH command, Arduino app requires more RAM. Another main drawback is on the firmware side, all the external functions required by app must be standby there (although the firmware may not use them) and exposed (in "mo_sym.h").

Dynamic Module Status

The feature in v0.6.0 of Arduino RT-Thread library is still under beta phase.

In the original code (RT-Thread project), beside DYN type of ELF file, the dynamic linker also supports REL type. However, after some tests, I find that at least for ARM Cortex-M architecture, only ".o" (object) files with type REL. So REL type of ELF file is not supported by Arduino RT-Thread library for now.

Furthermore, there are only two reallocation types are tested:

R_ARM_JUMP_SLOT
R_ARM_RELATIVE

I need some code to test the other types. So please help to raise issue if you encountered error with other types.

Finally, not all "libgcc" functions are exposed by default. For example, the switch helper functions are not exposed. You may add them to "mo_sym.h" or replace the "switch...case..." with "if...else..." in you App.

RTT-QRCode App

There is a more complicated example RTT-QRCode, which can be built as MSH command or Arduino app. Please check out the code and have fun!

Next Steps

Arduino App with RT-Thread

Have you ever wished for "Arduino app?" A kind of pre-compiled binary file that you may copy to SD card and deploy on multiple boards.