Skip to content


android  notes  hal  aidl  hidl  kernel  arduino  esp32  book  management  c/c++  arm  freertos  stm32  debug  sysview  jetson  linux  vuejs  nodejs  electron  chromium  dmm  hardware  raspberry-pi  camera  i2c  ffmpeg  performance  v4l2  picamera  stream  hls  dash  python  h264  mjpeg  ros  jinja  javascript  mkdocs  markdown  encryption  asm  rtt  swv  virtual-machine  docker  hyper-v  serial  yocto  gnss  lidar 


FreeRTOS »

Reentrancy in Newlib

Reentrancy is an attribute of a piece of code and basically means it can be re-entered by another execution flow, for example by an interrupt or by another task or thread. GNU ARM Embedded Toolchain distributions include a non-polluting reduced-size runtime library called Newlib. FreeRTOS has not fully supported reentrant for Newlib.

#arm   #freertos   #stm32  

Last update: 2022-06-29

Table of Content

STM32-Tutorials

Reentrant#

Reentrant is an attribute of a piece of code and basically means it can be re-entered by another execution flow, for example by an interrupt or by another task or thread.

Generally speaking, a function produces output data based on some input data (though both are optional, in general). Shared data could be accessed by any function at any time. If data can be changed by any function (and none keep track of those changes), there is no guarantee to those that share a datum that that datum is the same as at any time before.

Data has a characteristic called scope, which describes where in a program the data may be used. Data scope is either global (outside the scope of any function and with an indefinite extent) or local (created each time a function is called and destroyed upon exit).

Local data is not shared by any routines, re-entering or not; therefore, it does not affect re-entrance. Global data is defined outside functions and can be accessed by more than one function, either in the form of global variables (data shared between all functions), or as static variables (data shared by all invocations of the same function).

Reentrant is distinct from, but closely related to, thread-safety. A function can be thread-safe and still not reentrant.

Rules for reentrant#

  1. Reentrant code may not hold any static or global non-constant data.
  2. Reentrant code may not modify itself.
  3. Reentrant code may not call non-reentrant computer programs or routines.

Examples#

Two functions below are reentrant:

int f(int i) {
    return i + 2;
}

int g(int i) {
    return f(i) + 2;
}

However, if f() depends on non-constant global variable, both functions become non-reentrant, such as:

int v = 1;

int f(int i) {
    v += i;
    return v;
}

int g(int i) {
    return f(i) + 2;
}

Some functions are thread-safe, but not reentrant, such as below function. function() can be called by different threads without any problem. But, if the function is used in a reentrant interrupt handler and a second interrupt arises inside the function, the second routine will hang forever.

int function() {
    mutex_lock();
    {
        // function body
    }
    mutex_unlock();
}

Newlib implementation#

GNU ARM libraries use Newlib to provide standard implementation of C libraries. However, to reduce the code size and make it independent to hardware, there is a lightweight version Newlib-nano used in MCUs.

The Newlib library maps standard C functions to a specific implementation environment through a chain of functions, for example:

  1. write() invokes _write_r() with the current reentrant context (e.g. thread/task-unique errno);
  2. _write_r() invokes _write() and copies errno appropriately;
  3. _write() must be provided by something.

By default, the Newlib-nano library does not provide an implementation of low-level system calls which are used by C standard libraries, such as _write() or _read().

To make the application compilable, a new library named nosys (enabled with -specs=nosys.specs to the gcc linker command line) should be added. This library just provide a simple implementation of low-level system calls which mostly return a by-pass value. CubeMX, with nosys, will generate syscalls.c and sysmem.c to provide low-level implementation for Newlib-nano interface:


Function and data object definitions:

char __ environ;
int _chown (const char * path, uid_t owner, gid_t group);
int_execve (const char * filename, char * const argv[], char * const envp[]);
pid_t _fork (void);
pid_t _getpid (void);
int _gettimeofday (struct timeval * tv, struct timezone * tz);
int _kill (pid_t pid, int sig);
int _link (const char * oldpath, const char * newpath);
ssize_t _readlink (const char * path, char * buf, size_t bufsiz);
int _stat (const char * path, struct stat * buf);
int _symlink (const char * oldpath, const char * newpath);
clock_t _times (struct tms *buf);
int _unlink (const char * pathname);
pid_t _wait (int * status);
void _exit (int status);

File Descriptor Operations:

int _close (int fd);
int _fstat (int fd, struct stat * buf);
int _isatty (int fd);
off_t _lseek (int fd, off_t offset, int whence);
int _open (const char * pathname, int flags);
ssize_t _read (int fd, void * buf, size_t count);
ssize_t _write (int fd, const void * buf, size_t count);

Heap Management:

void * _sbrk (ptrdiff_t increment);

Newlib reentrant#

The Newlib library does support reentrant, but for Newlib-nano, the reentrant attribute depends on how its interfaces are implemented.

The most concerned functions of reentrant support are malloc() and free() which directly are related to dynamic memory management. If these functions are not reentrant, the information of memory layout will be messed up if there are multiple calls to malloc() or free() at a time.

Newlib maintains information it needs to support each separate context (thread/task/ISR) in a reentrant structure. This includes things like a thread-specific errno, thread-specific pointers to allocated buffers, etc. The active reentrant structure is pointed at by global pointer _impure_ptr, which initially points to a statically allocated structure instance.

Newlib requires below things to complete its reentrant:

  1. Switching context. Multiple reentrant structures (one per context) must be created, initialized, cleaned and pointing upon _impure_ptr to the correct context each time the context is switching

  2. Concurrency protection. For example of using malloc(), it should be lock() and unlock() in that function to make it thread-safe first

FreeRTOS supports Newlib reentrant#

Newlib support has been included by popular demand, but is not used by the FreeRTOS maintainers themselves.

Switching context#

FreeRTOS provides support for Newlib’s context management. In FreeRTOSconfig.h, add:

/* The following flag must be enabled only when using Newlib */
#define configUSE_NEWLIB_REENTRANT           1

By default, STM32 projects generated by STM32CubeIDE use Newlib-nano. Whenever FreeRTOS is enabled, IDE will prompt to enable Newlib Reentrant attribute:

A prompt asking to enable Newlib reentrant

With this option configUSE_NEWLIB_REENTRANT = 1, FreeRTOS does the following (intask.c):

  • For each task, allocate and initialize a Newlib reentrant structure in the task control block
  • Each task switch, set _impure_ptr to point to the newly active task’s reentrant structure
  • On task destruction, clean up the reentrant structure (help Newlib free any associated memory)

Concurrency protection#

There is one more thing to fully support Newlib reentrant: FreeRTOS Memory Management.

FreeRTOS internally uses its own memory management scheme with different heap management implementations in heap_x.c, such as heap_1.c, or heap_4.c.

If an application only uses FreeRTOS-provided memory management APIs such as pvPortMalloc() and vPortFree(), this application is safe for Newlib reentrant, because FreeRTOS suspends the task-switching and interrupts during memory management.

However, many third party libraries do use the standard C malloc() and free() functions. For those cases, the concurrency protection is not guaranteed. That is the reason that Dave Nadler implemented a new heap scheme for Newlib in FreeRTOS. Details in https://nadler.com/embedded/newlibAndFreeRTOS.html

FreeRTOS in STM32CubeMX

The FreeRTOS version shipped in STM32CubeMX does not fully resolve Memory Management for Newlib. Dave Nadler provides a version for STM32 at heap_useNewlib_ST.c. The usage will be covered in a below section.

Non-reentrant cause corrupted output#

F411RE_FreeRTOS_Non-Reentrant.zip

This example project demonstrates an issue when using printf() function without reentrant enabled for Newlib in FreeRTOS. In that case, the data printed out is corrupted.

To setup the project faster, we use STM32CubeMX to configure the projects and generate the source code.

Project setup#

Let’s create a new FreeRTOS application using STM32CubeMX with below settings:

  1. FreeRTOS is enabled with configs:

    • CMSIS-RTOS version: 2.00
    • FreeRTOS version: 10.3.1
    • USE_PREEMPTION: Enabled
    • MINIMAL_STACK_SIZE: 256 Words = 1024 Bytes
    • USE_MUTEXES: Enabled
    • Memory Management Scheme: heap_4

  2. Time base Source for HAL is moved to a general timer, such as TIM10 or TIM11 on STM32F411RE.

  3. Newlib setting:

    • USE_NEWLIB_REENTRANT: Disabled

Create 2 printing tasks#

Add 2 tasks: printTask1 and printTask2 which call to a the same function PrintTask but with different input messages message1 and message2. Note that two tasks have the same priority.

Add 2 printing tasks

Create a Mutex#

We will protect the standard output with a Mutext to ensure that at one time, there is only one Task can print out.

Add a mutex for write access

Define messages and print out#

Two different messages will be prepared. To make the issue happens, the length of messages will be chosen to be long enough, such as 64 bytes.

char* message1 = "................................................................";
char* message2 = "++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++";

The function PrintTask() will print out a message along with the task name, the Reentrant config, an increasing counter to see new messages clearly.

#include "FreeRTOSConfig.h"

void PrintTask(void *argument) {
  char* name = pcTaskGetName(NULL);
  char* message = (char*)argument;
  char counter = 0;
  for(;;) {
    printf("RE=%d %s: %03d %s\r\n",
        configUSE_NEWLIB_REENTRANT,
        name, 
        counter++, 
        message
    );
    osDelay(500);
  }
}

Redirect Standard Output to SWV#

A final step is to redirect printed data to an SWO port. In this function, before printing out, we request to acquire the mutex. When the printing is done, we release the mutex.

int _write(int file, char *ptr, int len) {
    osStatus_t ret;
    // wait for other to complete printing
    ret = osMutexAcquire( writeAccessMutexHandle, osWaitForever );
    if (ret == osOK) {
      int DataIdx;
      for (DataIdx = 0; DataIdx < len; DataIdx++) {
        ITM_SendChar(*ptr++);
      }
      // done our job
      osMutexRelease (writeAccessMutexHandle);
    }

    return len;
}

Compile and Run#

Build the project and run a target board, the output will be messed up as it can be seen that characters in the messages1 is printed in the line of the messages2.

Output of Task 2 is corrupted

Debug#

We should find out how the issue happened.

Place a breakpoint at the beginning of the function _write to check the passing argument char *ptr.


Step 1: Task 2 starts to write, Task 1 has not started yet

Task 2 runs to the _write() function with the argument char *ptr at the address 0x20005210.

Task 2 prints the message at 0x20005210


Step 2: Task 2 is interrupted by Task 1, Task 1 starts to write

When Task 1 runs the _write() function with the argument char *ptr, we notice that the address is still 0x20005210, but the message at that location is changed.

Task 1 prints the message which replaces the content of the Task 2’s message


Step 3: Task 1 is interrupted by Task 2, Task 2 resumes printing

At this step, the content at the address 0x20005210 was overwritten by Task 1, therefore Task 2 will print out corrupted data.

Turn on Newlib reentrant#

Still use the above project, but we set the configuration configUSE_NEWLIB_REENTRANT = 1. Recompile and run the project, the issue is fixed.

Different outputs on different reentrant settings RE

Debug#

Place a breakpoint at the beginning of the function _write to check the passing argument char *ptr.


Step 1: Task 2 starts to write, Task 1 has not started yet

Task 2 runs to the _write() function with the argument char *ptr at the address 0x20005488.

Task 2 prints the message at 0x20005488


Step 2: Task 2 is interrupted by Task 1, Task 1 starts to write

When Task 1 runs the _write() function with the argument char *ptr, we notice that the address is changed to 0x20005a40.

Task 1 prints the message at 0x20005a40


Step 3: Task 1 is interrupted by Task 2, Task 2 resumes printing

At this step, the content at the address 0x20005488 was unchanged therefore Task 2 will print out correct data.

Integrate Newlib memory scheme#

F411RE_FreeRTOS_Reentrant_Heap_Newlib.zip

As mentioned above, Dave Nadler provides a version for STM32 at heap_useNewlib_ST.c. It is not officially supported by ST.

A method to ensure thread-safe for malloc() and free() is to wrap Newlib malloc-like functions to use FreeRTOS’s porting memory management functions. However, FreeRTOS heap implementations do not support realloc().

The heap_usNewlib_ST scheme chooses another method to solve malloc-like functions’ thread-safe. This memory scheme implements thread-safe for malloc() and free() in Newlib, and then overwrites FreeRTOS’s memory function to use Newlib’s functions.

Here are step to integrate heap_usNewlib_ST into STM32 project:

  1. Exclude sysmem.c file from build. This file provides an implementation of _sbrk() which is used by malloc()
  2. Exclude FreeRTOS heap management such as heap_4.c which implements pvPortMalloc() and vPortFree()
  3. Include heap_useNewlib_ST.c to project.

  4. Define 2 configs below to support ISR stack check

    #define configISR_STACK_SIZE_WORDS      (128) // in words
#define configSUPPORT_ISR_STACK_CHECK   ( 1 )

  5. Set reentrant support

    #define configUSE_NEWLIB_REENTRANT      ( 1 )

References#

Comments