FileSystem

Key/Value storage filesystem API.

The following provides an interface for a key-value store, implemented using flash memory on the device. It contains functions to initialize and deinitialize the file system, set and retrieve key-value pairs in the file system, find number of keys stored, get key value at a given index, calculate available memory, calculate freeable memory, repack the file system for optimized memory usage, remove an entry, and calculate the wear level of the file system.

Ensure the filesystem is initialized (fs_init() or fs_init_ex()) before usage, and deinitialized (fs_deinit()) after completion of operations to free up resources.

Here is a sample use of the implemented key-value file system, which demonstrates the initialization, usage, and deinitialization of the file system.

Note: At any position of the code, the error checks are omitted for the sake of clarity in the example. Always check return values in real-world implementations!


int main() {
    void* hnd = fs_init(); // Initialize file system

    // The following key-value pairs will be used:
    uint16_t key = 0x1234;
    unsigned char data[] = "Hello, World!";
    uint16_t len = sizeof(data);
    uint16_t max_len = 1024;

    // Set a value associated with a given key in the file system.
    fs_set(hnd, key, data, len);

    // The following variables will hold the fetched data:
    unsigned char fetched_data[1024];
    uint16_t fetched_len;

    // Get a value associated with a given key from the file system.
    fs_get(hnd, key, fetched_data, max_len, &fetched_len);

    // Check how many unique keys are stored in the file system.
    uint16_t num_keys = fs_num_keys(hnd);

    // Fetch the key at the first index.
    uint16_t fetched_key = fs_key_at(hnd, 0);

    // Get available memory
    int avail_memory = fs_available_memory(hnd);

    // Get freeable memory
    int freeable_memory = fs_freeable_memory(hnd);

    // Repack the file system.
    fs_repack(hnd);

    // Remove the key-value pair associated with the given key from the file system.
    fs_remove(hnd, key);

    // Fetch the wear level of the file system
    int wear_level = fs_wear_level(hnd);

    fs_deinit(hnd); // DeInitialize file system

    return 0;
}

Functions Overview

	Name
void *	fs_init_ex(int dev, uint32_t start_address, uint32_t size_bytes, int auto_repack) initialize the key-value-storage filesystem
void *	fs_init(void ) Initializes the file system.
void	fs_deinit(void * hdn) Deinitialize the file system.
void	fs_erase_all(void * hnd) Erases all pages in the file system.
int	fs_get(void * hnd, uint16_t key, unsigned char * data, uint16_t max_len, uint16_t * len) Retrieves data from the file system based on the provided key.
int	fs_set(void * hnd, uint16_t key, unsigned char * data, uint16_t len) Sets the value associated with a given key in the file system.
uint16_t	fs_num_keys(void * hnd) Get the number of keys in the filesystem.
uint16_t	fs_key_at(void * hnd, uint16_t key) Retrieves the key value at the specified index.
int	fs_available_memory(void * hnd) Calculates the available memory in the file system.
int	fs_freeable_memory(void * hnd) Calculates the amount of freeable memory in the file system.
int	fs_repack(void * hnd) Repacks the file system storage to optimize memory usage.
int	fs_remove(void * hnd, uint16_t key) Remove an entry from the file system.
int	fs_wear_level(void * hnd)

Defines

Name
E_FS_OK	the function completed successfully
E_FS_ENTRIES_DIFFER	the compared entries differ
E_FS_NOT_FOUND	a file with the provided key is not found
E_FS_PARAM	the supplied handle is invalid
E_FS_KEY	the supplied key is invalid
E_FS_SIZE	the value exceeds the maximum entry length
E_FS_ALLOC	the filesystem has no more available space
E_FS_RAM	there is not enough ram for the filesystem index
E_FS_CHECKSUM	the retrieved data failed the checksum verification.
E_FS_FLASH	writing to the flash failed

Function Details

function fs_init_ex

cpp

void * fs_init_ex(
    int dev,
    uint32_t start_address,
    uint32_t size_bytes,
    int auto_repack
)

initialize the key-value-storage filesystem

Parameters:

dev handle to a flash device
start_address if 0 is passed, the flash driver uses the first available flash address
size_bytes if 0 is passed, the fs uses the maximum available space.
auto_repack pass 1 if the filesystem should check and perform repacking on every write automatically

Return: handle for using the filesystem

the filesystem always resides at the end of the available flash memory the size gets defined by the linker symbol __fs_reserved_space which in CMake builds can be set via the variable FILESYSTEM_SIZE_BYTES. if the flash has two switchable banks, the size will be doubled, by using banks. Therefore it is vital that the HAL flash configuration correctly defines FLASH_LENGTH as half the total flash size, i.e. the size of a single bank.sa The number of individual store operations before wearout will approximately: (FS_SIZE / (20 + AVG_ITEM_SIZE)) * 10000. For 16-byte values on a 64kB FS with 256B Pages this would be 18.2 million store operations, which translates to one write every 20 seconds over ~12 years

function fs_init

cpp

void * fs_init(
    void 
)

Initializes the file system.

Return: A pointer to the file system handle or NULL if there was an error during initialization.

This function is a convenienc function, that internally calls fs_init_ex with the default flash driver, the default flash address-space and available size. It sets the auto_repack parameter to 1.

function fs_deinit

cpp

void fs_deinit(
    void * hdn
)

Deinitialize the file system.

Parameters:

hdn A pointer to the file system object to be deinitialized.

This function is used to deinitialize the file system and free any allocated resources.

function fs_erase_all

cpp

void fs_erase_all(
    void * hnd
)

Erases all pages in the file system.

Parameters:

hnd Pointer to the file system handle.

Return: None.

Note:

This function assumes that the file system handle has been properly initialized.
This function locks the file system using the mutex before performing any operations.

This function erases all pages in the file system specified by the handle.

function fs_get

cpp

int fs_get(
    void * hnd,
    uint16_t key,
    unsigned char * data,
    uint16_t max_len,
    uint16_t * len
)

Retrieves data from the file system based on the provided key.

Parameters:

hnd Handle to the file system.
key The key used to identify the desired data.
data Pointer to the buffer where the retrieved data will be stored.
max_len The maximum length of the buffer.
len Pointer to a variable that will hold the actual length of the retrieved data.

Return: 0 if successful, otherwise an error code:

E_FS_PARAM if the handle is invalid.
E_FS_OK if the data is successfully retrieved.
E_FS_CHECKSUM if the retrieved data fails the checksum verification.
E_FS_NOT_FOUND if a file with the provided key is not found.

function fs_set

cpp

int fs_set(
    void * hnd,
    uint16_t key,
    unsigned char * data,
    uint16_t len
)

Sets the value associated with a given key in the file system.

Parameters:

hnd The file system handle.
key The key.
data The data to be set.
len The length of the data.

Return: Status code.

E_FS_OK: The value was set successfully.
E_FS_PARAM: Invalid handle parameter.
E_FS_ERROR: An error occurred during the operation.
E_FS_INSUFFICIENT_MEMORY: Insufficient memory to set the value.

This function sets the value associated with the specified key in the file system. If auto-repacking is enabled and the available memory is less than or equal to the page size, the file system will be repacked before setting the value.

function fs_num_keys

cpp

uint16_t fs_num_keys(
    void * hnd
)

Get the number of keys in the filesystem.

Parameters:

hnd A pointer to the handle of the filesystem.

Returns:

E_FS_PARAM The handle is NULL.

Return: The number of keys in the filesystem's index.

This function retrieves the number of keys in the filesystem. It checks if the given handle is valid and returns the number of keys stored in the filesystem's index.

function fs_key_at

cpp

uint16_t fs_key_at(
    void * hnd,
    uint16_t key
)

Retrieves the key value at the specified index.

Parameters:

hnd Pointer to the file system object handle.
key The index of the key value to retrieve.

Return: The key value at the specified index, or 0 if the index is out of range or if the file system handle is invalid.

This function retrieves the key value at the specified index in the file system.

function fs_available_memory

cpp

int fs_available_memory(
    void * hnd
)

Calculates the available memory in the file system.

Parameters:

hnd Pointer to the file system handle.

Return: The amount of available memory in bytes.

This function calculates the available memory in the file system based on the provided file system handle. It iterates over the file system regions and pages to determine the amount of free space available.

function fs_freeable_memory

cpp

int fs_freeable_memory(
    void * hnd
)

Calculates the amount of freeable memory in the file system.

Parameters:

hnd A pointer to the file system handle.

Return: The amount of freeable memory in bytes.

This function iterates over all the pages in the file system, checks if the page is empty, and calculates the total amount of memory that can be freed by deleting inactive entries.

function fs_repack

cpp

int fs_repack(
    void * hnd
)

Repacks the file system storage to optimize memory usage.

Parameters:

hnd Pointer to the file system instance.

Return: Returns E_FS_OK if the repack operation is successful, otherwise returns an error code.

This function repacks the file system storage by iterating over all the index entries, reading the data from the old location, and writing it back to a new location. It also checks for any inactive pages and frees them to optimize memory usage.

function fs_remove

cpp

int fs_remove(
    void * hnd,
    uint16_t key
)

Remove an entry from the file system.

Parameters:

hnd Pointer to the file system handler.
key The key of the entry to remove.

Return: Returns E_FS_OK on success. Returns E_FS_PARAM if hnd is NULL.

This function removes an entry with the specified key from the file system. The entry is invalidated and its corresponding page is freed if it no longer contains active entries.

function fs_wear_level

cpp

int fs_wear_level(
    void * hnd
)

Parameters:

hnd Pointer to the file system instance.

Return: The wear level of the file system as an integer value.

Calculates the wear level of the file system.

This function calculates the wear level of the file system by summing up the difference between the maximum erase count value (0xFFFFFFFF) and the page erase count for each page in all the regions of the file system. The wear level is then calculated as the sum divided by the total number of pages, multiplied by 100. The wear level is rounded up to the nearest integer and returned as an integer value.

Macros Documentation

define E_FS_OK

cpp

#define E_FS_OK (0)

the function completed successfully

define E_FS_ENTRIES_DIFFER

cpp

#define E_FS_ENTRIES_DIFFER (-1)

the compared entries differ

define E_FS_NOT_FOUND

cpp

#define E_FS_NOT_FOUND (-2)

a file with the provided key is not found

define E_FS_PARAM

cpp

#define E_FS_PARAM (-3)

the supplied handle is invalid

define E_FS_KEY

cpp

#define E_FS_KEY (-4)

the supplied key is invalid

define E_FS_SIZE

cpp

#define E_FS_SIZE (-5)

the value exceeds the maximum entry length

define E_FS_ALLOC

cpp

#define E_FS_ALLOC (-6)

the filesystem has no more available space

define E_FS_RAM

cpp

#define E_FS_RAM (-7)

there is not enough ram for the filesystem index

define E_FS_CHECKSUM

cpp

#define E_FS_CHECKSUM (-8)

the retrieved data failed the checksum verification.

define E_FS_FLASH

cpp

#define E_FS_FLASH (-9)

writing to the flash failed

FileSystem ​

Functions Overview ​

Defines ​

Function Details ​

function fs_init_ex ​

function fs_init ​

function fs_deinit ​

function fs_erase_all ​

function fs_get ​

function fs_set ​

function fs_num_keys ​

function fs_key_at ​

function fs_available_memory ​

function fs_freeable_memory ​

function fs_repack ​

function fs_remove ​

function fs_wear_level ​

Macros Documentation ​

define E_FS_OK ​

define E_FS_ENTRIES_DIFFER ​

define E_FS_NOT_FOUND ​

define E_FS_PARAM ​

define E_FS_KEY ​

define E_FS_SIZE ​

define E_FS_ALLOC ​

define E_FS_RAM ​

define E_FS_CHECKSUM ​

define E_FS_FLASH ​

FileSystem

Functions Overview

Defines

Function Details

function fs_init_ex

function fs_init

function fs_deinit

function fs_erase_all

function fs_get

function fs_set

function fs_num_keys

function fs_key_at

function fs_available_memory

function fs_freeable_memory

function fs_repack

function fs_remove

function fs_wear_level

Macros Documentation

define E_FS_OK

define E_FS_ENTRIES_DIFFER

define E_FS_NOT_FOUND

define E_FS_PARAM

define E_FS_KEY

define E_FS_SIZE

define E_FS_ALLOC

define E_FS_RAM

define E_FS_CHECKSUM

define E_FS_FLASH