ESP32 NVS Keys Part 2

In a previous post, I investigated the limits to what NVS accepted as a valid ASCII string. While I’m certain I did not iterate over ALL the acceptable possibilities, the following question arose: For the purpose of NVS, what features make keys considered identical?

The first string to consider is naïve, specifically:

const char key_0[] = "naïve";

An unrelated warning in the compiler clues us in that something is a bit off about this string.

warning: unused variable 'key_0' [-Wunused-variable] const char key_0[] = "naïve";

Let’s attempt to construct an equivalent string for the purpose of NVS.

const char key_1[] = "naïve";

Storing in key (key_0)... DONE
Value in key (key_0): naïve text is fine
Error (ESP_ERR_NVS_NOT_FOUND) retrieving value in key (key_1)!

That doesn’t work. Let’s output the value of key_0 and key_1 to the console.

Key key_0 is: naïve
Key key_1 is: naïve

In spite of the previous “warning”, it appears that the key is properly represented, somehow, in the ESP32’s memory and is properly output. Let’s try constructing an equivalent string.

char key_1[16] = "na";
strcat(key_1, "ï");
strcat(key_1, "ve");

Value in key (key_0): naïve text is fine
Value in key (key_1): naïve text is fine
Key key_0 is: naïve
Key key_1 is: naïve

Let’s try something a bit more Icarus-like (as in flying as close to the sun as possible).

char key_1[16] = "naive";
key_1[2] = 'ï';

Error (ESP_ERR_NVS_NOT_FOUND) retrieving value in key (key_1)!
Key key_0 is: naïve
Key key_1 is: na�ve

That compiling error showed up again but with some extra information.

warning: unsigned conversion from 'int' to 'char' changes value from '50095' to '175' [-Woverflow]

Let’s dump the string values directly to hex.

key_0 values are: 6e 61 c3 af 76 65

const char key_0[] = "naïve";
const char key_1[] = { 0x6e, 0x61, 0xc3, 0xaf, 0x76, 0x65, 0x00 };

Value in key (key_0): naïve text is fine
Value in key (key_1): naïve text is fine
Key key_0 is: naïve
Key key_1 is: naïve

That set of hex values implies this should work:

const char key_1[] = "naïve";

It doesn’t work, and the hex values in key_1 are different than we would expect.

Error (ESP_ERR_NVS_NOT_FOUND) retrieving value in key (key_1)!
Key key_0 is: naïve
Key key_1 is: naïve
key_1 values are: 6e 61 c3 83 c2 af 76 65

From this, I am concluding that two different things are happening: characters outside of the standard ASCII (0x00 – 0x7f) are being expanded into multi-byte values. The string and character-handling functions are properly handling these scenarios, but the NVS-related functions are operating at a lower level perhaps even treating the strings as byte arrays or something similar. Let’s test that out.

const char key_0[] = "naïve_is_fine_?";

Storing in key (key_0)... Error (ESP_ERR_NVS_KEY_TOO_LONG) storing value in key (key_0)!

What about 0x00?

const char key_0[] = "test\0pass";
const char key_1[] = "test\0fail";

Value in key (key_0): naïve text is fine
Value in key (key_1): naïve text is fine

Key key_0 is: test
Key key_1 is: test

key_0 values are: 74 65 73 74 0 70 61 73 73
key_0 characters are: is: t e s t  p a s s

key_1 values are: 74 65 73 74 0 66 61 69 6c
key_1 characters are: t e s t  f a i l

I had to play very dirty with memory to have this show up. The character encoding system using quotes WILL encode 0x00 into the string. Dumping the raw values demonstrates that the characters after are not lost. However, the NVS system ignores anything after the initial NUL character is encountered. Consider the following and their output:

const char key_0[] = { 0x00, 0x68, 0x65, 0x6c, 0x6c, 0x6f }; // \0hello
const char key_1[] = { 0x00, 0x77, 0x6f, 0x72, 0x6c, 0x64 }; // \0world

Value in key (key_0): naïve text is fine
Value in key (key_1): naïve text is fine
key_0 values are:
key_1 values are:

The NUL, or 0x00, or ‘\0’ special character masks data after it for the purpose of NVS reading/writing. The characters, however, are still there and could potentially be used.