fs_write failed

[AydinyanNarek]

Discussed in #70163

^{Originally posted by AydinyanNarek March 13, 2024}
Hello,

Introdcution

I am using STM32 microcontroller and Zephyr 3.6.0.
My application logger logs data to the Ram-FS, instead of flash memory. (Flash memory has a fixed count of write operations).
I have created Ram-FS(/etc and /tmp) using FALSH_SIMULATOR.
It looks like this.

/ {
	soc {
		flash_controller0: flash-controller@0 {
			compatible = "zephyr,sim-flash";
			reg = <0x00000000 DT_SIZE_K(64)>;

			#address-cells = <1>;
			#size-cells = <1>;
			erase-value = <0xff>;

			flash_sim0: flash_sim@0 {
				status = "okay";
				compatible = "soc-nv-flash";
				erase-block-size = <512>;
				write-block-size = <1>;
				reg = <0x00000000 DT_SIZE_K(64)>;

				partitions {
					compatible = "fixed-partitions";
					#address-cells = <1>;
					#size-cells = <1>;

					lfsram_partition0: partition@0 {
						label = "/tmp";
						reg = <0x00000000 0x00008000>;
					};

					lfsram_partition1: partition@1 {
						label = "/etc";
						reg = <0x00008000 0x00010000>;
					};
				};
			};
		};
	};
};

    fstab {
        compatible = "zephyr,fstab";
        lfs1: lfs1 {
			compatible = "zephyr,fstab,littlefs";
			read-size = <512>;
			prog-size = <512>;
			cache-size = <4096>;
			lookahead-size = <512>;
			block-cycles = <512>;
            partition = <&storage_partition>;
			mount-point = "/opt";
			automount;
        };

          lfsram0: lfsram0 {
	          compatible = "zephyr,fstab,littlefs";
	          mount-point = "/tmp";
	          partition = <&lfsram_partition0>;
	          automount;
	          read-size = <16>;
	          prog-size = <16>;
	          cache-size = <64>;
	          lookahead-size = <32>;
	          block-cycles = <512>;
          };
          
          lfsram1: lfsram1 {
	          compatible = "zephyr,fstab,littlefs";
	          mount-point = "/etc";
	          partition = <&lfsram_partition1>;
	          automount;
	          read-size = <16>;
	          prog-size = <16>;
	          cache-size = <64>;
	          lookahead-size = <32>;
	          block-cycles = <512>;
          };
    };

Notes.

1. /opt is using flash0 memory. (just skip this)
2. The combined log size is less than 10kb, which is less then the /tmp partition size
3. I am not writing 10kb log at once. I am doing small logs (max 512 bytes at a time).

Problems.

I am writing logs to /tmp directory using std::ofstream

1. The file remains empty Until I call a close function on the stream. (std::endl, std::flash, rdbuff(0,0,) ..etc doesn't help)
2. Using the open\close combination after each write helps the data to appear in the file, however when I am doing it multiple times(let's say more than 50) the fs_write returns error code -22 which is INVALID ARGUMENT ERROR and printing errno returns The Socket is not available.

Does someone have an idea about this ?

[de-nordic]

Somebody correct me if I am wrong, but the ofstream, indirectly, is supported in Zephyr by Posix subsys, which seems to not implement the sync. Also, I have searched for fs_sync callers in Zephyr and SDK and did not found anything that looks like std implementation to call the function.
fs_write by itself does not return the error, it comes from LittleFS logic. The LFS write also does not directly return this, need to get debug to figure out which of internally called function return this.
Btw block-cycles for SRAM should be something negative to turn off the wear leveling.

[AydinyanNarek]

It seems I found an issue.
First of all this is a call stack

#0  flash_sim_erase (dev=0x80b7fdc <__device_dts_ord_54>, offset=65792, len=256)
    at /home/nareka/projects/vts-0220/trunk/zephyr/drivers/flash/flash_simulator.c:306
#1  0x080924aa in lfs_api_erase (c=<optimized out>, block=<optimized out>)
    at /home/nareka/projects/vts-0220/trunk/zephyr/subsys/fs/littlefs_fs.c:198
#2  0x08058344 in lfs_ctz_extend (off=0x20060738 <_k_mem_slab_buf_file_data_pool+360>, block=0x20060734 <_k_mem_slab_buf_file_data_pool+356>, 
    size=3889, head=127, rcache=0x20002bdc <fs_data_2+116>, pcache=0x2006073c <_k_mem_slab_buf_file_data_pool+364>, lfs=0x20002bdc <fs_data_2+116>)
    at /home/nareka/projects/vts-0220/trunk/modules/fs/littlefs/lfs.c:2890
#3  lfs_file_flushedwrite (lfs=lfs@entry=0x20002bdc <fs_data_2+116>, file=file@entry=0x200606fc <_k_mem_slab_buf_file_data_pool+300>, 
    buffer=buffer@entry=0x20073a88, size=size@entry=106) at /home/nareka/projects/vts-0220/trunk/modules/fs/littlefs/lfs.c:3553
#4  0x0809b67a in lfs_file_rawwrite (lfs=lfs@entry=0x20002bdc <fs_data_2+116>, file=0x200606fc <_k_mem_slab_buf_file_data_pool+300>, 
    buffer=buffer@entry=0x20073a88, size=size@entry=106) at /home/nareka/projects/vts-0220/trunk/modules/fs/littlefs/lfs.c:3637
#5  0x0809b920 in lfs_file_write (lfs=lfs@entry=0x20002bdc <fs_data_2+116>, file=<optimized out>, buffer=buffer@entry=0x20073a88, 
    size=size@entry=106) at /home/nareka/projects/vts-0220/trunk/modules/fs/littlefs/lfs.c:6036
#6  0x08092570 in littlefs_write (fp=0x2001fc48 <desc_array>, ptr=0x20073a88, len=106)
    at /home/nareka/projects/vts-0220/trunk/zephyr/subsys/fs/littlefs_fs.c:388
#7  0x08044d06 in fs_write (zfp=<optimized out>, ptr=<optimized out>, size=<optimized out>)
    at /home/nareka/projects/vts-0220/trunk/zephyr/subsys/fs/fs.c:232
--Type <RET> for more, q to quit, c to continue without paging--ret
#8  0x080921fa in fs_write_vmeth (obj=<optimized out>, buffer=<optimized out>, count=<optimized out>)
    at /home/nareka/projects/vts-0220/trunk/zephyr/lib/posix/options/fs.c:184
#9  0x0803ed6a in write (fd=<optimized out>, buf=0x20073a88, sz=106) at /home/nareka/projects/vts-0220/trunk/zephyr/lib/os/fdtable.c:329
#10 0x0809d84a in (anonymous namespace)::xwrite(int, char const*, int) ()
#11 0x08064682 in std::basic_filebuf<char, std::char_traits<char> >::_M_convert_to_external(char*, int) ()
#12 0x080a4750 in std::basic_filebuf<char, std::char_traits<char> >::overflow(int) ()
#13 0x080a432c in std::basic_filebuf<char, std::char_traits<char> >::_M_terminate_output() ()
#14 0x080a43ca in std::basic_filebuf<char, std::char_traits<char> >::close() ()
#15 0x080a45f6 in std::basic_ofstream<char, std::char_traits<char> >::close() ()

In /zephyr/drivers/flash/flash_simulator.c file in the function flash_sim_erase it ischecking for flash_range_is_valid, and in
flash_range_is_valid function it fails on offset + len > FLASH_SIMULATOR_FLASH_SIZE + FLASH_SIMULATOR_BASE_OFFSET) condition.

Now let's look in my device tree again

/ {
	soc {
		flash_controller0: flash-controller@0 {
			compatible = "zephyr,sim-flash";
			reg = <0x00000000 DT_SIZE_K(64)>;

			#address-cells = <1>;
			#size-cells = <1>;
			erase-value = <0xff>;

			flash_sim0: flash_sim@0 {
				status = "okay";
				compatible = "soc-nv-flash";
				erase-block-size = <512>;
				write-block-size = <1>;
				reg = <0x00000000 DT_SIZE_K(64)>;

				partitions {
					compatible = "fixed-partitions";
					#address-cells = <1>;
					#size-cells = <1>;

					lfsram_partition0: partition@0 {
						label = "/tmp";
						reg = <0x00000000 0x00008000>;
					};

					lfsram_partition1: partition@1 {
						label = "/etc";
						reg = <0x00008000 0x00010000>;
					};
				};
			};
		};
	};
};

When I decrease erase-block-size to 256 it is not failing, but it is not the solution. From all of this, I can assume that when I log 2 x 50 bytes of data it will log in two blocks instead of 1 block. So I am wasting 512 bytes of storage for storing 100 bytes of data. (Please correct me if I am wrong).
I think the ideal case will be to set erase-block-size to be 1, 4, 8, or MAX 16 but when I set this the filesystem wasn't being mounted. There is a relationship between erase-block-size and other values set in dts file. However, I think that I can change some code in Zephyr's kernel but I don't want to do that because of portability.
Do you have an idea how can I decrease the value for erase-block-size to be minimal, manipulating only the device tree?

PS.
For now, let's skip the part about why std::flush doesn't write the data to the file, I think that can be opened in another thread.

[Laczen]

@AydinyanNarek your second partition is to large to fit in the specified 64kB of the simulator, reduce the size of the second partition or increase the simulator size.

[AydinyanNarek]

@Laczen what do you mean saying second partition ?

[Laczen]

@Laczen what do you mean saying second partition ?

lfsram_partition1 starts at 32kB and is 64kB, so this needs 96kB.

[AydinyanNarek]

@Laczen what do you mean saying second partition ?

lfsram_partition1 starts at 32kB and is 64kB, so this needs 96kB.

Thanks, it works now.
I made lfsram_partition0 and lfsram_partition1 to be 32kb each and in that case, I can set erase-block-size to be 64kb.

Should I open an issue related to std::flush related functional?
The problem is that std::ostream internal buffer's(rdbuf) data will not be written to the file without closing the file.