Performance of fwrite() on usb mass storage device?
Linux - NewbieThis Linux forum is for members that are new to Linux.
Just starting out and have a question?
If it is not in the man pages or the how-to's this is the place!
Notices
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
If you have any problems with the registration process or your account login, please contact us. If you need to reset your password, click here.
Having a problem logging in? Please visit this page to clear all LQ-related cookies.
Get a virtual cloud desktop with the Linux distro that you want in less than five minutes with Shells! With over 10 pre-installed distros to choose from, the worry-free installation life is here! Whether you are a digital nomad or just looking for flexibility, Shells can put your Linux machine on the device that you want to use.
Exclusive for LQ members, get up to 45% off per month. Click here for more info.
Performance of fwrite() on usb mass storage device?
Hi,
I am trying to write to usb flash drive using a C program. I have just created some files in USB device and doing my tasks using fopen,fread and fwrite on those files. I was just wondering what is the standard data rate that can be achieved using fwrite() on a file in usb. Secondly what is better way of transferring data to USB flash drive using C Program? Obviously I am a newbie
thanks jpollard,
My code reads data in the form a kernel module in the form of 4KB chunks and these chunks are written to files in USB. Expected data rate is 32MB/s and I am using Usb3.0 flash drive. I was wondering whether fwrite() can serve the purpose?
My implementation is i.e. there are 4 files in USB disk.
file0.txt
file1.txt
file2.txt
file3.txt
each time data from kernel module arrives in form of 4KB it has to be appended to a file (in round robin pattern). So I guess cp might not work as it will replace the file with new file. 'cp' takes directory locations (files) as parameter where I have only buffers in my program that will be copied to files.Also each time 'cp' is called fopen and fclose will also be called that might slow down overall operation. So I think I have only choice of fwrite() as per my understanding (which might be questionable ).
Concatenating modules doesn't exactly make any sense - they can't be used that way.
The problem of speed depends on the device. As I understand it, the flash is divided up into rather large blocks - to add a single 4K block to a file requires reading up to 64k, replacing the one 4K update, then writing the 64K somewhere else (wear leveling). So 4K could be rather bad on the device.
And that is much slower than a cp would be, and a lot less overhead (and wear and tear).
But you can use whatever you want. The system will handle it.
Data rates on flash media vary greatly and it depends on the type of write your attempting to do. Since these are 4k blocks your probably doing non sequential writes to the device. In that case then writes will be rather slower in magnitude to reads. USB flash drives are probably the worst for performance in this area considering they are usually a low cost solution. However there is variance even in them with regards to quality of flash memory used.
Like jpollard said writing data to the device in 4k blocks is probably a bad idea because of the general wear and tear on the device ( why putting a swap file on flash is also not good ).
You could however instead of writing to a file could create a buffer for the data until it reaches a particular size and then dump to the disk when full or whatnot. Several ways that could be implemented.
LinuxQuestions.org is looking for people interested in writing
Editorials, Articles, Reviews, and more. If you'd like to contribute
content, let us know.