Re: Flash drive internal fragmentation.
R. Clayton wrote:
> <<--deleted background details--
>
> $ ls --block-size=1k -ls
> total 896
> 64 -rwxr-xr-x 1 rclayton rclayton 18 2010-03-23 14:42 branniga.htm
> 64 -rwxr-xr-x 1 rclayton rclayton 3 2010-03-23 14:42 fig1.gif
> 64 -rwxr-xr-x 1 rclayton rclayton 1 2010-03-23 14:42 fig1.htm
> 64 -rwxr-xr-x 1 rclayton rclayton 3 2010-03-23 14:42 fig2.gif
> 64 -rwxr-xr-x 1 rclayton rclayton 1 2010-03-23 14:42 fig2.htm
> 64 -rwxr-xr-x 1 rclayton rclayton 3 2010-03-23 14:42 fig3.gif
> 64 -rwxr-xr-x 1 rclayton rclayton 1 2010-03-23 14:42 fig3.htm
> 64 -rwxr-xr-x 1 rclayton rclayton 6 2010-03-23 14:42 fig4.gif
> 64 -rwxr-xr-x 1 rclayton rclayton 1 2010-03-23 14:42 fig4.htm
> 64 -rwxr-xr-x 1 rclayton rclayton 5 2010-03-23 14:42 fig5.gif
> 64 -rwxr-xr-x 1 rclayton rclayton 1 2010-03-23 14:42 fig5.htm
> 64 -rwxr-xr-x 1 rclayton rclayton 7 2010-03-23 14:42 fig6.gif
> 64 -rwxr-xr-x 1 rclayton rclayton 1 2010-03-23 14:42 fig6.htm
> 64 -rwxr-xr-x 1 rclayton rclayton 5 2010-03-23 14:42 list1.htm
>
> $ cd /home/dr-dobbs/articles/CUJ/1990/9001/branniga
>
> $ ls --block-size=1k -ls
> total 88
> 20 -r-x------ 1 rclayton rclayton 18 2010-03-21 08:32 branniga.htm
> 4 -r-x------ 1 rclayton rclayton 3 2010-03-21 08:32 fig1.gif
> 4 -r-x------ 1 rclayton rclayton 1 2010-03-21 08:32 fig1.htm
> 4 -r-x------ 1 rclayton rclayton 3 2010-03-21 08:32 fig2.gif
> 4 -r-x------ 1 rclayton rclayton 1 2010-03-21 08:32 fig2.htm
> 4 -r-x------ 1 rclayton rclayton 3 2010-03-21 08:32 fig3.gif
> 4 -r-x------ 1 rclayton rclayton 1 2010-03-21 08:32 fig3.htm
> 8 -r-x------ 1 rclayton rclayton 6 2010-03-21 08:32 fig4.gif
> 4 -r-x------ 1 rclayton rclayton 1 2010-03-21 08:32 fig4.htm
> 8 -r-x------ 1 rclayton rclayton 5 2010-03-21 08:32 fig5.gif
> 4 -r-x------ 1 rclayton rclayton 1 2010-03-21 08:32 fig5.htm
> 8 -r-x------ 1 rclayton rclayton 7 2010-03-21 08:32 fig6.gif
> 4 -r-x------ 1 rclayton rclayton 1 2010-03-21 08:32 fig6.htm
> 8 -r-x------ 1 rclayton rclayton 5 2010-03-21 08:32 list1.htm
>
> $
>
> which meant my thumb drive space was getting eaten up by internal
> fragmentation. I understand that flash storage has special needs for most
> efficient operation, but are block-size overages of an order-of-magnitude or
> more really necessary? Can I remake the file system with smaller block sizes?
> How? (It is unclear to me if sector size is the same as file block size.)
> What bad things happen to flash storage if the file system has 1k block size?
>
> This happened on a debian testing system updated weekly.
>
>
Curious ... I have a 4G flash drive and have had no issues with it,
except for file deletion using a GUI file manager.
And you didn't mention if the stick you're using was brand new/never
used or not. If you've had files on it and then deleted them using a
GUI file browser, it may have simply moved them to a hidden trash
folder, which would not free up any space.
But you may have already checked that ;)
I tried the 'ls' command you used on my flash drive, and got this:
$ ls --block-size=1k -ls
total 23024
500 -rwxrwxr-x 1 root backup 498 2010-02-22 18:11 C1.jpg
644 -rwxrwxr-x 1 root backup 643 2010-02-22 18:11 C2.jpg
1160 -rwxrwxr-x 1 root backup 1159 2010-02-22 18:11 C3.jpg
852 -rwxrwxr-x 1 root backup 852 2010-02-22 18:11 C4.jpg
4896 -rwxrwxr-x 1 root backup 4895 2010-03-07 10:36 C5.jpg
3564 -rwxrwxr-x 1 root backup 3561 2010-03-07 10:45 C6.jpg
416 -rwxrwxr-x 1 root backup 416 2010-03-14 16:35 C7.jpg
5076 -rwxrwxr-x 1 root backup 5076 2010-03-13 02:43 C8.jpg
5916 -rwxrwxr-x 1 root backup 5913 2010-03-13 03:04 C9.jpg
And ...
$ ls --block-size=8k -ls
total 2878
63 -rwxrwxr-x 1 root backup 63 2010-02-22 18:11 C1.jpg
81 -rwxrwxr-x 1 root backup 81 2010-02-22 18:11 C2.jpg
145 -rwxrwxr-x 1 root backup 145 2010-02-22 18:11 C3.jpg
107 -rwxrwxr-x 1 root backup 107 2010-02-22 18:11 C4.jpg
612 -rwxrwxr-x 1 root backup 612 2010-03-07 10:36 C5.jpg
446 -rwxrwxr-x 1 root backup 446 2010-03-07 10:45 C6.jpg
52 -rwxrwxr-x 1 root backup 52 2010-03-14 16:35 C7.jpg
635 -rwxrwxr-x 1 root backup 635 2010-03-13 02:43 C8.jpg
740 -rwxrwxr-x 1 root backup 740 2010-03-13 03:04 C9.jpg
It looks like the numbers are matching up OK (at least on my system - ls
--version says "ls (GNU coreutils) 5.97" and still using etch [4.0]).
A couple of other differences:
My flash drive has no partition table, the filesystem is built on the
base device, like a floppy disk.
And, I used VFAT for the filesystem type:
$ mount|grep sde
/dev/sde on /var/autofs/usb/centon type vfat ...
I don't know if any of this is really helpful, though it may suggest an
alternative or two to play around with.
--
Bob McGowan
Reply to: