Sorting the SD card easily

This section is for discussing everything about Next hardware and latest updates.
AndyCalling
Posts: 28
Joined: Thu Feb 27, 2020 9:35 pm

Sorting the SD card easily

Postby AndyCalling » Fri Jan 22, 2021 12:22 am

OK, so I mentioned this in another thread but I think it deserves its own.

I used to have an old MP3 player that didn't sort files. To solve the issue, I used a utility like this:

http://www.anerty.net/software/file/DriveSort/?lang=en

I am still awaiting my new Next, but I have been told it shows files unsorted, so I suspect in raw FAT order. I have been advised to prep my games etc. in advance to write all at once in order, so the Next sees them sorted. So, I am thinking my old trick should work.

Could someone lucky enough to have a Next try this utility out on its card and report back? Does the Next do as I suspect, and see a sorted SD card once this utility is used?

Thanks in advance for your input, spechums.

Ped7g
Posts: 256
Joined: Mon Jul 16, 2018 7:11 pm

Re: Sorting the SD card easily

Postby Ped7g » Fri Jan 22, 2021 3:01 pm

I'm not going to try the utility (linux here), but I can confirm all your assumptions are correct, the Next does use FAT16 or FAT32 cards (FAT12 works too IIRC, but I'm not 100% sure), but not exFAT (!). (and for formatting use the suggested tool from the https://www.specnext.com/latestdistro page, as the windows built-in formatter seems to be not compatible).

The NextZXOS "browser" utility allowing you to navigate over card content in visual form displays content "as is", in the raw FAT order (as it is quite heavy computational task to read whole directory and sort it ... I mean this is not sarcasm, if you have like 5000 games in single directory, what is very fast and simple on modern PC does suddenly turn into full-time-job for Next, reading and sorting such directory would require most of the Next's memory and take probably from a second to several seconds, even minutes if programmed in suboptimal way).

Tools like "DriveSort" manipulating with the FAT table will help.

User avatar
Eratosthenes
Posts: 317
Joined: Mon May 29, 2017 8:17 pm

Re: Sorting the SD card easily

Postby Eratosthenes » Fri Jan 22, 2021 10:38 pm

@Ped7g Exactly

Running Linux it's just three steps and very simple commands to sort the file allocation table of a partition. I'm using fatsort which is part nearly every software repository.

Here a simple "HowTo".

After installing it, e.g. on Ubuntu via

Code: Select all

sudo apt install fatsort
the command is very simple.


Here a sample - The label of my sd card is "NEXTBOARD". It's inserted and can be read...
  • Identify the device name of the partition

Code: Select all

mount|grep -i nextboard
The respond is:
/dev/sdc1 on /media/aramis/NEXTBOARD type vfat (rw,nosuid,nodev,r...
/dev/sdc1 is the correct devicename.
  • Unmount it

Code: Select all

sudo umount /dev/sdc1
  • Sort the file allocation table (*1)

Code: Select all

sudo fatsort /dev/sdc1
Once you have identified the device name, you only need to unmount the card using the "eject- or unmount" menuitem on the desktop and execute fatsort as shown above.

Sorting the card of my Next takes about 4s. On the SD card 1.7GB of data are used. It contains aprox. 30.000 files as I have installed the whole WoS archive on the device.

(*1): fatsort has a bunch of parameters if you like it to do a more specific kind of sort.
By default all directories are sorted on top and the sort is case insensitive. Filenames starting with a dot (".") are sorted by the 2nd character. The defaults depend on the locale and other settings of the system and may be different on your PC/Mac.

A funny parameter is "-R" as it does a random order... ;-)

(*2) What should be mentioned?
WARNING: THE FILESYSTEM MUST BE CONSISTENT (NO FILESYSTEM ERRORS).
PLEASE BACKUP YOUR DATA BEFORE USING FATSORT. RISK OF CORRUPT FILESYSTEM!
FATSORT USER ASSUMES ALL RISK. FATSORT WILL NOT BE HELD LIABLE FOR DATA LOSS!

AndyCalling
Posts: 28
Joined: Thu Feb 27, 2020 9:35 pm

Re: Sorting the SD card easily

Postby AndyCalling » Sat Jan 23, 2021 1:51 am

Great, ta folks for confirming my thinking. Well, that's sorted then. Pun fully intended.

-R, heh, whomever wrote that command, I like the cut of their jib.

User avatar
sol_hsa
Posts: 273
Joined: Fri Jun 02, 2017 10:10 am
Location: Finland
Contact:

Re: Sorting the SD card easily

Postby sol_hsa » Sun Jan 24, 2021 7:31 am

The random order does have its uses. If nothing else, to test the tool itself. I can also see benchmarking use cases..

Magnus
Posts: 6
Joined: Thu May 21, 2020 1:48 pm

Re: Sorting the SD card easily

Postby Magnus » Mon Jan 25, 2021 6:15 pm

Thanks for the link to DriveSort! Nice to have things sorted :) .

User avatar
Andy P
Posts: 138
Joined: Mon May 29, 2017 9:15 pm
Location: Lancashire

Re: Sorting the SD card easily

Postby Andy P » Fri Feb 26, 2021 2:23 am

Eratosthenes wrote:
Fri Jan 22, 2021 10:38 pm

Running Linux it's just three steps and very simple commands to sort the file allocation table of a partition. I'm using fatsort which is part nearly every software repository.
Wow thanks for that. The good old Raspberry PI is what I use these days and It is in the PI OS (Raspbian) repository which is fantastic.
It took more like a minute to sort them all but there were thousands of unsorted TAP & TZX files.

User avatar
Eratosthenes
Posts: 317
Joined: Mon May 29, 2017 8:17 pm

Re: Sorting the SD card easily

Postby Eratosthenes » Sun Feb 28, 2021 3:43 pm

Nice to read @Andy P that also PI OS provides this package. I think I should install it onto my PI on the Next. Doing so I think it should be possible that the Next sorts it's card by itself... :roll:

I've got the whole World Of Spectrum Archive on my SD cards for the Spectrum. On my PC it usually takes about 20s or so to sort the FAT of such an SD card.

Alcoholics Anonymous
Posts: 775
Joined: Mon May 29, 2017 7:00 pm

Re: Sorting the SD card easily

Postby Alcoholics Anonymous » Sun Feb 28, 2021 6:35 pm

Eratosthenes wrote:
Sun Feb 28, 2021 3:43 pm
Nice to read @Andy P that also PI OS provides this package. I think I should install it onto my PI on the Next. Doing so I think it should be possible that the Next sorts it's card by itself... :roll:
It probably can. Enter ".ls -alR" at the root of the sd card to see how long it takes to sort your directories. Sorting is done on the long filenames -- you have to change the options to show them in the listing.

In the official distro, .ls is using a non-naive quicksort but it only runs at 14MHz. In gitlab (3.01.08), .ls is using shellsort but at 28MHz. It should be rebuilt to use quicksort as well but the person compiling didn't realize there is a setting for that.

It is risky to sort sd cards though.

User avatar
Eratosthenes
Posts: 317
Joined: Mon May 29, 2017 8:17 pm

Re: Sorting the SD card easily

Postby Eratosthenes » Mon Mar 01, 2021 9:25 pm

If you have so many SD cards as I have, and if you know that esxDOS doesn't have so many issues writing the FAT stuff, you would sort it without having a bad feeling... :D

I'll give it a try I think.

P.S.: The Next's FAT stuff is based on the esxDOS routines?! *smile


Who is online

Users browsing this forum: No registered users and 8 guests