Guidance on implementing FAT file system?

[SamurAchzar]

Woo, I'm back... here we go:

1) Sorry, don't have a floppy driver. My system is all-flash based. It should be pretty straightforward though - a block device is merely a device with read/write/open/close/seek functions. Nothing more, nothing else.
You could probably do it very easily through BIOS interrupts, but I'm by no means an expert for PC programming.

2) You are correct in your understanding of the FAT table, it's exactly like that.

3) No hidden entries in the FAT. completely linear in this regard.

4) The entries in the directory list the beginning cluster of their associated data.
You just seek using the FAT. In my code fat_vol.c provides some low-level lookups for finding the FAT location associated with a specific data cluster, with regard to FAT offset on the disk etc. - that should be useful for you. I strongly recommend that you closely read that file - it's quite simple and contains all the annoying little tidbits you need.

5) The root directory is a pain. Generally, it does NOT use the FAT table, as it's contiguous.
It's comprised of a fixed number of clusters and immediately follows the FAT table.
You can see I've included somewhat different handling when it comes to root directory entries. Again, check fat_vol.c for that.

6) Yep, sector #2 after the FAT. I don't remember the size of the table off the top of my head, but it's of course fixed.

So basically, fat_vol.c contains the key to all your low level annoying problems, read that in you should have most of the stuff figured out. Not that I don't enjoy helping you guys out

Hope that helps, good luck

 

[Jim Bancroft]

Hi Samur,

Just a quick update for your (and anyone who's interested's) reading pleasure. At this moment, my partner and I have a basic-basic-basic FAT12 system taking shape. We're using a floppy image on a hard drive as our block device, with system calls serving to read and write to it. I admit it's cheap and wheezy but we're waiting for our floppy driver team to give us an API, so meantime it'll have to do.

We've finished our initialization routines, with ideas borrowed from your fat_vol design. Our structures are much simpler and geared exclusively toward FAT12. In particular the method you used for discovering a file's clusters-- shift left four bits on an even cluster, divide by two on an odd, something like that-- helped big time. I was floundering on how to do this when reading the Microsoft FAT whitepaper.

We've just completed our open() and read() & seek() routines, mirrored after system calls. Now we can pass a filename into open(), scan the FAT, build a linked list of clusters pertaining to the file, load those clusters' data into a char* buffer, and let the user do what they like from there.

My partner is working on a write() routine to save any changes made. I'm thinking a good way might be to compare the file buffer's size with the number of clusters currently used by the file? Maybe overwrite the clusters and allocate new ones if needed, or give back old ones if the size has shrunk? Does this sound ok, or do you have a better idea?

I've just finished a truncate option that can be used in the open() routine. It builds a cluster chain, just like when we open a file, only then we erase all cluster data, zero the FAT entries for the clusters (so they're freed up), and update the file's directory info, setting its starting cluster to zero.

Still plenty of kinks to work out for everything we've written, and more still to write, but the basics appear to be functioning correctly. If you have any advice or see big holes in what we've done, potential problems etc., by all means speak your mind. Otherwise, thanks for the 10th time and I'll post another update when we've made progress.

 

[SamurAchzar]

Hey,

Sorry for the late reply as I'm somewhere in San Diego right now (away, away from home).

Seems like you've done a good job so far, that great. Few things:

1) Perhaps storing the file data as a buffer isn't the best thing to do. What happens when the user writes more data to it? You need to extend the buffer in a contiguous way, and we know you can't do that.
The user shouldn't have an access to a data buffer, only to read()/write()/seek().
If you'd like, you could cache the data in a buffer on the system side - but the user shouldn't be granted any access to it.
You're thinking as a USER right now, when you write such code you need to think as the SYSTEM Obviously, we users would love to have the entire file buffered when we open() it, but from a system point of view, this isn't possible/practical

2) The user can't shrink a file by issuing a write() command - he should manually truncate() it to a certain size. Sounds like you've implemented an O_TRUNC option into your open(), which is fine, but you're still going to need to provide the user with a separate trunc() function for times when a file has to be cut to a specific length.

You could send me your code back, I'd take a look at it, work permitting...

Hope it goes well

 

[itachi_uchiha]

You are a lucky man. I've implemented a full, standard FAT12/16/32 (with LFN support) for using in a commercial project of mine. It was meant to be used on an embedded platform. It's very well documented and has absolutely no dependencies on anything.

It's even thread safe and features file/directory locking (to provide concurrent access).
It might not be fully bug-free, but it's getting there (I'm using it ATM).
I also did not test it on anything other than FAT12 volumes, but it's designed to support FAT16/32.

You could use that as a reference if you like.

Additionally, look for a document named fatgen103.doc / .pdf from Microsoft, which seems like a half-arsed attempt of one of their engineers to document that (seems like he did at unwillingly at a gunpoint ).
There's plenty of information other than that. For volume information, you got the BPB (Boot Parameter Block) - look that up.

You can ask me whatever question you like, I'll do my best to help out.

My suggestion for you would be - start with the volume handling - mount (check partition type, get information, etc.), unmount (commit all cached data to the volume), read next cluster number off table, write next cluster to table.
Another suggestion would be implementing it over a generic Block Device implementation, makes debugging and porting really easy.
A block device essentially has read-from(), write-to(), close(), open() properties.

For example, initially I wrote it to work over a pseudo-Block Device which just used standard stdio (under Visual Studio) to access an image I made from a floppy drive.
Later I just substituted that with a block device which writes to a Flash chip, and also does caching.

It's a fun project. It's quite trivial unless you do that for a multiple access environment (multiple processes accessing the volume), then it's a pain, but I did that anyway...

hi @SamurAchzar
Currently i need to implement the FAT32 file system IP for SD inside the FPGA by using Vitis HLS,
So could you please provide any reference so that i can refer and implement IP ......
Regards,
Vinayak

 

[William Gaatjes]

Hi everybody,

Our operating systems class is working on a team project this semester-- building our own OS. My group is in charge of implementing the FAT file system. Trouble is, we have no idea where to begin on implementation.

We've been reading specs and general whitepapers on how FAT works, but we're lost as to how this might be implemented, even on a simplistic level, which is what we're aiming for. Boot sectors, FAT structures, etc....it's all new to us.

If anyone knows of some decent code or tutorials we can look over to get ideas on how to approach this I'd really appreciate it. Thanks!

Hi, i do not know how much freedom you have with selection of software. But for microcontrollers, there is an easy to use , ready to use library called fatfs.
I know it works, even with my "limited" programming skills compared to professionals, i got it to function. based on example code from somebody else to make an mp3 player reading directly from sd card, even usable with the windows compatible filesystem.
This all should get you get going. I am pretty sure of that.


Check out fatfs :

http://elm-chan.org/fsw/ff/

http://elm-chan.org/fsw/ff/doc/appnote.html

excerpt form site :
"
FatFs is a generic FAT/exFAT filesystem module for small embedded systems. The FatFs module is written in compliance with ANSI C (C89) and completely separated from the disk I/O layer. Therefore it is independent of the platform. It can be incorporated into small microcontrollers with limited resource, such as 8051, PIC, AVR, ARM, Z80, RX and etc. Also Petit FatFs module for tiny microcontrollers is available here.
Features

• DOS/Windows Compatible FAT/exFAT Filesystem.
• Platform Independent. Easy to port.
• Very Small Footprint for Program Code and Work Area.
• Various Configuration Options to Support for:
  • Long File Name in ANSI/OEM or Unicode.
  • exFAT Filesystem, 64-bit LBA and GPT for Huge Storages.
  • Thread Safe for RTOS.
  • Multiple Volumes. (Physical Drives and Partitions)
  • Variable Sector Size.
  • Multiple Code Pages Including DBCS.
  • Read-only, Optional APIs, I/O Buffer and etc...

"


And petitfs :

http://elm-chan.org/fsw/ff/00index_p.html

excerpt form site :
"
Petit FatFs is a sub-set of FatFs module for tiny 8-bit microcontrollers. It is written in compliance with ANSI C and completely separated from the disk I/O layer. It can be incorporated into the tiny microcontrollers with limited memory even if the RAM size is less than sector size. Also full featured FAT file system module is available here.
Features

• Very Small RAM Consumption (44 Bytes Work Area + Certain Stack).
• Very Small Code Size (2K-4K bytes).
• FAT12, FAT16 and FAT32.
• Single Volume and Single File.
• Streaming File Read.
• File Write Function with Some Restrictions.

"

The mp3 player reading from sd card :

ARM MP3/AAC Player – Mikrocontroller.net

www.mikrocontroller.net

 

[IronWing]

Today I learned that the Z80 chip is still in production. The current chip is built on a modern process and much smaller than the original but is basically the same design, almost fifty years later. Given the ancient age of the technology, naturally it is used in a TI calculator.

 

[William Gaatjes]

Today I learned that the Z80 chip is still in production. The current chip is built on a modern process and much smaller than the original but is basically the same design, almost fifty years later. Given the ancient age of the technology, naturally it is used in a TI calculator.

It seems pretty venerable. Thinking of the MSX era, The Turbo R had an Z80 derivative called R800 which was sort of a copy of a z800 (16 bit).