Click here for the Pscience5 Home Page Click here for My Software page Click here for the Xtra Software page Click here for the Articles page Click here for the Lost & Found page Click here for the Macros page Click here for the Colour Icons page Click here for the Overlays page Click here for the EPOC Links page Click here for the Contact page Click here for the Pscience5 Forum Please click here to donate towards Pscience5's upkeep


How to partition a Compact Flash (CF) card into multiple drives

Article written: August, 2005

Article updated: March 2006

Download a copy of this article in zipped EPOC Word and PDF format here: CFPartition.zip  (164KB)


Introduction:


It had never even crossed my mind that it might be possible to partition a CF card into multiple partitions (essentially multiple drives - the same way that a standard hard disk can be partitioned to appear as different drives on your PC). That was until I saw this thread on PDAStreet. The author linked to his own article describing how he'd done it under Linux and then showed a screenshot from his netBook showing that EPOC supported a CF partitioned into 4 drives!

EPOC with 4 drives??


So why would you want to partition a CF into multiple drives for use on your Psion?:


Well, various reasons:-

Speed / usable capacity. As can be seen in my original CF Formatting article, there is a trade-off obtained in varying the Sectors/cluster size between the write speed of the CF and the usable capacity (because of the increase in minimum cluster size that results). This effect is especially noticeable on larger (say, >256MB) cards and increases as the drives get bigger still. Hence by being able to essentially split a drive into 4 'mini-drives', you get a benefit in speed/usable capacity for any given formatting. Keep the formatting the same as the original and, simply put, it gets faster!

Directory read speed. Even without the above, just splitting all your files and directories so that (say) you've got a quarter as many on a drive as before gives significant benefit in terms of speed when it comes to reading the drive structure. Most of us are used to the delay when we accidentally press Ctrl+Tab in the system screen on d: drive - imagine it being 4 times faster?!

Variable formatting optimisation. Because Atelier's EDU (I'm pleased to say) can also 'see' partitioned drives, it's possible to optimise each partition depending upon how you intend to use it. For example, one of my new partitioned drives is used solely for storing zipped applications in (for backup purposes). Since all the files are relatively large, concern about minimum cluster size is much less of an issue and it's an easier choice to make to optimise the formatting for speed. In other words, some partitions can be targeted for speed with others maximising the storage efficiency for small files.

Ease of navigation. The ability to jump between drives easily from the system screen makes navigation even easier if you split the usage of the drives in a logical fashion.

Maximising capacity. In theory, a Psion can 'see' CF cards up to 2GB in size. By increasing the number of partitions (effectively the number of disks) it can see by four, you increase the maximum available capacity it can use fourfold as well - to 8GB! In fact, for a 'serious' Series 7 / netBook power-user - who was prepared to have memory in both their machine's slots - this number would double to 16GB! I'll bet that the Psion engineers never envisaged the day when someone might want THAT much memory on their devices!

With the ever-falling prices of CF cards, it's a perfect excuse to upgrade!

Okay, so how did I do it?:


As I mentioned above, the idea's original author used Linux to partition his card. The reason he did this was because Windows 2000/XP will not allow you to partition a CF card natively. In fact, CF cards are manufactured with a bit set in them that defines them as 'removable media' so none of the standard 3rd-party partitioning programs will let you do it either. Apparently some of the CF manufacturers have software tools that allow you circumvent these settings but, try as I might, I wasn't able to find them anywhere on the Net.

From looking at the above thread on PDAStreet, it seems that some people have managed to use a DOS boot floppy disk on their system combined with setting their PC's bios to support DOS USB peripherals so that DOS could 'see' the CF when connected to a USB reader. I had two problems with this however: 1) of the 3 PC's I have regular access to, only one still has a floppy drive on it - the other two are CD-only laptops., and 2) when I actually tried this solution on my own home-grown machine with a floppy drive, it still couldn't actually 'see' the CF - despite me trying two different CF USB readers.

Really what I wanted was a solution where I didn't have to muck around with creating bootable DOS floppy disks (or CDs) that included the necessary tools for partitioning (FDisk, GDisk, Ranish Partition Manager, etc.). I even went as far as creating a bootable Linux CD which would fire up Knoppix on any PC with a view to using the necessary command line instructions in Linux to partition the CF. Although the learning curve on this was slightly daunting...

Eventually however, sheer pigheadedness paid off! It finally dawned on me that the one sure-fire way of making sure that DOS would be able to see the CF drive was if DOS itself could be booted from it. "So big deal", you say, "how am I gonna achieve that?!" Well, I chanced upon a public utility published by HP here (local copy here - 2.0MB) that will format and make a USB flashdrive bootable - and effectively a CF on the end of a USB adaptor is just a USB flashdrive. HP's utility was intended for use with their own devices but fortunately is generic and seems to work with just about any flash device on USB. You need to have the necessary DOS files on your PC in order to make a bootable drive but thoughtfully HP also provided these (local copy here - 295KB). In addition to this of course, you need a CF-USB reader. I have a desktop version (which I haven't tried here actually) but I actually did this work on a new XP laptop with a compact plug-in adapter. These can be picked up for less than 5 (US$9) very easily - online is probably cheapest.

Crucial USB<>Compact Flash adapter


The HP utility runs under windows and formats the CF card whilst (optionally) making it bootable too. The next thing you need to do is set the bios settings of your PC so that it will try to boot from a USB drive before the main disk. To complicate matters, it seems that different motherboard manufacturers (and hence different bios's) regard these types of drives in different ways. Some classify them as USB-ZIP (USB Zip drives), some as USB-FDD (USB Floppy drives), and some as USB-HDD (USB hard drives). So the thing to do is set the boot order such that the PC will try all these options before the standard built-in hard drive (usually called IDE0) if you can. Then re-boot your PC with the newly formatted/bootable CF plugged in via its adapter.

Assuming all goes well, your PC will very quickly boot into DOS with the standard C:> prompt - a quick check should verify that C:> is in fact your CF card. Now unplug the card and re-boot your machine into Windows. You need to add a few bits and pieces onto your CF drive so that when you boot from it again you've got the necessary tools to allow you to partition it and format it initially. A copy of FDisk.exe and Format.com from a full DOS installation (e.g. FreeDOS) would do the job perfectly well but I preferred to use something like Ranish Partition Manager (RPM) since a) it does both jobs in one utility, b) it's got a nicer - albeit slightly non-intuitive - interface, and c) it's free.

Once you've copied the necessary files onto your now-bootable CF, just re-boot your PC again so that it starts up in 'CF-DOS' and fire up your chosen utility.

Partitioning begins!:


The first thing I should probably say here is "BE CAREFUL!". Whilst the HP utility mentioned above is fairly safe insofar as it will only let you format a USB-connected drive, the tools you are about to play with now can effectively 'wipe' your PC if you're not careful which drive you partition/format. If you've followed the above procedure and are (say) using RPM as I did then it should be fairly obvious which drive is which. However, I will not be in the least bit sympathetic to anyone who contacts me saying "I just wiped my PC - I'm'a gonna kill you!" Caveat emptor.

If using RPM, make sure therefore that you're pointing at the right disk - the difference in disk sizes should make it immediately obvious. Next you need to decide how you're going to partition your CF card. It's not mandatory that you partition it into 4 - you could split it into 2 or 3 disks instead. I played around for some considerable time before I realised that EPOC only seems to be able to recognise primary partitions however - so don't waste your time trying to set up extended DOS partitions and the like; it seems it won't work.

Ranish Partition Manager (NB: being used on a standard disk drive - not a CF here)


Using RPM, the first thing you have to do is delete the existing partition by selecting it and hitting the Delete key (D'uh!). Next you decide how much space to allocate to to each of the partitions you want. I found that the easiest way of splitting the partition sizes was to use the Wizard, select FAT-16, and then work out the number of cylinders you need to give you the right ratio of space for your chosen partition. One tip: the cylinders count down from the maximum - not up from zero - in this Wizard. The first time I did this I just split the partitions evenly: 4x 512Mb each (approximately) on my 2GB card. I eventually came back and changed this but I'll explain that later. Once you've done this and nominally got your partition sizes sorted out, you need to order them correctly for the Master Boot Record (MBR) - move the cursor under the 'Row' heading and type the order that you want them in the MBR (1-4 assuming 4 partitions). You'll be pleased to hear that none of the changes you make at this point actually get committed to the CF card until you hit save - and then it'll flag the fact that you may not have formatted them all either. Oh and the quick-formatting option is fine to use as well - it saves a lot of sitting around!

Another piece of good news is that - whilst re-partitioning and re-formatting the card of course deletes its ability to boot from DOS - the HP Utility appears to be sufficiently robust that it can still reformat a card that you've messed up partitioning/formatting-wise. It saved my skin on a number of occasions when I was experimenting!

And that's pretty much it. Once you've partitioned and formatted the way you want, you can unplug the card and stick it in your Psion! And lo, there were 4 new drives. And he was greatly impressed.

Four new drives on my netBook!


Using your 'new' CF card:


With a Series 7 / netBook I'd probably recommend using a PCMCIA adapter to play with your newly partitioned/formatted card initially - simply because it'll make transferring files from an existing CF card so much easier. I'm pleased to be able to report that PsiWin can see all the new drives just fine and so of course, 5mx/etc. users can transfer files to/from their PC as normal and/or restore from a backup. Similarly, if you're doing this with an existing card that you just want the benefits of partitioning with, then a backup/restore would probably be your main option anyway.

Before you start putting any files onto your newly partitioned card, I'd suggest testing it to see how it performs. I've updated DiskBench so that it can now cope with all the drives up to K: drive for exactly this purpose - details and download on my software page. And if you don't know what I'm talking about then read my article on formatting Compact Flash cards.

DiskBench2 results...



The observant reader will note that there's one fewer drive measured here than shown in the 'new drives' screenshot above. That's because this is actually a set of measurements taken after I'd figured out how really I wanted to partition up the card (i.e. instead of 4x 512MBish) and formatted accordingly. What's interesting is that once I'd done a few formatting tests with Atelier's EDU (see my 'Lost & Found' collection for a download), I decided to format all the new drives the same way. In my case I opted for FAT16 with 64 Sectors per Cluster for each partitioned drive. This isn't very efficient from a space utility point of view but since I was upgrading from a 1GB card which still had about 200MB free on it to a new 2GB card, I felt that I could fairly extravagant with my space efficiency and blow it all on speed! The interesting bit is that the variances in speed measure above are therefore entirely due to the relative difference in size between the partitioned drives - just goes to show!

One of the first things I'd already decided to do - even before playing around with the partitioning and formatting - was how I was going to utilise the new drives (talk about spending your money before you've earned it!). The first obvious thing to do was to put the System folder (back) onto D: drive. That way, I wouldn't have to change any of the pointers/shortcuts I use in my preferred program-launcher (currently ExtraBars). And the other thing I always keep on my D: drive is a hidden copy of OS.img - lest the worst should ever happen! The remaining drives are split between a) a collection of backup zip files, b) the Pscience5.net website, and c) pretty much everything else - categorised simply as 'General'. You can of course name your new drives and hence on my machine they now appear as:


My new drives!  :).



A by-product of the fact that you can only create primary partition drives is that the drive letters jump up by 2. After D:, E: is reserved for the PCMCIA slot on a netBook anyway so it goes to F:, H:, and J:. I suppose that I could very easily use a utility like DriveMap, YFH Subst, View+, etc. to change the mapping of the drives to something more 'orderly' - but I haven't played around with that yet.

PsiWin a winner!:


Probably not something you'd expect to have seen written about Psion's backup software in years gone by. However, I've said it before and I'll say it again; PsiWin v2.3.3 running on Windows 2000 or XP is a pretty good piece of software. And it comes to the rescue with a partitioned CF card as well; having no problem at all seeing and backing up the new drives. The downside of course is that it's limited by it 115Kbps serial cable connection speed, Backing up about 800MB over that is a slow process! The good thing about PsiWin however is that you can do even a complete backup incrementally. In other words, it remembers what you've backed up so that if you decide to stop a backup half way through and then to resume it (say) the following day, it'll resume where it left off from (allowing for any other files that have changed in between). From previous experience I can say that I know that MicroSoft's ActiveSync won't let you do that. If you break it off half way through a full backup, it insists on starting from scratch again!

Other thoughts:


Whilst of course I've only tried this technique on my EPOC-based Psions, I imagine that it's possible that PDAs/Smartphones based on other OS's may also be able to see and use partitioned memory cards. I guess that the obvious first candidate to try in this respect would be Symbian derivatives - since they were originally based on EPOC.

The corollary from this of course is that the above technique is likely to work with some other memory card types - SD and MMC would be the obvious ones to try I think.

Other useful reading material:


http://www.aaltonen.us/archive/2004/03/01/tip-boot-from-usb-key-addendum/

http://fuzzymunchkin.dyndns.org:8080/tdot/usbkeyfob/index.php

http://www.weethet.nl/english/hardware_bootfromusbstick.php

http://www.marlow.dk/site.php/tech/usbkeys


CF Update - 6 months on:


Slightly more than 6 months have passed since I first explored and then wrote about this topic (August '05). And - I'm pleased to say - a large CF partitioned into '4 drives' is the heart of my netBook's memory. However, it's time I came clean about something. It's not the same large CF that I started this investigation with - and most of the period during the past 6 months have been spent using a smaller, non-partitioned card! Why? Read on...

The card I originally purchased for this evaluation was a 2GB DaneElec CF card. I've always worked on the principle that all memory cards are created more or less equal. Yes, there are differences in performance (speed) but that these can be compensated for by optimal formatting and the cost savings gained by buying cheaper cards more than justify the compromises made in terms of speed v's. usable capacity (again, see my original Compact Flash article for details). Ah but too much hubris; not enough humility methinks!

I started seeing some file errors after a few days of 'switching' to my '4-way' 2GB DaneElec card from my steadfastly reliable, single partition 1GB SanDisk Ultra II. However what confused me for quite a while initially was the fact that the errors only appeared to be occurring in my large PsiDat file that holds all the Pscience5 pages in it. Whilst slightly scary, the situation was (thankfully) always recoverable due to the large number of backups I tend to keep in various places. For quite some time I blamed PsiDat itself as being unable to cope with the very large file size that the whole of the Pscience5 site now required. With 20/20 hindsight I can see that that blame was almost certainly misplaced.

In reality what was almost certainly happening was that the card itself was starting to generate random bit write errors. The only reason that the PsiDat file was seen as repeatedly falling foul of these was a) that it was a file that was repeatedly and frequently (at least daily) being rewritten to the CF card, b) the file itself was large (almost 2MB), and c) errors in the file were almost immediately obvious (thanks to the file checking built into PsiDat) v's other less frequently used files. Hence statistically it's not really very surprising that 'random' events hit that file a lot.

My initial response (after recovering the data every time!) was to split the PsiDat database into 3 parts. This because I thought that it was the size of the database file itself which was causing 'instability' problems. Of course this didn't solve the problem - although it did make specific occurrences of the errors slightly less frequent simply because of the reduced file sizes. And splitting the PsiDat database has had a few useful spin-off results (which I won't bore anyone here with!). Nevertheless, it eventually became obvious - and judicious use of EDU's CheckDisk soon confirmed - that the CF media itself was causing problems.

In the event, I was able to quite quickly switch back to my unpartitioned 1GB card and recover the stability that I now craved (doing this sort of investigative stuff soaks up free time like a sponge!). It also made me pause for thought. What if, just perhaps, it was the partitioning of the CF card that was causing the problem? Well, I still had a reasonable amount of free space on my 1GB card - so figuratively speaking I just sat down and waited to see what happened. Fortunately (for everyone concerned!), all the email and feedback I've had back from everyone who's used this technique has been positive. Whew! The conclusion therefore was inescapable - I'd had a (semi-)dud card.

Having waited almost 6 months, the one good thing to report was that CF prices had continued to fall. I was able to buy a high-quality 2GB SanDisk Ultra II for almost the same price I paid for a (seemingly not so high-quality!) DaneElec card a few months earlier. I followed my own instructions (I printed them out - just to make sure that I understood myself! <grin>) and sure enough everything worked as it was supposed to. I ended up with 4 partitions all considerably faster than anything I'd used previously.

Well, I've been using my '2nd attempt' card for two weeks - with frequent tests by CheckDisk - and I'm pleased to report that it's been working flawlessly. So it would seem that I need to revise my previous assumptions about Compact Flash cards with a couple of conclusions based on this somewhat cautionary tale:

1) Compact Flash card reliability is not necessarily binary (working / not working) - there are degrees of reliability.
2) The old adage in life applies: you get what you pay for!

Oh, and in case anyone is wondering what I did with the old 2GB DaneElec card? Well, it'd had a few weeks of use before I realised that the card had problems so I didn't really feel that I could return it for a refund. No, it's been pensioned off for part-time service in my Canon Ixus camera - which appears to be somewhat more tolerant of occasional bit errors than my mission-critical Psion activities! MG.


^ Top of Page

< Previous Page

<< Pscience5 Home Page

Saturday, 1 April 2006