Question Can't the industry clear up the confusion over units?

[pmv]

After struggling to work out which folders would fit on a memory card, and then stumbling on this post, was wondering why the terminology is still so inconsistent, despite the supposed introduction of terms like Kibibyte?

I mean, they introduced the different terms to clean up the confusion, but, as that post says, everyone in the industry just carried on using the existing Kb,Mb,Gb,Tb,etc while the drive and memory card manufacturers, it seems, still use the terms to mean the decimal version. They can get away with it because even under the new terminology regime they are technically correct, so presumably can't be accused of false-advertising, and it's _everyone else_ who is "wrong" by not using the binary terms (and they can't be forced to change because they aren't gaining any unfair advantage by using those terms to denote the larger units).

Thus the confusion continues. Seems to me that what should have happened was to change the term for the "decimal" units, as then the storage manufacturers could have been legally obligated to use those terms.

That said, I'm not at all clear on whether the fact that hard-drives and memory cards come out with less than the stated capacity is due to the difference between binary and decimal units or is just down to space lost in formatting.

 

[yottabit]

It is the decimal vs binary that leads to the confusion. Windows incorrectly lists drive capacity in TB instead of TiB.

MacOS X does use the decimal system so the reported drive size matched the physical disk size apparently. Looks Ubuntu is clear about it too

Warum zeigt meine Festplatte eine geringere Speicherkapazität als auf dem Festplatten-Etikett angegeben? | Support Seagate US

www.seagate.com

UnitsPolicy - Ubuntu Wiki

I’m not sure what great change it would provide other then helping filter past dumb 1 star reviews on Amazon of people who don’t know how this stuff works and review bomb a great drive when it doesn’t show up with the capacity they wanted

Another issue is the block size when formatting, with the disparity of “size” vs “size on disk”, if you happen to be storing tons of small files the block size starts to really matter.

 

[Seba]

After struggling to work out which folders would fit on a memory card

Normally the used space and the free space are displayed using the same units, so for your purpose the 1000/1024 ambiguity should not matter at all. As long as you do not use the capacity from the printed labels, but the one reported by the software (Windows Explorer or whatever).

 

[pmv]

Normally the used space and the free space are displayed using the same units, so for your purpose the 1000/1024 ambiguity should not matter at all. As long as you do not use the capacity from the printed labels, but the one reported by the software (Windows Explorer or whatever).

Yeah, but it was a question of knowing what I needed to buy, and the labels and advertising seems to be based(as far as I can tell now) on the decimal units. Apart from everything else, only belatedly realised that cards marked as 1Tb are not equivalent to two cards each marked as 512Gb. It seems odd that, given they seem to use the decimal/SI terms, that they even sell cards with 256Gb, 512Gb, etc rather than just going with 250Gb and 500Gb, as it seems the range of cards sold goes 128, 256, 512....1000.

 

[pmv]

It is the decimal vs binary that leads to the confusion. Windows incorrectly lists drive capacity in TB instead of TiB.

The microsoft blog article I linked to in my post gives a justification for that - namely that almost nobody in the industry uses the 'correct' terms, so MS reckon it would just confuse people if they did the 'right' thing.

Partly my issue was wondering whether one can exceed the FAT32 2Tb limit - seems that you can, to a degree, because it's really a 2Tib limit, i.e. about 2.2TB.
(That's leaving aside my further confusion as to whether that limit is really a FAT32 limit as such or if it's rather something to do with how MBR works)

 

[mikeymikec]

The microsoft blog article I linked to in my post gives a justification for that - namely that almost nobody in the industry uses the 'correct' terms, so MS reckon it would just confuse people if they did the 'right' thing.

The method of measuring displays changed at the same time as the move from CRT to LCD, so yes it is possible that the storage industry could change, but they won't.

IMO the how mebibyte (MiB) business is just a poor workaround for the problem caused by the storage industry, which is why I don't buy into it.

 

[pmv]

The method of measuring displays changed at the same time as the move from CRT to LCD, so yes it is possible that the storage industry could change, but they won't.

IMO the how mebibyte (MiB) business is just a poor workaround for the problem caused by the storage industry, which is why I don't buy into it.

I assume the justification the storage people would use is that the K, M, G, etc prefixes are existing SI unit prefixes and for everything else (distance, volume, etc) they use the factors of 1000, thus they are just following established standard practice for everything outside of computing.

The computer industry kind-of misappropriated those prefixes very early on, and the attempt to correct that with the new prefixes doesn't seem to have worked, because few people use them.

I'm belabouring the point, but it's bugging me because I've recently been faffing about with memory cards a lot (particularly combining multiple ones) and this caused a lot of confusion, and I've only just worked out what the issue was. Funny that while I was aware Gb != Gib, I forgot that more discrepancies appeared once you got on to Tb. Really seems sneaky of the memory card sector to go, in terms of capacity in Gb, 16, 32, 64, 128, 256, 512...1000 (the last labelled as "1Tb").

Though with all the fakes out there I suppose sneakiness is built-in to that sector. Have seen cards labelled as "982Gb" - no idea what's going on there, have assumed those were fakes.

 

[yottabit]

The microsoft blog article I linked to in my post gives a justification for that - namely that almost nobody in the industry uses the 'correct' terms, so MS reckon it would just confuse people if they did the 'right' thing.

Partly my issue was wondering whether one can exceed the FAT32 2Tb limit - seems that you can, to a degree, because it's really a 2Tib limit, i.e. about 2.2TB.
(That's leaving aside my further confusion as to whether that limit is really a FAT32 limit as such or if it's rather something to do with how MBR works)

“nobody in the industry” like MacOS and Linux? Who both display it correctly?

Edit: that article is from 15 years ago and it seems other OS have gotten with the times since then. I don’t think there’s a great justification for Windows sticking with incorrect terminology in this day and age

 

[pmv]

“nobody in the industry” like MacOS and Linux? Who both display it correctly?

Well, just relaying the MS guy's argument, not necessarily agreeing with it. I mean, one could say it's a bit of a circular argument, as to a degree Microsoft _are_ the industry, they set the 'norms'.

But, for example, memory manufacturers don't seem to describe their sticks of RAM with the Kibi, Mebi, Gibi Tebi, etc terms, as far as I have seen.

 

[yottabit]

Maybe in Win12 we’ll be able to ask Cortana BingGPT AI how big our disks are in kibibytes. At least if you pay for the annual subscription DLC SaaS package

 

[mikeymikec]

I assume the justification the storage people would use is that the K, M, G, etc prefixes are existing SI unit prefixes and for everything else (distance, volume, etc) they use the factors of 1000, thus they are just following established standard practice for everything outside of computing.

Seagate's own KB article uses the word "marketed", which aligns with my cynical presumption that it was/is entirely for self-serving ends.

 

[fzabkar]

On the subject of decimal and binary kilo/mega/giga/terabytes, one really needs to understand the historical background. I grew up with memory chips and storage devices during the late 1970s and early 1980s. RAM chips were, and still are, accessed via data and address buses. These were almost always binary in nature. The address bus was either 4-bit or 8-bit or 16-bit and so on. The data bus was 1-bit or 4-bit or 8-bit, etc. Therefore the capacities of memory chips were nearly always expressed in powers of 2.

OTOH, hard drives and floppy diskettes were assigned all sorts of capacities which were never usually binary in nature. For example, a 3.5" floppy diskette has two sides, 80 tracks, 18 sectors per track, for a total of 2880 sectors. The sector size is 512 bytes, which is a power of 2. The total capacity is 1474560 bytes which is 1.40625 MiB. So where does the 1.44MB figure come from?

Typical 10MB and 20MB MFM HDDs of the 1980s had 17 sectors per track. Here is a 42MB Miniscribe HDD:

https://stason.org/TULARC/pc/hard-drives-hdd/miniscribe/M3650-42MB-5-25-HH-MFM-ST506.html

The CHS geometry was 809 cylinders, 6 heads, 17 sectors/track, 512 bytes/sector. The capacity was 42249216 bytes which is 42.24MB or 40.29MiB

Now how should the manufacturer specify the capacity? In short, memory capacity has historically been binary while storage capacity has been decimal.

Moreover, today we have SSDs which add a new twist to the capacity issue. Being memory devices, the NAND arrays will have a binary capacity, but part of this capacity will be reserved for firmware and overprovisioning. For example, a 1TB SSD will actually have 1TiB of memory, but 9% of it will be set aside for internal use. Creating an SSD with 1TiB of usable capacity would therefore be logistically impossible.

Furthermore, we now have a capacity standard that is set by IDEMA. In fact, you will nearly always be getting more than the advertised capacity in your storage device, whatever it may be.

https://idema.org/wp-content/downloads/2169.pdf

 

[mikeymikec]

The CHS geometry was 809 cylinders, 6 heads, 17 sectors/track, 512 bytes/sector. The capacity was 42249216 bytes which is 42.24MB or 40.29MiB

That does not seem like a particularly challenging problem: Round it down to 40. The rest can be part of the reserve.

 

[pmv]

Seagate's own KB article uses the word "marketed", which aligns with my cynical presumption that it was/is entirely for self-serving ends.

It can be both. Competition would tend to incentize using the method that gives a bigger number - because your competitors are probably going to do that. But there's still the justifying logic at the back of your mind of deciding it's fine because you are just following 'standard practice' for every other domain than IT.

@fzabkar probably outlines correctly why there was no internal logic driving storage makers to use the powers-of-two system (as opposed to makers of dynamic memory - one could argue they were the ones who caused the problem by using those long-existing prefixes slightly-differently from everyone else instead of coming up with kibi and mebi, and so on, right from the start), though I still think someone should have knocked everyone's heads together early on and agreed a single standard terminology for all parts of the IT industry, given that ultimately they are all dealing with the same stuff - 'information'.

 

[bba-tcg]

It does seem odd to belabor this point while using "b" in a place where it's obvious the "B" is the correct letter in context. Not trying to pick nits so much as to point out the irony.

 

[fzabkar]

Apart from everything else, only belatedly realised that cards marked as 1Tb are not equivalent to two cards each marked as 512Gb. It seems odd that, given they seem to use the decimal/SI terms, that they even sell cards with 256Gb, 512Gb, etc rather than just going with 250Gb and 500Gb, as it seems the range of cards sold goes 128, 256, 512....1000.

What is stranger still is that the IDEMA capacity of two 500GB drives exceeds the IDEMA capacity of a 1TB drive by 21168 sectors. This means that you can't clone a 2 x 500GB RAID 0 onto a single 1TB drive, for example.

 

[manly]

Moreover, today we have SSDs which add a new twist to the capacity issue. Being memory devices, the NAND arrays will have a binary capacity, but part of this capacity will be reserved for firmware and overprovisioning. For example, a 1TB SSD will actually have 1TiB of memory, but 9% of it will be set aside for internal use. Creating an SSD with 1TiB of usable capacity would therefore be logistically impossible.

I've never paid much attention, but I've never seen this applied or said about SSDs before. Some quick Googling didn't support the claim. In the following thread, one user adamantly states what you just did but doesn't make a compelling case IMO:

How much SSD capacity is lost in formatting?

Say I buy an SSD that is spec'd at 500GB. What should I expect the 'actual (available) capacity' of the drive to be after Windows formats it?

www.wilderssecurity.com

We know that a 1 TiB HDD formats to about 931GB (per above thread). If you're correct, the OS would report approx 848 GB free space (after deducting another 9%).

Most of us are aware of SSD overprovisioning, but that isn't directly "lost" or viewable to the end user. Do you have a reference for this claim?
I believe the overprovisioning is invisible to end users, as it is extra cells on top of the "advertised" capacity. Different controllers use different algorithms so the actual amount of overprovisioning is opaque.

For example, this guy formatted a 512GiB SSD and the OS displays 476GB. As expected:

[SOLVED] - After formatting SSD lost space

Just as the title says: After formatting SSD lost space Here's the story: Had my Win10 OS on my Samsung 960 Pro 512GB. Something went wrong so I had to format the SSD. Now that I'm ready to re-install the OS on the SSD I notice that the space is dramatically lower than it should be. (Probably...

forums.tomshardware.com

It does seem odd to belabor this point while using "b" in a place where it's obvious the "B" is the correct letter in context. Not trying to pick nits so much as to point out the irony.

Kilo should be a lowercase k too.

 

[fzabkar]

I've never paid much attention, but I've never seen this applied or said about SSDs before. Some quick Googling didn't support the claim. In the following thread, one user adamantly states what you just did but doesn't make a compelling case IMO:

How much SSD capacity is lost in formatting?

Say I buy an SSD that is spec'd at 500GB. What should I expect the 'actual (available) capacity' of the drive to be after Windows formats it?

www.wilderssecurity.com

We know that a 1 TiB HDD formats to about 931GB (per above thread). If you're correct, the OS would report approx 848 GB free space (after deducting another 9%).

You are confusing TiB with TB.

1TB = 931 GiB

1TiB = 1024 GiB

 

[manly]

You are confusing TiB with TB.

1TB = 931 GiB

1TiB = 1024 GiB

Oops, I reversed the two abbreviations. Sorry for the confusion.

I carefully re-read your post and I think I understand what you're saying. All SSDs do overprovisioning; while it's likely to be around 7-10%, it varies by the controller. IIRC enterprise SSDs overprovision more than consumer SSDs.

This really isn't a twist to the capacity issue since you still get what's "advertised," which are in the customary decimal units. I had misunderstood what you had written, and then confused it further with bad abbreviations.

 

[bba-tcg]

Kilo should be a lowercase k too.

Indeed.

 

[fzabkar]

This really isn't a twist to the capacity issue since you still get what's "advertised," which are in the customary decimal units.

By "twist", I meant that a manufacturer could not make a 1TiB SSD, at least not without a convoluted design. Therefore, those people who insist on getting a 1TiB SSD instead of 1TB would be complaining in vain.

 

[kschendel]

You want an industry that conflates transfer rates (MT/s) and clock speeds (MHz) to do something rational about storage size units? I fear you're an optimist.

 

[pmv]

It does seem odd to belabor this point while using "b" in a place where it's obvious the "B" is the correct letter in context. Not trying to pick nits so much as to point out the irony.

Not sure I see irony, given I'm not running any part of the IT industry or engaged in product marketing. Thanks for the reminder though, I was sloppy there, and given the audience should try and get that stuff right (now you mention it I recall 40+ years ago doing computer studies at school explicitly learning about 'b' for bits vs 'B' for bytes, but - probably unlike most on here, I assume - I don't really deal with the topic on a regular basis so don't always remember...Probably the same reason why it took me a while to realise the "1TB" card wasn't the same capacity as two 512GB cards).

Still puzzled why there are multiple cards advertised as "982GB" capacity. Given the reviews and the nothing-you've-ever-heard-of brand names, I'm assuming they are fakes, but why would they draw attention to the fakeness by making them non-standard sizes? It's like they are not even trying.

 

[bba-tcg]

Not sure I see irony, given I'm not running any part of the IT industry or engaged in product marketing. Thanks for the reminder though, I was sloppy there, and given the audience should try and get that stuff right (now you mention it I recall 40+ years ago doing computer studies at school explicitly learning about 'b' for bits vs 'B' for bytes, but - probably unlike most on here, I assume - I don't really deal with the topic on a regular basis so don't always remember...).

The irony, at least from my viewpoint. which is where I would see it from, is wanting an entire industry to normalize on a definition for something while using example language that doesn't conform to an already normalized definition. E.g., bits vs bytes.

It's just one of my (many) idiosyncrasies and should generally be ignored. I use a program called "Old Timer's Temp File Cleaner", which does an excellent job, but at the end it'll say something like 500 mb cleaned. Millibits? Really?

But. I do agree that it would be nice if there was just one standard for designating storage capacity in the PC industry.