Superfetch + Readyboost explained. *LONG*

[n7]

OP, thanx for this; great info.

Originally posted by: rseiler
How intelligent is Superfetch exactly?

I was watching its behavior recently via the Reliability/Performance monitor, because I started noting my disk on solid red for egregiously long periods (this is a 2GB RAM system).

What was it interested in at the time? A 170MB Nero installer that I'd used recently; you'd think it would realize it was an installer and therefore not something you're apt to run again anytime soon, if ever (UAC can detect installers, why not Superfetch). Far worse, it was also reading an 8GB VM file. Yes, 8GB. And it was reading it (I could even see how many MB/s). And this was happening while I was otherwise using the system (but not that file)--so much for background operation.

So I ask given just these two examples how smart Superfetch actually is and why disabling it wouldn't be a net gain.

rseiler, how do you tell what Superfetch is doing?

I can open Reliability/Performance monitor, but i'm not sure how to see what you described.

Thanx.

 

[n7]

Originally posted by: n7
OP, thanx for this; great info.

Originally posted by: rseiler
How intelligent is Superfetch exactly?

I was watching its behavior recently via the Reliability/Performance monitor, because I started noting my disk on solid red for egregiously long periods (this is a 2GB RAM system).

What was it interested in at the time? A 170MB Nero installer that I'd used recently; you'd think it would realize it was an installer and therefore not something you're apt to run again anytime soon, if ever (UAC can detect installers, why not Superfetch). Far worse, it was also reading an 8GB VM file. Yes, 8GB. And it was reading it (I could even see how many MB/s). And this was happening while I was otherwise using the system (but not that file)--so much for background operation.

So I ask given just these two examples how smart Superfetch actually is and why disabling it wouldn't be a net gain.

rseiler, how do you tell what Superfetch is doing?

I can open Reliability/Performance monitor, but i'm not sure how to see what you described.

Thanx.

Bumped for answer to this.

I pretty much got 4 GB of RAM for Vista - since i do quite like it!

But i'd really like to be able to see what it's doing w/ my RAM.

I just got Ultimate 64 installed (was using 32-bit before), & initially, it was showing tons of RAM free.
Now it shows all but 46 MB cached or in use, but i'd like to find out on what.
Partially because loading UT2k4 has actually gotten slower, something i'm finding a tad annoying, since in theory, considering the amount i run the game, i'd've thought Superfetch mighta helped me out a little.

The really weird thing i just remembered while typing this is that i took out my 2 GB Readyboost when i got the 4 GB of RAM...so hmm...possible it was helping for loading UT2k4, but Superfetch doesn't?

Or are things a bit slower because my new 4 GB RAM is much slower than my former Team Xtreem 2 GB?

Questions, questions...

Any insight welcome.

 

[TallBill]

2) But it doesnt *have* to be external! Better than even the most well tuned USB stick would be an internal drive. A large, fast SATA flash drive would be good. Flash right on the motherboard, with direct memory access - even better. And they're coming *very* soon. They'll be tuned for random access, with less overhead since they won't need USB or encryption. And since they aren't going anywhere, they can cache across boots and standby. They'll put the best USB sticks to shame.

That is very exciting.

 

[Pabster]

Originally posted by: MikeyLSU
My current USB drive does not pass the test and can not be used for this. I'm very close to pulling the trigger on this one. I'm just not sure how much it will help things.

It would be the cheapest performance improvement you could give your machine. You can find a 2GB for next to nothing and 4GB are getting very cheap as well.

 

[Smilin]

Originally posted by: delco007
Hi Mark,
I loved the way you have explained the working of primary and secondary memory but it reminded me of a question which has been ringing at the back of my head for a looong time .
I have heard that these features were already present in a certain MAC os , if that is correct then wouldnt it be true that Microsoft has copied their features ? Also i would like to know if Microsoft can be credited for making a good Operating system or its just that they have copied features from here and there and Jumbled them together and named it their new Operating system ???

I would love to know the truth .

Bye for now
Guru .

If it was true wouldn't that mean..??? Go find out if it is true first. :roll:

I would not be surprised if apple is working on something similar to ready booste and superfetch. If they have it already I would be interested to know if it is superior or inferior.

If Apple has such a thing I would be very surprised if they came up with the idea first. Apple is spending like $534 million in research (2005 numbers I found). Last year Microsoft spent $6.5 billion (with a B) and filed 3,467 patents.

Heck Apple didn't even manage to fully design the iPod. Microsoft owns several patents used in the iPod interface.

As far as MS copying features from others I'm sure some of that occurs in both directions. However Vista was already in development when the very first OSX was released back in 2001. OSX was of course not developed from the ground up by Apple either. Apples OS's and hardware were falling further and further behind Wintel so they finally discarded the OS (and backwards compatibility as they've done several times) and designed a new OS based on FreeBSD (others work). To remedy the hardware situation they are now switching to "windows hardware" in the form of the x86.

If we should start accusing companies of copying others why don't we start by accusing Apple of stealing computer designs from Fisher Price and Playschool.



..Good Post by the way BD!

 

[bsobel]

Originally posted by: rseiler
How intelligent is Superfetch exactly?
I was watching its behavior recently via the Reliability/Performance monitor, because I started noting my disk on solid red for egregiously long periods (this is a 2GB RAM system).
What was it interested in at the time? A 170MB Nero installer that I'd used recently; you'd think it would realize it was an installer and therefore not something you're apt to run again anytime soon, if ever (UAC can detect installers, why not Superfetch). Far worse, it was also reading an 8GB VM file. Yes, 8GB. And it was reading it (I could even see how many MB/s). And this was happening while I was otherwise using the system (but not that file)--so much for background operation.
So I ask given just these two examples how smart Superfetch actually is and why disabling it wouldn't be a net gain.

Are you sure that was the prefetcher and not the indexer? If not, you make a good point about installers. One thing you didn't factor in however is that the prefetecher does all of it's IO at idle priority, so when it is loading it's not doing it at the defirment of other IO you need done 'NOW'.

 

[bsobel]

The really weird thing i just remembered while typing this is that i took out my 2 GB Readyboost when i got the 4 GB of RAM...so hmm...possible it was helping for loading UT2k4, but Superfetch doesn't?

Readyboost and prefetch work together, the 2bg stick was probably helping. Try with it back and see if the issue resolves...

 

[Jeff7181]

Originally posted by: n7
OP, thanx for this; great info.

Originally posted by: rseiler
How intelligent is Superfetch exactly?

I was watching its behavior recently via the Reliability/Performance monitor, because I started noting my disk on solid red for egregiously long periods (this is a 2GB RAM system).

What was it interested in at the time? A 170MB Nero installer that I'd used recently; you'd think it would realize it was an installer and therefore not something you're apt to run again anytime soon, if ever (UAC can detect installers, why not Superfetch). Far worse, it was also reading an 8GB VM file. Yes, 8GB. And it was reading it (I could even see how many MB/s). And this was happening while I was otherwise using the system (but not that file)--so much for background operation.

So I ask given just these two examples how smart Superfetch actually is and why disabling it wouldn't be a net gain.

rseiler, how do you tell what Superfetch is doing?

I can open Reliability/Performance monitor, but i'm not sure how to see what you described.

Thanx.

I had the same problem with Superfetch trying to cache a huge file used by World of Warcraft.

You can find out what it's doing using the Reliability/Performance monitor by expanding the section that shows hard drive usage and looking in the File column to see what file is being accessed by what process. I believe Superfetch will be an instance of svchost... just look for abnomally high disk reads.

 

[bsobel]

I had the same problem with Superfetch trying to cache a huge file used by World of Warcraft. You can find out what it's doing using the Reliability/Performance monitor by expanding the section that shows hard drive usage and looking in the File column to see what file is being accessed by what process. I believe Superfetch will be an instance of svchost... just look for abnomally high disk reads.

That is not neccisarily the best way to determine this, the usage could be from other high io apps like the indexer.

If it was prefetch, it sounds like it was doing what you'd want with WOW, no?

 

[Jeff7181]

Originally posted by: bsobel

I had the same problem with Superfetch trying to cache a huge file used by World of Warcraft. You can find out what it's doing using the Reliability/Performance monitor by expanding the section that shows hard drive usage and looking in the File column to see what file is being accessed by what process. I believe Superfetch will be an instance of svchost... just look for abnomally high disk reads.

That is not neccisarily the best way to determine this, the usage could be from other high io apps like the indexer.

If it was prefetch, it sounds like it was doing what you'd want with WOW, no?

Caching a 3+ GB file when I only have 2 GB of RAM is a bad idea.

The Indexer isn't called svchost in the reliablity/performance monitor, it's labeled SearchIndexer or something like that. It also wouldn't load an entire 3+ GB file into memory just to index it.

 

[n7]

Originally posted by: Jeff7181
You can find out what it's doing using the Reliability/Performance monitor by expanding the section that shows hard drive usage and looking in the File column to see what file is being accessed by what process. I believe Superfetch will be an instance of svchost... just look for abnomally high disk reads.

Apparently i'm an idiot, because i can't see anything there
http://img511.imageshack.us/img511/6338/reliabilityperformancemol2.jpg

Could you could explain how to get to see what you're referring to?

 

[cmdrdredd]

this is osomething I'm interested in monitoring. Just curious what particular things are being loaded up.

 

[Jeff7181]

Originally posted by: n7

Originally posted by: Jeff7181
You can find out what it's doing using the Reliability/Performance monitor by expanding the section that shows hard drive usage and looking in the File column to see what file is being accessed by what process. I believe Superfetch will be an instance of svchost... just look for abnomally high disk reads.

Apparently i'm an idiot, because i can't see anything there
http://img511.imageshack.us/img511/6338/reliabilityperformancemol2.jpg

Could you could explain how to get to see what you're referring to?

You need to provide admin credentials for the Reliability/Performance monitor to work.

 

[Jeff7181]

Here's an example of what I'm talking about. See the Disk Defragmenter creating disk activity? See the Search Indexer creating disk activity?

When Superfetch is creating disk activity the "Image" will be svchost.exe and the "File" will be what ever file it's caching.

 

[bsobel]

Caching a 3+ GB file when I only have 2 GB of RAM is a bad idea.

Superfetch isn't doing that... At best it would cache part of the file based on your free memory, it's in NO WAY paging out used bits to cache stuff you might use.

 

[HardWarrior]

:thumbsup: Very nice, BD!

 

[cmdrdredd]

today I got a message "Superfetch has stopped working" and my available physical memory went from 4GB to 0

What happened? Did windows eat my memory cause it felt like it?

 

[Jeff7181]

Originally posted by: bsobel

Caching a 3+ GB file when I only have 2 GB of RAM is a bad idea.

Superfetch isn't doing that... At best it would cache part of the file based on your free memory, it's in NO WAY paging out used bits to cache stuff you might use.

How can you be so sure Superfetch wasn't doing that? Obviously it wouldn't page anything out to make room for a larger cache (I never said it was)... but what other explanation would there be for over 1200 MB read from the 3.56 GB file, common.MPQ by the svchost.exe process?

 

[bsobel]

How can you be so sure Superfetch wasn't doing that? Obviously it wouldn't page anything out to make room for a larger cache (I never said it was)... but what other explanation would there be for over 1200 MB read from the 3.56 GB file, common.MPQ by the svchost.exe process?

You said superfetch was caching a 3gig file but now you say it was caching 1.2gig of a 3gig file. Big difference. If the file was cachable and the system io idle, I don't see what the problem is.

 

[rseiler]

Originally posted by: Jeff7181

You can find out what it's doing using the Reliability/Performance monitor by expanding the section that shows hard drive usage and looking in the File column to see what file is being accessed by what process. I believe Superfetch will be an instance of svchost... just look for abnomally high disk reads.

Yes, I figured it out by inference, since the indexer is identified as such in Reliability and Performance Monitor. Also, like I mentioned earlier, it often follows you around like a kitten: access that 8GB VM file, and not long after -- and it doesn't bother to wait until you've walked away from the computer -- superfetch will be caching it for...I still haven't figured out for what, exactly, or even how.

Finally, if you disable Superfetch, like I've done now, all this nonsense goes away. That's the best proof of all.

It's just too much fetching for me on a 2GB system. I shudder at what it would do with more memory.

It kinds of reminds of me of people who let defraggers run for hours to make their disks run faster.

 

[Smilin]

Originally posted by: cmdrdredd
today I got a message "Superfetch has stopped working" and my available physical memory went from 4GB to 0

What happened? Did windows eat my memory cause it felt like it?

When you have something crash all bets are off. Who knows? Fix the crash.

btw is that the exact text of the error? I'm not seeing a single support case has ever been opened on this nor are there any bugs. On the 'net there is one instance that appears to be hardware related and another that may or may not have been fixed by ATI drivers. I would wonder if that's not a symptom rather than a cause. ie physical memory going to zero from something causing superfetch to fail rather than the other way around.

 

[Jeff7181]

Originally posted by: bsobel

How can you be so sure Superfetch wasn't doing that? Obviously it wouldn't page anything out to make room for a larger cache (I never said it was)... but what other explanation would there be for over 1200 MB read from the 3.56 GB file, common.MPQ by the svchost.exe process?

You said superfetch was caching a 3gig file but now you say it was caching 1.2gig of a 3gig file. Big difference. If the file was cachable and the system io idle, I don't see what the problem is.

The problem is that you want to be right so badly that you won't acknowledge the problem, which is being experienced by multiple people. /shrug

 

[bsobel]

The problem is that you want to be right so badly that you won't acknowledge the problem, which is being experienced by multiple people. /shrug

You really haven't stated what the problem is. You claimed that superfetch was paging in a 3gig file on your 2gig system when asked for more details suddenly you think it's paging in a 1.2gig of that 3gig file. If the memory was free, I don't see what the issue is, SF wouldbe doing it's job. I'm happy to acknowledge the issue, once you state what it is.

 

[Smilin]

Originally posted by: Jeff7181
Here's an example of what I'm talking about. See the Disk Defragmenter creating disk activity? See the Search Indexer creating disk activity?

When Superfetch is creating disk activity the "Image" will be svchost.exe and the "File" will be what ever file it's caching.

Jeff I don't follow this.

There are several instances of svchost and each holds several services. It would be more complicated than that to figure out what hosted service is doing what. You can take a good guess depending on the file I guess.

That screenshot doesn't even show svchost though. It's got searchindexer highlighted which should be indexing your disk to speed up search results. I'm not sure this is at all related to superfetch. Also, the disk activity is like 60,000-70,000 Bytes per min. A modern drive will do 60,000,000 Bytes per *second* so whatever is going on it isn't much. Your defragger appers to be chugging away there. It's background though so if you decide to do something that will fall quickly.

Again, I'm just a bit confused.

 

[bsobel]

kitten: access that 8GB VM file, and not long after -- and it doesn't bother to wait until you've walked away from the computer -- superfetch will be caching it for...I still haven't figured out for what, exactly, or even how.

If you've accessed that VM file, it's already in the cache. If your seeing access to the prefetch directory that is the cache manager writing OUT the prefetch information for the file. That info is used by prefetch, superfetch, and the defragger.